Memory caching. The human edition

One of the things I don’t “know” are how many ounces are in a litre. It is something that I don’t use on a regular basis, and is easily accessible via a quick web search. Perhaps a better question is “should this be something that I know off the top of my head?”. What do we mean by “knowing” in the first place? Is it enough to know where to find the knowledge? This post is about how I like to think of our human brains working in cached layers to deal with information overload.

Tier 0: When it’s in our head

0-1 seconds retrieval

Your birthday operates in Tier 0 space – you don’t have to pull out your drivers license to be able to tell someone when asked what your birthday is. We take this for granted, but what is it about that particular fact that makes it a permanent resident in this tier? And how does our brain work within this tier?

Scientists break down memory into multiple different types, and the two relevant ones for this post are described below. Each of these different types helps our brain meet a particular purpose:

• Episodic Memory: What we probably think of as memory most often. This is kind of like rewinding a tape in our head and going back into a previous point in time. It is relevant for the idea of memory caching because we may be able to remember roughly where we were or what we were doing when we learned of a particular fact, and if we can retrieve the fact through Tier 1 or Tier 2.
• Semantic Memory: This is our ability to remember facts and other tidbits of information. In this article, this is what I am referring to when I say that something can directly live in Tier 0. We can retrieve the fact or piece of information directly from our head with no external references.

I like this quote from the semantic memory Wikipedia page to help distinguish between the two:

For instance, semantic memory might contain information about what a cat is, whereas episodic memory might contain a specific memory of stroking a particular cat.

Often times we will want to store something in our Tier 0 memory through force of will. If we have an exam coming up, we are highly incentivized to be able to recall the facts that will be tested. Yet, research shows that even monetary incentives cannot help us remember things more effectively. Memory is finicky and we still don’t understand everything about it, including the factors that make some things more memorable than others.

Luckily, we now have digital systems to help us out, but they aren’t always as organized as we think they are. A survey conducted by Adobe in 2023 found that “nearly 3 out of 4 employees say poor digital organization interferes with their ability to work effectively”. If only everything could live in Tier 0!

Tier 1: When we know exactly where it is

Less than 1 minute retrieval

This is the realm of the notebooks, the documentation sites, and the todo items. To know whether something exists in Tier 1, if someone were to ask you “I need to know how to do process XYZ” would you be able to say “Ah yes, one second” and go to a specific place to retrieve the information?

Some examples from my own life that I use frequently:

1. How to startup the ssh-agent with a key, I always know to go directly to my ssh page and scroll to the bottom.
2. The documentation on read_csv within Pandas can be found here.

The speed at which the information is retrieved is directly proportional to the time that it takes to get to that resource. With a digital device and a hyperlink, it’s the time to navigate to the site. If it’s a notebook, it is the time that it takes to flip to a particular page. Not instantaneous, but pretty quick.

Crucially, I would argue that we need a pointer stored in Tier 0 to tell us where to move in Tier 1. In my head I have a very small piece of semantic memory that says “SSH Agent information is stored at notes.markpitblado.me”. Or “all passwords are stored in this password manager”. Through this, we are able to compress the information we store in Tier 0; we don’t have to remember everything, we just have to remember where everything is.

Having fewer places where information is stored definitely helps. Another is minimizing the friction that it takes to get something from Tier 1 back into Tier 0, ideally it should be fewer than three clicks. If it is more, we will skip straight over Tier 1 into Tier 2.

Tier 2: When we know the process to determine where to find it

5-10 minutes retrieval

The best example for this I can think of is knowing that you searched for something online in the past week. You know that you can go back into your search history and begin the process to get back to the exact website, but you don’t know on which day in particular. It will take some searching, but it is slightly faster than just searching the web again (Tier 3).

A lot of the recent product launches, especially featuring AI, have focused on making Tier 2 easier. Take microsoft recall, it seeks to consolidate the entirety of Tier 2 into a single place. A security nightmare, but effective at reducing the amount of places one has to look.

Tier 3

10+ minutes retrieval

This the state that you never want to hit, but ultimately has to used for some things. Bits of information will naturally drift to this tier. Instead of seeing it as a bad thing, or a limitation of the human condition, I like to think of it as a positive; Tier 3 makes space for everything else to function properly. We can’t have the entirety of the world’s knowledge one second away.

Conclusion

We cannot remember everything, but we can strategically pick what should be in Tier 1 and Tier 2, and how quickly we can retrieve that information back into the context in which we are working. As I was writing this, Linux After Dark did a great episode that got across many of the ideas that I am trying to convey here. If you have reached the end of this piece and have some more spare time, I would highly recommend going and giving that a listen. I think that the human memory cache approach can fit in quite nicely.