What Is Context, Really? (And Why Your AI Keeps Forgetting Things)

Chapter 1: What Is Context, Really? (And Why Your AI Keeps Forgetting Things)

Imagine you are fifteen minutes into a chat with a customer support bot. You typed in your name, your order number, the exact phrase "my package never arrived," and a small novel about how you have already tried refreshing the tracking page three times. The bot is polite. It apologizes. It asks for your name. You give it. It asks for your order number. You give it. It asks what is wrong. You tell it, again, that your package never arrived. The bot apologizes once more, and then — with a kind of cheerful amnesia — asks for your name.

It feels broken. It feels rude. It feels, honestly, a little insulting. You, a paying customer, are doing the AI's job for it, and it is still forgetting.

Here is the thing: in the vast majority of cases, that bot is not stupid. The model behind it might be the same model that can write a sonnet about your cat, debug a regex, and explain quantum entanglement in three paragraphs. The problem is not intelligence. The problem is that the bot has lost its *context*. It literally cannot see what you just said three messages ago, even though that information is sitting right there in its conversation history, right next to it, in a place it should be able to read.

If you have ever built anything with an AI, you have hit a version of this. If you have ever used anything built with an AI, you have hit a version of this. It is the single most common failure mode in the entire field, and it is also the most preventable — once you understand what is actually going on.

That understanding is what this chapter is for. By the end of it, you will have a precise, working definition of "context" that you can carry through the rest of this book, plus a mental model that turns the entire field of context engineering into a problem you already know how to think about: packing a backpack.

1.1 The Word "Context" Is Doing Too Much Work

Let's start with the word itself, because the word is the first trap.

In everyday English, "context" is a fuzzy, generous word. We use it to mean almost anything that surrounds a thing and helps make sense of it. The context of a conversation is the room you're in, the mood you're in, what you said yesterday, the news from last week, and possibly what you ate for breakfast. The context of a sentence is the paragraph around it. The context of a historical event is the century it happened in. When someone says, "you don't have the full context," they mean: there is a vast, invisible ocean of relevant information that you're not seeing.

This is the human sense of context, and it is wonderfully rich. A human being, in a normal day, draws effortlessly on decades of memory, current sensory input, social cues, language, and prior conversation. We do not even notice we are doing it. We just do it. Context, for a person, is more like a fog than a list — soft at the edges, always present, mostly invisible.

Now here is the trap. When AI engineers say "context," they do not mean the fog. They mean something much narrower, much more specific, and much more annoying. They mean: *the exact, finite, packaged bundle of information that the model is allowed to look at, right now, in order to produce its next response.*

That's it. That is the whole technical meaning. The model does not have access to your mood. It does not have access to last week's news. It does not have access to its own past conversations, or your past conversations with it, or the room you are sitting in, or the click you just made on a different tab. It has the bundle. The bundle is what it sees. Outside the bundle is darkness.

Once you internalize this, the field of AI engineering starts to make a lot more sense. All those mysterious techniques — RAG, prompt engineering, agents, long-context models, tool use, memory systems — they are all, in the end, different ways of managing what goes into the bundle. They are all variations on the same theme: how do we get the *right* information into the model's view, in the *right* order, in the *right* amount, at the *right* moment?

Context engineering is the discipline of answering that question. And the reason it is suddenly one of the most important skills in software is that the bundle is small. Embarrassingly small. Comically small. Small enough that a human being could read it in about a minute, even when the model is supposedly using its "long" context. We'll get to the exact size in the next chapter. For now, just know: the bundle is small, and the moment you understand that, a huge amount of AI weirdness starts to make sense.

1.2 What Context Means for Humans (A Quick Detour)

Before we dive into the AI side, it is worth lingering for a moment on the human side, because the human version of context is what we are trying to approximate. Understanding the human version well is what lets you build AI that feels like a competent human instead of a forgetful goldfish.

A human being in a conversation is doing at least four different things with context, all at once, without thinking about it.

1. Memory of what just happened. A minute ago, you said your name was Sam. The human in front of you remembers this. Not because they wrote it down — because short-term memory in humans is good at exactly this kind of thing. They remember the last few minutes of conversation, the last few topics, the last few emotional beats.

2. Long-term knowledge. You mentioned quantum entanglement and the human knew what it was. That knowledge is not in the room. It is not in the conversation. It is in decades of accumulated experience, education, and reading. It is part of who they are.

3. Sensory environment. You are in a coffee shop, the barista is grinding beans, your friend can hear all of it, and that sound is part of how they are tracking the world. They might not be paying attention to the grind, but it is part of the context of being there.

4. Shared assumptions and social context. When you say "the usual," the barista knows what you mean. When you say "we'll talk later," your colleague knows not to ask when. Humans carry around enormous amounts of shared, unspoken, social context. It is the air we breathe in conversation.

Now, when an AI engineer says "context," they have to be very specific about which of these four things they mean — because the model can only do the first one reliably, can sort of do the second one, and basically cannot do the third or fourth at all. The model has no ears. It cannot see your face. It does not know it is in a coffee shop. It has no shared social background with you. If you say "the usual," it has no idea what that means.

This is why "context" in AI is a more limited and more deliberate concept than "context" in human life. We are not trying to give the model a human's full context. That is impossible. We are trying to give it a *carefully selected* slice of relevant information, and we are doing this in a way that the model can actually use.

That last clause is the one most people miss. Just because you can give the model information does not mean the model can use it. A model given too much information, or the wrong information, or information in the wrong order, will produce outputs that are confidently wrong. The bundle has to be designed. That is what the engineering is for.

1.3 Mapping Human Context Onto AI Context

Let's draw an explicit map between the four human capacities and what an AI model has. This map is the foundation for the rest of the book, and it is worth slowing down on.

Memory of what just happened → Conversation history (the prior turns). This is the part where the AI is most like us, and also the part where it fails most spectacularly. A model *does* see the previous messages in a conversation. They are right there in the bundle. If the chatbot forgot your name, it is because the previous messages were either not in the bundle, or were in the