Introduction to Context Engineering: The New Discipline Behind Reliable LLM Applications

Introduction to Context Engineering: The New Discipline Behind Reliable LLM Applications

Most developers who start building with large language models begin the same way: they open a chat interface, type a question, read the answer, and walk away believing the model simply "knows" things. That intuition is harmless during casual experimentation, but it collapses the moment a real product requirement enters the picture. A real LLM application is not a conversation with a knowledgeable assistant. It is a pipeline of decisions about what information the model should see, in what form, in what order, and with what guarantees about cost, latency, and accuracy. The discipline that governs those decisions is called context engineering, and it is the difference between a demo that impresses in a meeting and a system that survives contact with real users.

This chapter introduces context engineering as a distinct engineering practice, separates it from the more familiar discipline of prompt engineering, and walks through the architecture of a real application. The goal is not to teach you a specific framework, vendor, or model. The goal is to give you a mental model of the entire information pipeline that surrounds an LLM call, so that every decision you make later—from choosing a chunk size to designing a re-ranker to deciding what to put in a system prompt—fits into a coherent picture rather than feeling like an isolated trick.

The Problem with Prompts-Only Thinking

Prompt engineering captured the imagination of the LLM community in 2021 and 2022, when the dominant interaction pattern was a single user typing a single query into a chat box. In that world, the quality of the response was almost entirely a function of the prompt. A better instruction produced a better answer. Few-shot examples, role assignments, and chain-of-thought scaffolding were the levers that mattered, and the community developed an impressive vocabulary for describing them. Books were written, conferences were organized, and entire job categories were created around the idea that the prompt was the product.

That model of LLM usage is now a small minority of what is actually built and shipped. The dominant pattern in 2024 and 2025 is agentic and retrieval-augmented systems: applications that read documents, query databases, call external APIs, maintain long-running conversations, and orchestrate multiple model invocations to complete a task. In these systems, the prompt is one component among many, and it is often not the most important one. What matters more is the context—the structured body of information that surrounds the prompt and conditions the model's response.

Consider three scenarios that look superficially similar but require very different engineering:

1. A user asks a one-off question such as "Explain quantum entanglement to a high school student." The right answer is largely a function of the prompt. There is no retrieval, no memory, no external state. Prompt engineering is sufficient.

2. A user asks a customer-support chatbot "What is your return policy for opened electronics?" A correct answer requires retrieving the relevant section of a return policy document, knowing which region the user is in, knowing whether the product category is restricted, and remembering the user's previous interactions in the session. The prompt is now one ingredient in a recipe.

3. A user asks an AI coding assistant "Refactor the authentication module to use our new error-handling pattern." A correct answer requires retrieving the relevant code files, knowing the user's coding conventions, knowing what the new error-handling pattern looks like, and tracking which files have already been edited in this session. The prompt is now a small visible part of a much larger information system.

The first scenario is what prompt engineering was designed for. The second and third are what context engineering is designed for. Conflating them is the most common mistake beginners make, and it produces systems that look reasonable in isolation but fail unpredictably when faced with multi-turn dialogue, large documents, evolving knowledge, or strict operational constraints.

The deeper issue is that LLMs are stateless functions with respect to anything not present in their input. They do not remember previous conversations, do not know what documents exist in your company, do not know what tools are available, and do not know what the user has done in the session. Every piece of information the model uses to produce its response must be assembled and placed in the input. That assembly is the job of context engineering, and it is much harder than it sounds because the model's input is constrained by a hard ceiling: the context window, the maximum number of tokens the model can process at once.

When a developer treats the prompt as the only thing that matters, they tend to write increasingly elaborate instructions to compensate for missing context. They try to encode the entire user manual in the system prompt, paste long documents into the user message, and rely on the model to "figure out" which parts are relevant. This works for toy problems and breaks in production, because every token in the context window costs money, costs latency, and competes for the model's finite attention. Pasting a 200-page manual into the prompt is not context engineering; it is context dumping, and it produces unreliable, expensive, slow systems.

The shift from prompt engineering to context engineering is therefore not a stylistic preference. It is a recognition that the prompt is the *output* of a pipeline, not the input to the model. Building that pipeline well is what makes LLM applications reliable.

From Prompt Engineering to Context Engineering

The two disciplines are related but not interchangeable, and understanding their relationship is the first step toward designing systems that scale.

Prompt engineering is the practice of crafting the textual instructions and examples that go directly into the model call. It is concerned with questions like: How should I phrase the instruction to get the desired output? Should I use a system message, a user message, or both? How many few-shot examples will improve accuracy without exceeding the budget? Which output format is least likely to be misinterpreted?

Context engineering is the practice of designing the entire information environment that surrounds the model call. It is concerned with questions like: Which documents should be retrieved for this query? How should those documents be chunked, summarized, or compressed to fit the budget? What information from previous turns should be included, and what should be discarded? How should tool definitions, function schemas, and structured outputs be presented? How should the response be parsed, validated, and used to update the application's state?

In other words, prompt engineering optimizes a single query. Context engineering optimizes a pipeline.

The distinction becomes clearer with an example. Suppose you want a model to summarize a legal contract. A prompt engineer would write something like:

> "You are an expert legal analyst. Read the contract below and produce a structured summary with sections for parties, term, payment terms, termination clauses, and liabilities. Be concise."

A context engineer would instead design a pipeline that, for a given contract, does the following:

1. Classifies the document type to choose the right prompt template. 2. Splits the contract into chunks small enough to fit within the model's context budget. 3. For each chunk, runs an extraction pass to identify candidate clauses. 4. Aggregates the extractions into a unified structured representation. 5. Renders that representation into a final summary prompt. 6. Validates the output against a JSON schema and retries if validation fails.

The prompt still exists in this pipeline, but it is the *result