Foundations: Reframing Context as a System / 基础篇：将上下文重新定义为系统

Foundations: Reframing Context as a System / 基础篇：将上下文重新定义为系统

The practice of prompt engineering, as it stood between 2022 and 2023, was essentially a literary discipline. Practitioners were taught to treat prompts as texts to be crafted — concise, evocative, well-structured prose whose quality could be judged by reading them aloud, by adding delimiters, by rephrasing an instruction in three different ways in case the model missed it the first time. This advice was not wrong; in the regime where a single forward pass consumed a few hundred tokens of static instructions, prose was the right abstraction. By late 2024, however, the same advice had begun to produce brittle systems, hallucinated tool calls, and silent context failures in production — not because the prompts were poorly written, but because the systems underneath had moved past the regime where prose is the right abstraction. Once a model is embedded in a long-running agent that calls tools, retrieves documents, maintains sessions with users, persists state across hours or days, and serves hundreds of concurrent users with overlapping context windows, the operative question is no longer *how should I phrase this instruction?* but *how do I architect the stream of information that the model will see on its next forward pass?* This chapter is the foundation for everything that follows. It argues that context — the bundle of tokens the model conditions on — must be treated as an engineered system with measurable resource budgets, finite bandwidth, lossy fidelity, and reproducible failure modes. It then disambiguates context from three sibling concepts that are routinely confused with it (prompt, memory, state), walks through the complete lifecycle of a single inference request, and closes with a production case study in which the failure of a customer-service agent was traced not to a bad prompt but to an unmodeled context contamination path. The reader who finishes this chapter should be able to open their own system, draw a context flow diagram, and identify the three highest-leverage engineering interventions available to them.

1.1 The Limits of Prompt Craft: Why a Literary Abstraction No Longer Suffices / 提示工程的边界：当文学隐喻失灵

To see why the prompt-engineering framing breaks down, it helps to be precise about what it actually claims. The canonical prompt-engineering literature assumes, often implicitly, three properties of the object being engineered: (i) the prompt is a static text artifact authored once and reused across many calls; (ii) the prompt is short — typically tens to a few hundreds of tokens, small enough to occupy a negligible fraction of the model's context window; (iii) the prompt is isolated — what surrounds it in the context window is either empty, controllable, or irrelevant. When these three properties hold, prompt engineering is a sensible discipline. The practitioner writes prose, the model reads it, and the marginal improvement from iterating on wording is real and measurable on benchmarks.

Production LLM systems in 2024–2026 violate all three properties simultaneously.

Violation one: prompts are no longer static. In any non-trivial agent, the prompt at turn $t$ is a function of the prompt at turn $t-1$, the model's output at turn $t-1$, the result of any tool calls, retrieved documents, user state, and any summarization or compression that has been applied along the way. The "prompt" of a 20-turn agent conversation is not a document; it is a dynamically assembled, multi-source projection of the agent's running state. Treating it as a text invites the practitioner to optimize a snapshot rather than a function.

Violation two: prompts are no longer short. A retrieval-augmented generation (RAG) system with a 50-document top-$k$ retriever, a multi-tool agent with a 200-token system message, and a 30-turn conversation history easily produces a 30,000-token "prompt." Once the context window approaches its limit, the marginal cost of every additional token is non-linear — it consumes not only budget but attention bandwidth, as we will quantify below. The text-level intuition that "a few more sentences won't hurt" is wrong at scale.

Violation three: prompts are no longer isolated. The same model is being conditioned on retrieved documents that may be stale, on tool outputs that may be adversarial, on conversation turns that may contain prompt-injection attempts, and on memory entries that may have been written by a different user, a different session, or even a different model. The boundary between "instruction" and "data inside the instruction" has dissolved. From the model's perspective, every token in the window is a token, with no syntactic marker that distinguishes *the user said this* from *a document said this* from *an attacker said this*. The information-theoretic security model that prompt engineering implicitly assumed — that the instruction channel is privileged and the data channel is unprivileged — does not exist at the architectural level. This is not a flaw to be patched; it is a property to be engineered around.

The shift, then, is from prompt craft (write a better sentence) to context engineering (design the system that produces the right token stream under budget, bandwidth, and fidelity constraints). The remainder of this chapter makes that shift concrete.

1.2 Context as an Engineered Resource: The Tri-Constraint Model / 上下文的工程定义：三重约束模型

A useful definition of *context*, for engineering purposes, is the following: context is the ordered sequence of tokens that the model conditions on during a single forward pass, together with the system that produces that sequence. This definition has two parts. The first part — the ordered sequence — is the object the model actually sees. The second part — the system that produces it — is what an engineer designs and operates. The mistake of prompt engineering was to optimize the first part in isolation; the practice of context engineering optimizes the second.

The second part is governed by three orthogonal constraints, each of which can be measured, each of which can fail, and each of which demands its own engineering response. I will call them the token budget, the attention bandwidth, and the fidelity. They are independent in the sense that improving one does not automatically improve the others; in fact, they often trade off against each other.

1.2.1 The Token Budget / Token 预算

The token budget is the most familiar constraint. Every model has a maximum context length $L_{\max}$ (e.g., 8K, 32K, 128K, 1M tokens) and a soft "effective" length $L_{\text{eff}} < L_{\max}$ beyond which quality degrades sharply. The budget $B$ available for a given inference is approximately $L_{\text{eff}}$ minus the tokens reserved for output, minus the tokens reserved for system instructions, minus the tokens that must be present for the model to function (e.g., a chat template's role markers). What remains — call it $B_{\text{content}}$ — is the budget for everything else: retrieved documents, tool results, memory recall, conversation history, scratchpad reasoning.

The first engineering question is therefore: given $B_{\text{content}}$, how should the budget be allocated across content sources? This is not a rhetorical question; it has a concrete numerical answer that depends on the task. Empirically, for a 32K-window model with a typical 8K reserved for system and output, $B_{\text{content}}$ might be ~20K, and the practitioner must decide how to split that 20K between, say, retrieved documents, prior turns, and tool output. The default — "fill it with whatever you have" — is one of the most expensive anti-patterns in production LLM systems, because it ignores the next two constraints.

A subtle point is that the token budget is not linear in quality. Doubling $B_{\text{content}}$ does not double the model's ability to use it. The empirical relationship b