Foundations of Context Engineering / 上下文工程的形式化基础

第 1 章 上下文工程的形式化基础 / Foundations of Context Engineering

1.1 问题的提出：为什么 "Prompt" 已经不够用

When we inspect the inference logs of any production-grade foundation model deployment — a coding agent, an enterprise search assistant, a multi-turn customer service bot — the input that actually drives the next token is rarely a single user utterance. It is a carefully constructed bundle comprising system instructions, tool definitions, retrieved documents, conversation history, schema constraints, and intermediate reasoning traces. The bundle may weigh a few hundred tokens for a casual chat, or exceed one million tokens for an agent ingesting a full repository. The behavior of the model is governed not by what the user said, but by what the system *decided* to put in front of the model.

This shift — from "what the user wrote" to "what the system assembled" — is the empirical fact that motivates this book. We argue that once the engineered bundle crosses a certain complexity threshold, the *construction* of context becomes a first-class engineering problem. It has its own constraints (finite context windows, quadratic attention cost, KV-cache memory pressure, dollar budgets), its own failure modes (lost-in-the-middle, instruction dilution, retrieval collisions, stale state), and its own design patterns (selection policies, refresh rules, compositional grammars, verification loops). Calling this practice "prompt engineering" is, at best, a misnomer; at worst, it conceals the engineering work that determines whether a system actually works.

The goal of this opening chapter is therefore narrow and structural. We do not yet discuss long-context extensions (Part II), compression (Part III), memory architectures (Part IV), or agent state machines (Part V). We lay a formal substrate: a definition of context as a four-tuple $(S, B, \pi, \rho)$, an ontology that separates prompt, context, working memory, external memory, and state, a cost model that exposes the economic tradeoffs, a six-stage engineering method that will reappear in every later chapter, and a worked example — a code-review assistant — that walks the reader end-to-end through the pipeline. Readers who internalize this chapter should be able to read any later chapter and recognize the same skeleton beneath the surface variations.

我们刻意从形式化入手而非从经验入手。这并非学院派的洁癖，而是因为这一领域的术语长期处于混乱状态：业界文献中 "prompt"、"context"、"memory"、"RAG context"、"system prompt"、"history"、"scratchpad"、"state" 等词被混用，导致大量本可避免的架构错误。形式化的首要价值不在于美感，而在于它强迫我们精确地说出每一层机制究竟在操作什么、消耗什么、产生什么。

1.2 Context 的形式化定义 / A Formal Definition of Context

1.2.1 核心元组 $(S, B, \pi, \rho)$

We define a context construction problem as a tuple

$$ \mathcal{C} \;=\; (S,\; B,\; \pi,\; \rho) $$

where each component is itself a formal object. We unpack them in turn.

Source stream $S$. The source stream is an ordered, possibly infinite, possibly heterogeneous sequence of *content units* drawn from a set $\mathcal{U}$. Content units may be natural-language tokens, structured records (JSON, code, table rows), tool outputs, embeddings, pointers to external storage, or any mix thereof. Formally,

$$ S = (u_1, u_2, u_3, \ldots),\qquad u_i \in \mathcal{U} $$

The stream has a partial order defined by its native semantics — a chat log is temporally ordered, a code repository is tree-ordered, a knowledge base is graph-ordered — and this ordering constrains how content can later be selected and composed. The stream is *unbounded* in principle: even if a particular log is finite, the *class* of all possible logs the system must handle is open-ended.

Budget $B$. The budget is a vector of resource constraints rather than a scalar. We write

$$ B = (b_{\mathrm{tok}},\, b_{\mathrm{lat}},\, b_{\mathrm{mem}},\, b_{\$}) $$

representing, respectively, the maximum number of input tokens, the maximum acceptable latency (typically broken into time-to-first-token and total completion latency), the maximum resident KV-cache memory, and the maximum dollar cost per request. A well-engineered system treats these as a Pareto frontier, not as a single number: a 30% latency reduction may be worth 50% more tokens if the dollar budget permits. The budget is therefore better thought of as a feasible region in a four-dimensional resource space than as a hard cap.

Selection policy $\pi$. The selection policy is a (possibly stochastic, possibly learned) function

$$ \pi : \mathcal{U}^{\le \infty} \;\longrightarrow\; \mathcal{P}(\mathcal{U}) $$

that maps the (truncated, observable portion of the) source stream to a subset of content units to be composed into the final context. $\pi$ encodes *what to keep*. Classical instantiations include recency-based truncation ($k$ most recent turns), relevance-based top-$k$ retrieval, recency-and-relevance hybrids, learned rerankers, and tool-call-conditioned selectors. A subtle but important point: $\pi$ is evaluated *with respect to a query or task*; "good" content is content that is useful for the *current* downstream task, not content that is generically important.

Refresh rule $\rho$. The refresh rule is a function

$$ \rho : \mathcal{C}_t \;\longrightarrow\; \mathcal{C}_{t+1} $$

that maps the current context state to the next, given new content arriving in the stream. $\rho$ encodes *when and how to drop, summarize, or re-rank*. Examples include: drop-oldest when budget is exceeded, summarize-and-replace for long chat histories, periodic re-retrieval when the source stream mutates, and triggered refresh on certain event types (tool failure, user correction). $\rho$ is what makes context engineering a *dynamic* discipline rather than a one-shot prompt-writing exercise.

A *context engineering problem* is then: given $(S, B)$, find $(\pi, \rho)$ such that downstream task performance is maximized subject to budget constraints. This framing is deliberately close to classical resource-constrained selection problems in operations research; we will see that many of the same techniques (greedy, dynamic programming, Lagrangian relaxation, learning-to-rank) reappear in Chapter 6 on compression and Chapter 9 on memory.

1.2.2 Context vector space / 上下文向量空间

Beyond the input-level tuple, we also need a representation-level view. We define the context vector space $\mathcal{V}$ as the tensor product of several substrates:

$$ \mathcal{V} \;=\; \mathcal{V}_{\mathrm{text}} \otimes \mathcal{V}_{\mathrm{embed}} \otimes \mathcal{V}_{\mathrm{kv}} \otimes \mathcal{V}_{\mathrm{struct}} $$

where $\mathcal{V}_{\mathrm{text}}$ is the discrete token space, $\mathcal{V}_{\mathrm{embed}}$ is the continuous hidden-state space produced by the model, $\mathcal{V}_{\mathrm{kv}}$ is the KV-cache tensor space (key and value tensors indexed by token position and layer), and $\mathcal{V}_{\mathrm{struct}}$ is the structural space of message objects (system, user, assistant, tool), schema tags, and explicit delimiters.

These four substrates are not redundant — they are different *views* of the same context. A token sequence corresponds to a unique embedding trajectory and a unique KV-cache state under a given model; a structural arrangement corresponds to a particular attention mask pattern that the model interprets during prefill. When we say a context is "the same" across two requests, we can mean it at any of these levels: identical token sequence, identical embedding, identical KV-cache, or identical structure. Production systems exploit this multiplicity aggressively: prompt caching reuses $\mathcal{V}_{\mathrm{kv}}$ across requests (Chapter 11), structural templates manipulate $\mathcal{V}_{\mathrm{struct}}$ (Chapter 4), embedding-based retrieval operates in $\mathcal{V}_{\mathrm{embed}}$ (Chapter 7).

1.2.3 为什么这是一个工程问题而不是一个 UX 问题 / Why this is engineering, not UX

A common objection is that context construction is "just deciding what to show the model" and therefore a UX concern. The objection fails for three reasons. First, the cost is non