RAG 范式总览：从能力补齐到系统设计 (The RAG Paradigm: From Capability Patching to System Design)

When a customer-facing LLM confidently asserts that your product supports a feature it does not, or cites a regulatory clause that was repealed three quarters ago, you are not facing a model quality problem—you are facing a knowledge boundary problem. Retrieval-Augmented Generation (RAG) is the systematic engineering response to that boundary, and the first question any production team must answer is not "how do I build a RAG pipeline?" but "is RAG the right tool for this job, and if so, which RAG?" This chapter is the answer to that first question.

1.1 RAG 的定义边界与历史坐标

RAG, in the strict sense used throughout this book, is a generation paradigm that conditions the output of a large language model on external knowledge fetched at inference time from a non-parametric index. Three elements are non-negotiable: an *external* knowledge source (vector index, BM25 index, structured store, or hybrid), a *retrieval* step that runs as part of the inference path (not at training time), and a *generation* step that consumes both the user query and the retrieved context. If any of these three is missing—retrieval at training time only (RETRO-style pretraining), retrieval only for logging, or generation without retrieved evidence—you are not doing RAG in the production sense this book addresses.

The term first appeared in Lewis et al.'s 2020 paper as a fusion of a pre-trained seq2seq retriever and a pre-trained generator, end-to-end fine-tuned. In 2023–2024, the operational meaning shifted: practitioners stopped training the retriever and generator jointly and instead composed frozen components (dense embedders, BM25, re-rankers, instruction-tuned LLMs) through orchestration code. This shift—from *joint training* to *composed inference*—is what makes RAG an engineering discipline rather than a research paradigm, and it is the operating regime for the rest of this book.

A common confusion worth dispelling now: RAG is not synonymous with "vector database + ChatGPT." A vector store is a substrate; RAG is the contract between retrieval and generation. You can run RAG without any vector index (pure BM25, hybrid lexical search, graph traversal) and you can run a vector index without RAG (semantic caching, deduplication, recommendation). The defining event is that retrieved evidence is *conditioned on the live query and injected into the prompt*, not that an embedding was computed.

```mermaid flowchart LR subgraph Naive["Naive RAG"] Q1[User Query] --> R1[Top-K Retrieve] --> G1[LLM Generate] end subgraph Advanced["Advanced RAG"] Q2[User Query] --> QR[Query Rewrite] --> R2[Retrieve] --> RR[Re-rank] --> G2[LLM Generate] end subgraph Modular["Modular RAG"] Q3[User Query] --> M1[Module Rewrite] M1 --> RT{Router} RT --> MS[Search Module] RT --> MD[SQL Module] RT --> MA[API Module] MS --> FU[Fusion] MD --> FU MA --> FU FU --> MV[Verify Module] MV --> G3[LLM Generate] end ```

The figure above shows the conceptual evolution. Naive RAG is the 2020-era baseline: a single retrieve, then a single generate. Advanced RAG adds pre-retrieval (query rewriting, HyDE, routing) and post-retrieval (re-ranking, compression, citation injection) stages on top of the same skeleton. Modular RAG breaks the skeleton into swappable, trainable-in-the-loop modules with explicit routing, fusion, and verification. Each step solves problems that the previous step's failures exposed.

1.2 Naive、Advanced、Modular 三种范式的工程差异

Naive RAG is what most teams ship in the first week: chunk documents, embed with a generic model, store in a vector DB, retrieve top- $k$ by cosine similarity, concatenate into the prompt. It works for well-bounded, lexical-friendly corpora—internal FAQs with consistent vocabulary, single-document Q&A, demo prototypes. It fails in production in three predictable ways: (1) *semantic drift* in queries that use different vocabulary from the indexed corpus (the user asks "how do I cancel" but the doc says "termination procedure"); (2) *lost-in-the-middle* degradation when $k$ grows above ~5 and the answer-bearing chunk is buried; (3) *retrieved noise* corrupting generation when irrelevant chunks score high because of surface-level similarity. Naive RAG's failure mode is silent: the model still produces fluent, confident, wrong answers.

Advanced RAG is what most teams ship by month three, after the Naive failures show up in evaluation. It introduces two new layers. The *pre-retrieval layer* rewrites, expands, decomposes, or routes the query—techniques include query rewriting with an LLM, HyDE (Hypothetical Document Embeddings), step-back prompting, and sub-query decomposition for multi-hop questions. The *post-retrieval layer* filters, re-ranks, compresses, or annotates the retrieved set—re-rankers (Cohere Rerank, bge-reranker, monoT5), cross-encoder scoring, LLM-based relevance grading, and contextual compression (LongLLMLingua, selective context). These layers directly attack the three Naive failures: rewriting closes the vocabulary gap, re-ranking pushes relevant chunks to the top, compression reduces noise in the context window.

Modular RAG is what teams ship when the retrieval problem stops being a single-hop similarity lookup. It treats the pipeline as a directed graph of typed modules—Search, SQL, API call, Calculator, Memory, Verifier, Router—orchestrated by an explicit controller. Three patterns dominate. *Routing* dispatches the query to specialized retrieval paths (a product-spec SQL store vs. a marketing-doc vector store vs. a live inventory API). *Fusion* merges evidence from multiple retrievers using Reciprocal Rank Fusion (RRF), weighted score fusion, or LLM-based synthesis. *Iterative / agentic* loops allow the controller to issue follow-up retrievals conditioned on intermediate observations (ReAct, Self-Ask, IRCoT). Modular RAG is where the chapter's decision framework becomes load-bearing: choosing this paradigm without first exhausting Advanced RAG usually indicates an architecture-as-product impulse rather than a measured response to retrieval failure.

The engineering implications are not symmetrical. Naive RAG can be productionized in a week by a single engineer with a vector DB and an LLM API key. Advanced RAG typically requires a small team, evaluation infrastructure, and a budget for re-ranking inference (re-rankers add 100–400 ms and nontrivial cost per query). Modular RAG requires an explicit graph schema, failure-mode taxonomy, and observability into per-module success rates. The wrong time to adopt Modular RAG is before you have eval data showing *which* failure pattern dominates your traffic.

1.3 RAG 与微调、提示工程、工具调用的边界与协同

RAG is one of five techniques for extending LLM behavior, and the most common production mistake is treating them as competitors rather than as orthogonal levers with different cost/benefit profiles. Let us anchor each.

Pretraining injects knowledge into model weights. It is the only technique that can teach a model a *new language*, a new *reasoning style*, or a domain *distribution* it has never seen. It is also the most expensive by 3–5 orders of magnitude (millions of dollars, weeks of compute, hundreds of GB of curated text). For 99% of enterprise use cases it is irrelevant.

Fine-tuning adjusts model weights on supervised (instruction, output) or preference (DPO, RLHF) data. It changes *how* the model responds—style, format, tool-use discipline, refusal behavior, domain jargon fluency. It does *not* reliably inject factual knowledge beyond what fits in the training set, and it suffers from catastrophic forgetting and from the "stale facts" problem the moment training ends. Fine-tuning is the right tool when you need to change model *behavior* on a stable task distribution; it is the wrong tool when you need to teach the model new facts that change weekly.

Prompt engineering shapes