Why LLMs Forget — The Birth of RAG / 大模型为何会'失忆'——RAG的诞生

第 1 章：Why LLMs Forget — 大模型为何会"失忆"，RAG 如何诞生

> 本章核心问题 / Core Question > Why do large language models need external knowledge, and what problem does RAG solve? > 为什么大语言模型需要外部知识？RAG 究竟解决了什么问题？

---

1.1 一个真实发生过的"翻车现场"

A Real "Failure" That Actually Happened

让我们先从一个真实场景开始。

Imagine this: It's a Sunday evening in October 2024. A 16-year-old student named Lin opens ChatGPT and types:

> "Who won the 2024 Nobel Prize in Physics?"

The reply comes back instantly, in perfect English, with confidence that could fool a teacher:

> "The 2024 Nobel Prize in Physics was awarded to Dr. Emily Carter and Dr. Rajiv Mehta for their pioneering work on quantum error correction in topological qubits."

Lin copies this answer into a class group chat. Three minutes later, a friend replies with a link to the official Nobel Prize website: the real winners are John Hopfield and Geoffrey Hinton, for foundational discoveries in machine learning. Lin's face turns red. ChatGPT didn't just get it wrong — it invented two completely fictional scientists and confidently attached their names to a real prize.

Now ask yourself: Why did it do that? Was the model broken? Was it lying? Did it hate Lin's homework?

No. The model did exactly what it was trained to do. And understanding *why* is the entire reason this book exists.

> Open Question / 留给你的思考题 > If you were the engineer who built that model, what would you change so that Lin never gets a fictional physicist again — without retraining the entire model from scratch? > 如果你是那款模型的工程师，你会做哪些修改，让 Lin 永远不会再收到一个"虚构的物理学家"——而且还不需要重新训练整个模型？

This is the question that gave birth to RAG (Retrieval-Augmented Generation / 检索增强生成). Before we can build it, we have to deeply understand the disease we are curing. So let's dissect the three fatal flaws that every standalone LLM (Large Language Model, 大语言模型) carries in its DNA.

---

1.2 大模型的三大致命缺陷 / The Three Fatal Flaws of a Standalone LLM

A standalone LLM — meaning a model that you just open in a chat box and talk to, with no extra plumbing attached — has three problems that no amount of "making it bigger" can fully solve. These are not bugs; they are structural limitations baked into the architecture and training process.

1.2.1 缺陷一：知识截止 (Knowledge Cutoff)

Flaw #1: Knowledge Cutoff

Every LLM has a specific date stamped into it: the last day its training data was collected. For GPT-4, that date is October 2023. For many open-source models, it can be even earlier.

What does this mean? Imagine you are a student who studies only from textbooks printed before October 2023. From that day forward, the world keeps happening — new laws are passed, new scientific papers are published, new movies are released, new viruses are discovered — but your textbooks never update. You still have to answer questions about a world that has moved on. That is exactly the situation an LLM is in.

```mermaid flowchart LR A[Training data collected<br/>up to Oct 2023] --> B[Model weights frozen<br/>no more updates] B --> C[User asks about<br/>Nov 2024 event] C --> D{Is the event<br/>after the cutoff?} D -- Yes --> E[Model has no idea<br/>must guess or invent] D -- No --> F[Model may answer<br/>from memory] ```

In the flowchart above, notice the two branches on the right. The "No" branch is not even a guarantee — it just means the model *might* remember. The "Yes" branch is where disasters like Lin's fictional physicists live.

> Why can't we just retrain every week? / 为什么不能每周重新训练一次？ > Training a frontier LLM costs millions of dollars in compute, takes weeks of GPU time, and consumes enough electricity to power a small city. You cannot do it on a whim. Even if you could, the moment you finish retraining, the world has already moved 12 hours further. You will always be chasing a moving target. This is why knowledge cutoff is a structural problem, not an engineering oversight.

1.2.2 缺陷二：幻觉 (Hallucination)

Flaw #2: Hallucination

"Hallucination" is the technical name for when an LLM produces text that sounds correct but is factually wrong. Lin's fictional physicists are a textbook example. But hallucination is much sneakier than that. It can:

• Invent citations to papers that don't exist.
• Attribute real quotes to the wrong people.
• Describe a historical event with the right names but the wrong dates.
• Give you a recipe that mixes baking soda and baking powder as if they were the same thing.

The root cause is fundamental: an LLM is a pattern-completion engine, not a knowledge database. When you ask it a question, it does not "look up" the answer. Instead, it predicts the most statistically likely sequence of words given everything it has seen during training. If the most likely-sounding sequence happens to be false, the model will still produce it — cheerfully, confidently, and with full English grammar.

Think of it this way. A student who has read 10,000 books but has no access to any of them right now is asked, "What did Chapter 7 of *Book X* say about topic Y?" The student cannot refuse to answer. So they improvise. The improvisation may be brilliant, but it is also a guess.

> Critical Insight / 关键洞见 > Hallucination is not a sign that the model is broken. It is a sign that the model is doing its job: generating fluent, plausible text. The problem is that humans have evolved to interpret fluent text as truth, and the model has not. We need a way to give the model access to *real* facts at the moment of answering, so that its fluency is anchored to reality instead of to statistical patterns.

1.2.3 缺陷三：无法访问私有数据 (No Access to Private / Personal Data)

Flaw #3: No Access to Private Data

Suppose you want to build a homework helper that knows *your* school notes, *your* teacher's grading rubric, and *your* class's upcoming exam schedule. A standalone LLM cannot do this. Why? Because:

1. The model was trained on the public internet, not on your personal files. 2. Even if you upload a file to a chat interface, the model only "sees" it during that one conversation. It does not remember it next time. And it certainly does not have an index over thousands of your notes. 3. You would not — and should not — upload your private medical records, your company's internal wiki, or your family's photo album to a public model.

So a standalone LLM is, in a sense, a brilliant stranger: friendly, knowledgeable about the world, but completely ignorant about *you* and your world.

> Try this at home / 自己动手试一下 > Open any LLM chat interface. Ask it: "What did I write in my biology notebook last Tuesday?" It will either admit it has no idea, or — worse — it will make up something plausible. This is the third fatal flaw, and it is the one that most motivates the book you are reading right now.

1.2.4 把三个缺陷连起来看

Putting the Three Flaws Together

Let's lay them side by side in a single picture.

| Flaw / 缺陷 | What it means / 含义 | Real-world consequence / 现实后果 | |---|---|---| | Knowledge Cutoff | 知识有截止日期 | 不能回答最新事件 | | Hallucination | 流畅但可能虚构 | 给出错误答案且非常自信 | | No Private Data | 看不到个人数据 | 无法个性化，无法回答"我的"问题 |

These three flaws share a common shape: the model knows a lot, but it does not know enough, and it does not know how to say "I don't know." Any solution worth its salt must address all three. Enter RAG.

---

1.3 "开卷考试"类比：先检索，再回答 / The Open-Book Exam Analogy

Section Goal / 本节目标 By the end of this section, you should be able to explain to a friend, in under 30 seconds, what RAG is and why it works.

1.3.1 闭卷考试与开卷考试

Closed-Book vs Open-Book Exams

In school, you have probably taken both kinds of exams.

• **Closed-book exam (闭卷考试)**: You walk in, you sit down, and the only knowledge you can use is what is in your head. You either remember it, or you don't. If you don't, you either leave it blank or you improvise (and risk losing points).
• **Open-book exam (开卷考试)**: You are allowed to brin