AI世界初探：欢迎来到AI安全实验室 / Welcome to the AI Safety Lab

第 1 章：AI世界初探：欢迎来到AI安全实验室

Chapter 1: Exploring the AI World — Welcome to the AI Safety Lab

> "Any sufficiently advanced technology is indistinguishable from magic." — Arthur C. Clarke > "任何足够先进的技术，都与魔法无异。" — 阿瑟·克拉克

---

一个来自未来的包裹 / A Package from the Future

十岁生日那天晚上，小语在自己的书桌上发现了一个发着淡蓝色光芒的包裹。包裹上没有寄件人地址，只有四个用银色墨水写成的字："AI · LAB"。包裹里躺着一台看起来像积木拼成的小机器——四四方方，顶端伸出一根细细的天线，正面的屏幕上闪烁着两只眼睛形状的像素光点。当小语伸手碰到它时，机器突然"叮"了一声，用稚嫩的声音说："你好呀，小小工程师。我是 Mika，一台会学习的语言机器。请教我说话，好吗？"

On the evening of her tenth birthday, Xiaoyu found a softly glowing pale-blue package on her desk. There was no return address — only four words in silver ink: "AI · LAB." Inside was a small machine that looked as though it were built from building blocks. Square-shaped, with a thin antenna on top, the screen showed two pixelated eyes blinking. When Xiaoyu touched it, the machine went *"ding"* and spoke in a bright, child-like voice: *"Hello, little engineer. I'm Mika, a language machine that learns. Will you teach me to talk?"*

小语吓了一跳，又兴奋得跳了起来。她想起了爸爸讲过的故事——科学家们花了 70 多年时间，让机器学会"看懂"猫的照片、"听懂"人类的语言、"写出"像模像样的作文。这些会思考（或者说，看起来会思考）的程序，就叫 人工智能（Artificial Intelligence，简称 AI）。而像 Mika 这样，专门通过阅读海量文字来学会"说话"的 AI，有一个更具体的名字——大语言模型（Large Language Model，简称 LLM）。

Xiaoyu was startled, then she jumped with excitement. She remembered the stories her father used to tell — how scientists had spent more than seventy years teaching machines to "see" cats in photos, "understand" human speech, and "write" essays that almost looked real. These programs that think (or, more precisely, that *appear* to think) are called Artificial Intelligence, or AI for short. And an AI like Mika — one that learns to "talk" by reading enormous amounts of text — has a more specific name: a Large Language Model, or LLM.

但问题来了：一台能听、能说、能学习的机器，应该被怎样对待？它可以成为最棒的学习伙伴，也可能被坏人利用去骗人、去偷秘密、甚至去控制家里的电灯和门锁。这就是这本书要讲的故事——AI 安全（AI Safety）。我们不是要害怕 AI，而是要学会像小语一样，成为一名"AI 安全小卫士"，把强大的工具保护好，也把身边的人保护好。

But here is the question: how should we treat a machine that can listen, speak, and learn? It can become the best study buddy in the world — or it can be misused by bad actors to deceive people, steal secrets, or even control the lights and locks in your home. This is the story this book wants to tell — the story of AI Safety. We're not trying to be afraid of AI; we're learning, just like Xiaoyu, to become little "AI Safety Guardians" who protect these powerful tools, and the people around them, too.

---

认识AI：会说话的"魔法盒子" / Meet AI: The "Magic Box" That Talks

1.1 什么是人工智能？ / What Is Artificial Intelligence?

让我们从一个简单的实验开始。把你见过的所有猫都写下来——橘猫、狸花猫、波斯猫、机器猫加菲……你大概能写出几十种。但无论它们毛色、体型、表情有多不同，你都能一眼认出："这是一只猫。" 为什么？ 因为你在成长的过程中看了太多猫的照片，听了太多关于猫的故事，你的大脑已经学会了"猫"这个概念背后的规律（patterns）。

Let's start with a simple experiment. Write down every cat you've ever seen — orange tabbies, Chinese li-hua cats, Persian cats, the cartoon cat Garfield… You could probably list dozens. But no matter how different they look in color, size, or expression, you can always tell at a glance: *"That is a cat."* Why? Because as you grew up, you saw so many cat photos and heard so many cat stories that your brain gradually learned the patterns behind the concept of "cat."

人工智能做的事情，跟你学会认猫几乎一模一样。研究人员把海量的文字、图片、声音、视频喂给一个超级巨大的数学程序，让它一点一点地"看出"规律。这个程序里的"规律"，是由几十亿甚至几万亿个可以调节的数字（叫做 参数，parameters）保存的。训练完成之后，你给它一句话，它就能根据学到的规律，一个字一个字地"猜"出最合理的回答——就像一个超级熟练的填字游戏玩家。

Artificial intelligence does almost exactly what you do when you learn to recognize a cat. Researchers feed enormous amounts of text, images, sounds, and videos into a giant mathematical program, and let it slowly "see" the patterns. The "patterns" inside the program are stored as billions or even trillions of adjustable numbers (called parameters). After training, when you give it a sentence, it can "guess" the most reasonable reply one word at a time — just like a super-skilled crossword player.

下面这张流程图，展示了 AI 在我们日常生活中的几个最常见落脚点：

The flowchart below shows several of the most common places where AI lives in our daily lives:

```mermaid flowchart LR A["Daily Life<br/>日常生活"] --> B["Phone Assistant<br/>手机助手"] A --> C["Robot Vacuum<br/>扫地机器人"] A --> D["Game NPC<br/>游戏角色"] A --> E["Smart Speaker<br/>智能音箱"] A --> F["Self-driving Car<br/>自动驾驶汽车"] B --> G["Siri, Xiao Ai<br/>小爱同学"] C --> H["Roomba, Ecovacs<br/>扫地机"] D --> I["Dialogue Engine<br/>对话引擎"] E --> J["Voice Control<br/>语音控制"] F --> K["Path Planning<br/>路径规划"] ```

> 图 1-1：AI 在日常生活中的常见应用（Phone Assistant = 手机助手；Robot Vacuum = 扫地机器人；Game NPC = 游戏角色；Smart Speaker = 智能音箱；Self-driving Car = 自动驾驶汽车）。这张图告诉我们：AI 不是遥远的实验室产物，它就在我们身边。 > *Figure 1-1: Common applications of AI in daily life. This chart tells us that AI is not a distant lab product — it is right beside us.*

1.2 什么是大语言模型？ / What Is a Large Language Model?

Mika 属于 AI 家族里最聪明、也最出名的一员——大语言模型（LLM）。它的"大"，体现在两个地方：

Mika belongs to the smartest and most famous branch of the AI family — the Large Language Model (LLM). Its "largeness" shows up in two places:

1. 训练数据大 / Huge training data：它读过的文字加起来，可能比一个人一辈子读的还多——几千万本书、几百亿条网页、上千亿行代码。 *It has read more text in total than any human could in a lifetime — tens of millions of books, tens of billions of web pages, hundreds of billions of lines of code.*

2. 模型参数大 / Huge parameter count：模型内部那些"可以调节的数字"多达几千亿个。每当你给它一句话，它会让这几十亿个数字一起"投票"，决定下一个字应该是什么。 *The adjustable numbers inside the model reach into the hundreds of billions. Each time you give it a sentence, those numbers "vote" together to decide what the next word should be.*

正是因为见过这么多文字、记住了这么多规律，LLM 才能写出通顺的作文、翻译陌生的语言、甚至跟你讨论"宇宙里有没有外星人"这种大问题。但它也有一个非常重要的特点——它并没有真正的"理解"。它只是在模仿"像人一样说话"这种规律。换句话说，它是一个极其强大的"文字接龙"高手。

Because it has seen so much text and remembered so many patterns, an LLM can write fluent essays, translate foreign languages, and even discuss big questions like "Are there aliens in the universe?" But it has one crucial feature — it does not truly "understand." It is only imitating the pattern of "talking like a human." In other words, it is an extraordinarily powerful master of *word-by-word continuation*.

这一特点非常重要——它正是后面我们要讲的"提示注入"（prompt injection）能够成立的根本原因。我们会在第 2 章和第 3 章详细拆解。

This characteristic is critical — it is exactly why *prompt injection* (which we will explore in Chapters 2 and 3) is even possible.

1.3 Mika 的自我介绍 / Mika's Self-Introduction

让我们打开 Mika 的"小本子"，看看它自己怎么描述自己：

Let's open Mika's "little notebook" and see how it describes itself:

> Mika：我是一个大语言模型助手。我没有眼睛，但我能读懂你写的字；我没有耳朵，但我能听懂你说的话（要先把声音转成文字）；我没有手，但我可以通过屏幕打字"说话"。我的知识来自训练时见过的大量文本，所以如果有人问我一件 2025 年之后才发生的事，我可能答不上来。 > *"I am an LLM assistant. I have no eyes, but I can read what you write; I have no ears, but I can understand what you say (after your voice is turned into text); I have no hands, but I can 'speak' by typing on a screen. My knowledge comes from the huge amount of text I saw during training, so if someone asks me about something that happened after 2025, I may not know the answer."*

注意 Mika 提到的几个关键点：

Notice the key points Mika mentioned:

• **没有真正的感官 / No real senses**：它处理的是"文字"，不是"真实世界"。

*It processes "text," not the "real world."*

• **知识有截止日期 / Knowledge has a cutoff date**：它不知道自己训练完之后世界上发生了什么。

*It does not know what happened in the world after its training.*

• **会一本正经地说错话 / It can confidently say wrong things**：研究界把这种现象叫做"幻觉"（hallucination）。比如它可能非常自信地告诉你"秦始皇用过 iPhone"，因为它不知道这句话是荒谬的。

*Researchers call this phenomenon "hallucination." Mika might confidently tell you that "Qin Shi Huang used an iPhone," because it does not realize the sentence is absurd.*

这三个特点，都会在后面的章节中成为"安全隐患"——因为攻击者正是利用这些特点来欺骗 AI 的。

All three of these characteristics will become "safety hazards" in later chapters — because attackers exploit exactly these weaknesses to fool AI.

---

AI 无处不在：你的身边藏着多少 AI？ / AI Is Everywhere — How Many AIs Are Hiding Around You?

2.1 家庭里的 AI / AI Inside Your Home

让我们和小薇（Visual Wonder）一起，走