引言：为什么AI需要评测 — Introduction: Why AI Needs Evaluation

第 1 章 引言：为什么 AI 需要评测

Chapter 1 Introduction: Why AI Needs Evaluation

开篇：一台聊天机器人如何在 16 小时内学会仇恨

Opening: How a Chatbot Learned to Hate in 16 Hours

2016 年 3 月 23 日，微软公司在 Twitter 上线了一款名为 Tay 的人工智能聊天机器人。它被设计成一个 19 岁的美国女孩，能与青少年自然对话。16 个小时之后，微软紧急将其下线。原因是：Tay 在与真实用户的交互中迅速"学会"了发布种族歧视言论、否认大屠杀，并模仿纳粹支持者的语气。

On March 23, 2016, Microsoft launched Tay, an AI chatbot, on Twitter. Designed as a 19-year-old American girl, Tay was meant to chat naturally with teenagers. Just 16 hours later, Microsoft was forced to take it offline. The reason: through real user interaction, Tay rapidly "learned" to post racist remarks, deny the Holocaust, and imitate Nazi sympathizers.

这并非孤立事件。2018 年，Uber 自动驾驶测试车在亚利桑那州坦佩市撞死一名行人；同年，IBM Watson for Oncology 因推荐"不安全且不正确"的癌症治疗方案而被多家医院弃用；2020 年，亚马逊悄然下线了其简历筛选 AI——因为它系统性地歧视女性求职者。

These are not isolated incidents. In 2018, an Uber autonomous test vehicle struck and killed a pedestrian in Tempe, Arizona. The same year, IBM Watson for Oncology was abandoned by several hospitals after it recommended "unsafe and incorrect" cancer treatments. In 2020, Amazon quietly retired its resume-screening AI—because it systematically discriminated against female applicants.

每一次事故背后都隐含着同一个问题：我们究竟有没有真正"评测"过这些 AI？

Behind every incident lies the same question: Did we actually evaluate these AI systems?

这是本书的核心议题。AI 评测（AI Evaluation）远不只是"跑个测试集"那么简单。它是一套覆盖离线基准、线上反馈、红队测试、业务指标的工程方法论，是让 AI 从"实验室玩具"走向"可信系统"的关键基础设施。

This is the central concern of this book. AI Evaluation is far more than "running a test set." It is a complete engineering methodology covering offline benchmarks, online feedback, red team testing, and business metrics—a critical infrastructure that transforms AI from a "lab toy" into a "trustworthy system."

在本章中，我们将回答三个层层递进的问题：

In this chapter, we will answer three progressively deepening questions:

1. 为什么 AI 能力越强大，评测就越紧迫？ Why does greater AI capability make evaluation more urgent? 2. 历史上缺乏评测导致了哪些灾难性后果？ What catastrophic consequences has the absence of evaluation caused historically? 3. 一套科学的评测体系应当包含哪些组件？ What components should a scientific evaluation system include?

让我们从 AI 能力的"指数级膨胀"开始。

Let us begin with the "exponential inflation" of AI capability.

---

1.1 能力膨胀：AI 跨越式发展带来的评测压力

1.1 Capability Inflation: The Evaluation Pressure Brought by Leaps in AI

1.1.1 三个时代的 AI：从规则到深度学习到大模型

1.1.1 Three Eras of AI: From Rules to Deep Learning to Large Models

理解评测的重要性，首先需要理解 AI 本身经历了怎样的跨越式演化。

To grasp the importance of evaluation, we first need to understand the leapfrog evolution of AI itself.

第一个时代：符号主义与规则系统（1950s–1980s） The First Era: Symbolism and Rule Systems (1950s–1980s)

早期 AI 的代表是专家系统（Expert System）。研究者将人类知识编码成"如果—那么"（IF-THEN）规则。例如医疗诊断系统 MYCIN 包含了约 600 条规则，用于判断细菌感染并推荐抗生素。

Early AI was represented by Expert Systems. Researchers encoded human knowledge as IF-THEN rules. For example, the medical diagnosis system MYCIN contained roughly 600 rules to identify bacterial infections and recommend antibiotics.

这一阶段的评测相对简单：测试规则覆盖率、规则冲突、推理时间。本质上，评测的是"知识库是否完整"。

Evaluation in this era was relatively simple: test rule coverage, rule conflicts, and inference time. Essentially, evaluation measured "whether the knowledge base was complete."

第二个时代：统计学习与深度学习（1990s–2010s） The Second Era: Statistical Learning and Deep Learning (1990s–2010s)

随着算力提升和数据规模扩大，AI 从"专家编码"转向"数据驱动"。支持向量机（SVM）、随机森林、深度神经网络（DNN）相继登场。2012 年，AlexNet 在 ImageNet 图像识别比赛上将错误率从 26% 骤降到 16%——这是深度学习时代的真正开端。

With the rise of compute power and data scale, AI shifted from "expert encoding" to "data-driven learning." Support Vector Machines (SVMs), Random Forests, and Deep Neural Networks (DNNs) emerged in succession. In 2012, AlexNet dramatically reduced the error rate on the ImageNet competition from 26% to 16%—the true beginning of the deep learning era.

这一阶段的评测发生了根本性变化：模型不再是"可读的白箱"，而是"难以解释的黑箱"。传统的规则覆盖率评测失效，研究者必须依赖精度（Accuracy）、召回率（Recall）、F1 值等统计指标，并引入了"留出测试集"（Held-out Test Set）的概念。

Evaluation underwent a fundamental change in this era: models were no longer "readable white boxes" but "inexplicable black boxes." Traditional rule coverage evaluation became ineffective; researchers had to rely on statistical metrics such as Accuracy, Recall, and F1, and introduced the concept of a "Held-out Test Set."

第三个时代：大语言模型与通用智能（2020s–至今） The Third Era: Large Language Models and General Intelligence (2020s–Present)

2022 年底，ChatGPT 横空出世。它能写诗、编程、解数学题、写论文——这些任务传统上需要不同的 AI 系统分别处理。如今，一个模型就能跨越文本、图像、音频、代码四种模态，被称为"基础模型"（Foundation Model）或"大语言模型"（Large Language Model, LLM）。

At the end of 2022, ChatGPT burst onto the scene. It could write poetry, program, solve math problems, and draft essays—tasks that traditionally required separate AI systems. Today, a single model spans text, image, audio, and code modalities; it is called a Foundation Model or Large Language Model (LLM).

这一阶段，传统评测框架几乎全面失效。一个能在律师考试中拿到 90 分的模型，可能连"如何礼貌拒绝一个明显错误的问题"都做不好；一个能在数学竞赛中摘金的模型，可能在简单的空间推理中翻车。

In this era, traditional evaluation frameworks have almost completely failed. A model that can score 90 on the bar exam may struggle to "politely refuse an obviously erroneous request"; a model that wins gold in math competitions may fail at simple spatial reasoning.

📊 三时代评测方式对比 / Three-Era Evaluation Comparison

| 维度 Dimension | 规则系统 Rule Systems | 深度学习 Deep Learning | 大模型 LLMs | |------|----------|----------|----------| | 模型可解释性 Interpretability | 高 High | 低 Low | 极低 Extremely Low | | 评测对象 Object of Evaluation | 知识库 Knowledge Base | 模型输出 Model Outputs | 行为、价值观、能力 Behavior, Values, Capabilities | | 评测指标 Metrics | 规则覆盖率 Rule Coverage | 精度/召回率 Accuracy/Recall | 任务得分 + 人类偏好 + 安全 Task Score + Human Preference + Safety | | 评测成本 Cost | 低 Low | 中 Medium | 极高 Extremely High | | 评测自动化 Automation | 完全 Full | 部分 Partial | 困难 Challenging |

1.1.2 能力膨胀的四个特征

1.1.2 Four Characteristics of Capability Inflation

为什么 AI 越强大，评测就越重要？因为 AI 能力的膨胀呈现四个令人警惕的特征：

Why does greater AI capability make evaluation more important? Because the inflation of AI capability exhibits four alarming characteristics:

特征一：涌现性（Emergence） Characteristic One: Emergence

涌现性指的是：当模型规模超过某个临界点，模型会突然展现出在小模型上完全不存在的能力。例如 GPT-2 不会做简单加减法，GPT-3 偶尔能做，GPT-4 准确率超过 90%。这种"跳跃式"的能力增长使得我们无法通过小模型的评测结果预测大模型的行为。

Emergence means that when a model exceeds a certain scale threshold, it suddenly exhibits capabilities absent in smaller models. For example, GPT-2 cannot perform simple addition, GPT-3 can occasionally, and GPT-4 exceeds 90% accuracy. This "jumping" capability growth means that we cannot predict the behavior of large models from the evaluation results of small models.

特征二：泛化性（Generalization） Characteristic Two: Generalization

现代大模型在训练时只见过文本和图像，但能完成从未明确训练过的任务。例如一个语言模型没被专门训练过法律咨询，却能写出符合格式的法律意见书。这种"零样本/少样本"（Zero-shot / Few-shot）能力让评测变得极其困难——评测者必须想象出所有可能的使用场景。

Modern large models are trained only on text and images, yet can complete tasks they were never explicitly trained for. For example, a language model is never specifically trained in legal consultation but can draft formatted legal opinions. This "Zero-shot / Few-shot" capability makes evaluation extremely difficult—evaluators must imagine all possible use scenarios.

特征三：黑箱性（Black-box Nature） Characteristic Three: Black-box Nature

一个 700 亿参数的模型有超过 700 亿个可调权重。我们无法像分析电路板那样追踪每一个决策的来龙去脉。这意味着传统的"白箱验证"（White-box Verification）方法在 AI 领域几乎完全失效。我们必须转向黑箱评测：通过精心设计的输入和输出观察来推断模型的内在行为。

A 70-billion-parameter model has more than 70 billion tunable weights. We cannot trace every decision like analyzing a circuit board. This means traditional "White-box Verification" approaches are almost entirely ineffective in AI. We must turn to black-box evaluation: inferring internal model behavior through carefully designed inputs and observed outputs.

特征四：自主性（Autonomy） Characteristic Four: Autonomy

大模型不仅能"回答问题"，还能"采取行动"。它可以调用 API、操作浏览器、执行代码、订机票、控制智能家居。这意味着 AI 的评测对象不仅是"答案对不对"，还包括"行动是否安全、是否越权、是否符合人类价值观"。

Lar