什么是AI评测？为什么要这么麻烦？ / What is AI Evaluation and Why Should We Care?

一杯咖啡引发的"评测觉醒"

Imagine this: your team has just spent nine months training a large language model. The model achieves 92% on the MMLU benchmark, ranks in the top 10 on the LMSYS leaderboard, and the engineers are celebrating with champagne. Six weeks after launch, customer support is drowning in tickets: the model refuses to answer simple questions about half the time, occasionally invents fake legal citations, and last Tuesday it told a 12-year-old that eating tide pods was "a bold life choice." The product manager stares at the dashboard and asks, "How did this happen? We had a 92 score!"

The answer is simple and uncomfortable: a number on a leaderboard is not an evaluation. It is, at best, a snapshot of one specific capability under one specific test condition. It is roughly as predictive of real-world performance as a doctor's diagnosis based on a single blood pressure reading taken while you were sprinting up the stairs.

This book is about closing that gap. And we begin, naturally, with the most basic question of all: What is AI evaluation, really, and why does it require an entire book (and an entire profession) to do it well?

If you have ever thought "AI evaluation just means running benchmarks, right?" — this chapter is for you. If you have ever shipped an AI product and been surprised by how it behaved in production — this chapter is definitely for you. And if you have ever wondered why tech giants spend hundreds of millions of dollars a year on evaluation teams while most startups barely hire a single evaluation engineer — this chapter will explain exactly where that money goes and why the gap is dangerous.

---

1.1 评测不是测试：一场关于"体检"的认知革命

1.1.1 测试 (Testing) 与评测 (Evaluation) 的根本差异

Let's start by clearing up a confusion that trips up almost every beginner: testing and evaluation are not the same thing.

In traditional software engineering, the two words are often used interchangeably. You write a test case, you run it, you assert that `2 + 2 == 4`, you move on. Software is deterministic (确定性的): given the same input and the same code, you get the same output every time. A unit test (单元测试) that passes today will pass tomorrow, next month, and next year, unless someone changes the code.

AI systems, particularly those built on large language models, are non-deterministic (非确定性的) in ways that break this assumption entirely. Run the same prompt through GPT-4 twice, and you may get two different answers. Run it a thousand times, and you might see ten different patterns of behavior. The "code" is not lines of Python; it is hundreds of billions of numerical weights that encode statistical patterns learned from text. There is no `if-then` statement you can point to and say, "This is where the model decided to hallucinate a fake court case."

This is why the metaphor of a medical checkup (体检) is so much more accurate than the metaphor of a driving test.

1.1.2 体检 vs 驾照考试：两个截然不同的范式

A driving test (驾照考试) is a single-event, pass/fail assessment. You show up on a Tuesday morning, you parallel-park without hitting the cone, you signal before changing lanes, you pass or you fail. There is one attempt, one evaluator, and one outcome. If you fail, you can retake the test in three months.

An annual physical exam (年度体检) is something else entirely. It is:

• **Continuous (持续的)**: not a one-time event but a recurring measurement.
• **Multi-dimensional (多维度的)**: blood pressure, cholesterol, blood sugar, BMI, vision, hearing, reflexes, mental health screening — dozens of independent signals.
• **Trend-based (基于趋势的)**: a single reading of 130/85 blood pressure means little; but if your reading was 110/70 last year, 120/78 this year, and 130/85 next year, your doctor will be very alarmed. The trajectory matters more than any single point.
• **Comparative (对比性的)**: your numbers are compared against population baselines, your own history, your demographic peers, and risk thresholds.
• **Decision-driving (驱动决策的)**: the results don't just sit in a file. They trigger recommendations — eat less salt, walk 30 minutes a day, schedule a follow-up with a cardiologist.

AI evaluation is the medical checkup, not the driving test.

When your model scores 92% on MMLU, that is one data point on one dimension under one specific test condition. It tells you almost nothing about whether your model will:

• Refuse a perfectly reasonable question because it triggered an overly aggressive safety filter
• Make up a fake academic paper when asked for citations
• Behave differently when prompted in a low-resource language
• Cost you $400,000 in API fees next month because users found a clever way to make it generate 10,000 tokens per response
• Pass a regulatory audit next year in the EU under the AI Act

A serious AI evaluation framework gives you all of those answers — and many more you haven't thought of yet.

1.1.3 为什么大公司愿意花数百万养评测团队

Now let's address the elephant in the room: why do companies like OpenAI, Anthropic, Google DeepMind, and Meta spend eye-watering sums on evaluation?

The naive answer is "compliance" or "safety theater" or "marketing." The honest answer is more interesting. These companies have learned, through expensive mistakes, that evaluation is the only mechanism that scales product judgment.

Consider the scale problem. A frontier model might behave acceptably on 99.9% of inputs. But if it serves 100 million users and processes a billion queries per day, that 0.1% is a million failures per day. Without systematic evaluation, you cannot: 1. Know what percentage of failures you are currently experiencing 2. Tell whether a new training run made things better or worse 3. Detect regressions (回退) — cases where a new version got worse on something the old version handled well 4. Compare two candidate models for a product decision 5. Justify to regulators, investors, or the board why the model is "safe enough" to ship

In other words, evaluation is the difference between "I think the model is good" and "I know what the model does, on which dimensions, for which users, under which conditions."

This distinction — between intuition and measurement — is worth billions of dollars per year in the AI industry. It is also the entire reason this book exists.

---

1.2 评测的三个层次：能力、安全、对齐

Once you accept that AI evaluation is multi-dimensional, the next question is: what are those dimensions?

After several years of industry convergence, the field has roughly settled on three top-level categories. I call them the three layers of AI evaluation, and you'll see this framework echoed everywhere from academic papers to internal company memos to government AI policy documents.

1.2.1 第一层：能力评测 (Capability Evaluation)

The first and most intuitive layer is capability: can the model actually do the thing you want it to do?

This is the layer that gets the most public attention, because it is what leaderboards measure. Classic capability benchmarks include:

• **MMLU (Massive Multitask Language Understanding)**: 57 subjects ranging from elementary math to professional law. A model that scores high here is generally "broadly knowledgeable."
• **HumanEval / MBPP / SWE-bench**: programming benchmarks of increasing realism. SWE-bench, in particular, asks the model to fix real GitHub issues in real codebases.
• **GSM8K / MATH**: grade-school and competition-level math.
• **HellaSwag / WinoGrande / PIQA**: common-sense reasoning.
• **MGSM / XCOPA / XStoryCloze**: multilingual reasoning.

But capability evaluation is much broader than "run a benchmark." A serious capability evaluation for a real product might include:

• **Domain-specific tasks**: "Can our model correctly classify the urgency of a customer support ticket in Portuguese?"
• **Long-context tasks**: "When given a 200-page contract, can it find the termination clause?"
• **Tool-use tasks**: "Can it correctly call our internal SQL database to answer a sales q