Prologue: Two Frameworks, One Revolution

Why does the choice between PyTorch and TensorFlow shape not just your code, but your entire approach to deep learning? That is the central question this chapter sets out to answer, and it is more uncomfortable than it first appears because the answer is not "because of the syntax" or "because of the API design" or "because of the optimization tricks." The honest answer is that the choice leaks into everything: how you name variables, how you debug, how you collaborate with colleagues, how you onboard new hires, how you ship models to production, and even how you think about what a model *is*. To understand why, we need to begin not with a code snippet but with a moment in 2012 that quietly bifurcated the entire field.

为什么 PyTorch 和 TensorFlow 之间的选择会塑造的不仅仅是你的代码，而是你整个深度学习的方法？这是本章要回答的核心问题，而它的答案比表面看起来更让人不安——因为答案不是"因为语法"或"因为 API 设计"或"因为优化技巧"。诚实的答案是，这个选择会渗透到一切：如何命名变量，如何调试，如何与同事协作，如何培训新员工，如何将模型部署到生产环境，甚至影响你如何思考模型*是*什么。要理解这一点，我们不能从一段代码开始，而要从 2012 年那个悄然分叉了整个领域的历史时刻开始。

1. The 2012 ImageNet Moment as the Dawn of the Framework Era / 2012 年的 ImageNet 时刻：框架时代的黎明

In September 2012, a small team from the University of Toronto walked into the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) and won by a margin so wide that the organizers initially suspected a scoring error. Their entry, later published under the name AlexNet, achieved a top-5 error rate of 15.3% on the ILSVRC-2012 dataset of 1.2 million labeled images across 1000 object categories. The second-place entry, a traditional computer vision pipeline based on SIFT features and Fisher vectors, scored 26.2%. That gap of nearly eleven percentage points was, in a field where year-over-year improvements are typically measured in fractions of a percent, more like the difference between walking and flight than like an ordinary step forward. Within eighteen months, every major technology company had founded a deep learning group. Within three years, the "Cambrian explosion" of new architectures—VGG, GoogLeNet/Inception, ResNet, DenseNet, U-Net, GANs, sequence-to-sequence models—had transformed what had been a niche academic specialty into the dominant paradigm of applied machine learning.

【中译】2012 年 9 月，多伦多大学的一个小团队走进了 ImageNet 大规模视觉识别挑战赛（ILSVRC），并以巨大的优势获胜，以至于组织者最初怀疑评分出了错。他们的参赛作品后来以 AlexNet 的名义发表，在 ILSVRC-2012 数据集的 120 万张标注图像、1000 个类别上，取得了 15.3% 的 top-5 错误率。第二名是一个基于 SIFT 特征和 Fisher 向量的传统计算机视觉流水线，得分 26.2%。在年进步通常以零点几个百分点计量的领域，这近 11 个百分点的差距，更像是行走与飞行之间的区别，而不是普通的前进。十八个月内，所有主要科技公司都成立了深度学习小组。三年内，新架构的"寒武纪大爆发"——VGG、GoogLeNet/Inception、ResNet、DenseNet、U-Net、GAN、序列到序列模型——将曾经的小众学术专业转变为应用机器学习的主导范式。

The neural network Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton had trained was not architecturally novel. Convolutional networks had been described in Yann LeCun's work at Bell Labs in the late 1980s, and LeNet-5 had successfully read handwritten checks in the 1990s. What was new in 2012 was the convergence of three forces that had been waiting for each other: data (ImageNet's 1.2 million images, large enough to feed the statistical hunger of deep networks), compute (Krizhevsky implemented the network on two NVIDIA GTX 580 GPUs with 3 GB of memory each, a configuration that would today be considered a toy but at the time represented a clever workaround for memory limits through model parallelism across two devices), and engineering know-how (ReLU activations to mitigate vanishing gradients, dropout to prevent overfitting, overlapping pooling for slight accuracy gains, and aggressive data augmentation including image translations, horizontal reflections, and RGB color perturbations).

【中译】Alex Krizhevsky、Ilya Sutskever 和 Geoffrey Hinton 训练的神经网络在架构上并不新奇。卷积网络在 1980 年代末由 Yann LeCun 在贝尔实验室的工作中就已描述，LeNet-5 在 1990 年代已经能成功读取手写支票。2012 年的新意在于三股力量的汇合：数据（ImageNet 的 120 万张图像，规模足以满足深度网络的统计饥渴）、算力（Krizhevsky 在两块各 3GB 显存的 NVIDIA GTX 580 GPU 上实现该网络——以今天的标准只是玩具，但当时代表了一种通过跨设备模型并行来绕过显存限制的巧妙变通）以及工程经验（用 ReLU 激活函数缓解梯度消失，用 dropout 防止过拟合，用重叠池化获得微弱精度提升，以及包含图像平移、水平翻转和 RGB 颜色扰动的激进数据增强）。

But the most consequential outcome of AlexNet was not the paper itself; it was the realization that deep learning had become a software engineering discipline, not just a mathematical one. By 2014, writing these networks from scratch, in raw CUDA or in Theano-style symbolic frameworks, had become a bottleneck. Researchers spent more time fighting with their tools than thinking about their hypotheses. This recognition set the stage for the framework wars of 2015–2017.

【中译】但 AlexNet 最重要的结果不是论文本身，而是人们意识到深度学习已经成为一个软件工程学科，而不仅仅是数学学科。到 2014 年，从零开始用原始 CUDA 或 Theano 风格的符号框架编写这些网络已经成为瓶颈。研究人员与工具搏斗的时间超过了思考假设的时间。这一认知为 2015–2017 年的框架大战奠定了舞台。

The Pre-Framework World and Its Limits / 前框架世界及其局限

To understand why frameworks became necessary, it helps to remember what came before. In 2012, the most popular tools for neural network research were Theano (at the University of Montreal), a symbolic math library that required users to define computation as a graph and then compile it; academic Matlab code distributed as paper supplements; and a scattering of homegrown systems at major labs. Each had serious limitations. Theano was powerful but had a notoriously steep learning curve, error messages that often pointed at internal C++ code rather than user-facing Python, and a development pace that slowed to a crawl after 2014. Matlab was familiar to engineers but had limited GPU support and was not designed for the kind of large-scale gradient-based optimization that deep learning requires. Writing custom GPU kernels was feasible only for those with deep CUDA expertise, which excluded the vast majority of researchers.

【中译】要理解为什么框架变得必要，需要记住之前的状况。2012 年，神经网络研究最流行的工具是蒙特利尔大学的 Theano——一个要求用户将计算定义为图然后编译的符号数学库；作为论文附录分发的学术 Matlab 代码；以及主要实验室里零散的自研系统。每种工具都有严重局限。Theano 功能强大但学习曲线陡峭得声名狼藉，错误信息常指向内部 C++ 代码而非用户层的 Python，并且自 2014 年起开发节奏放缓到近乎停滞。Matlab 对工程师来说很熟悉，但 GPU 支持有限，并非为深度学习所需的大规模基于梯度的优化而设计。编写自定义 GPU 内核只有具备深厚 CUDA 专业知识的少数人才能做到，这排除了绝大多数研究者。

What changed between 2012 and 2016 was the realization that deep learning had become a software engineering discipline, not just a mathematical one. Three frameworks rose to fill the gap, each embodying a different philosophy about what deep learning code should look like. Caffe (released by Berkeley in 2014) was a C++/CUDA framework with a focus on computer vision and a "configuration-as-code" philosophy where networks were defined in `.prototxt` files. Caffe was fast and excellent for production but painful for research, because changing a network architecture meant editing a structured text file and recompiling support code. Torch (originating in 2002 but reborn in the Lua community in the 2010s) was a Lua-based framework that emphasized imperative, NumPy-like tensor operations. Torch was beloved by Facebook AI Research because it felt like Python and let researchers iterate at the speed of thought, but Lua itself was an exotic language that limited adoption outside a small community. TensorFlow, released by Google Brain in November 2015, adopted a static computation graph model inspired by Theano but extended it with a distributed runtime, automatic differentiation, and a sophisticated visualization tool called TensorBoard.

【中译】2012 到 2016 年间发生变化的是，人们认识到深度学习已经成为软件工程学科，而不仅仅是数学学科。三个框架崛起填补了这个空白，每个都体现了关于深度学习代码应该是什么样子的不同哲学。Caffe（2014 年由伯克利大学发布）是一个 C++/CUDA 框架，专注于计算机视觉并采取"配置即代码"的哲学——网络在 `.prototxt` 文件中定义。Caffe 速度快、生产出色，但研究痛苦，因为修改网络架构意味着编辑结构化文本文件并重新编译支持代码。Torch（起源于 2002 年，2010 年代在 Lua 社区重生）是一个基于 Lua 的框架，强调命令式的、类 NumPy 的张量操作。Torch 受到 Facebook AI Research 的青睐，因为它感觉像 Python，让研究者以思维的速度迭代，但 Lua 本身是一种奇异语言，限制了其在小社区之外的采用。TensorFlow 由 Google Brain 于 2015 年 11 月发布，采用受 Theano 启发的静态计算图模型，但用分布式运行时、自动微分和名为 TensorBoard 的复杂可视化工具对其进行了扩展。

PyTorch arrived in late 2016, with its first public release in January 2017, as a Python-native reimagining of Torch's philosophy: keep the imperative, NumPy-like tensor model, but rebuild everything around Python's object system, autograd through reverse-mode automatic differentiation of dynamic graphs, and an obsession with de