联邦学习概论 | Introduction to Federated Learning

联邦学习概论

1 引言

The era of big data has witnessed an unprecedented proliferation of data generated from mobile devices, Internet of Things (IoT) sensors, healthcare systems, and financial institutions. This massive volume of data, however, resides in isolated silos, creating significant barriers to collaborative analysis and model training. Traditional machine learning approaches require centralized data collection, which often conflicts with privacy regulations, competitive advantages, and organizational boundaries. In this context, federated learning (FL) emerges as a revolutionary paradigm that enables multiple parties to collaboratively train machine learning models without sharing raw data, thereby preserving data privacy while leveraging collective intelligence.

The concept of federated learning was first introduced by Google researchers in 2017 as a solution to training machine learning models on distributed mobile devices. Since then, it has evolved into a comprehensive research field encompassing privacy-preserving machine learning, secure distributed optimization, and incentive mechanism design. The fundamental principle underlying federated learning is deceptively simple yet profoundly impactful: instead of bringing data to the model, we bring the model to the data. This paradigm shift addresses the core tension between the utility of data and the necessity of privacy protection.

This chapter provides a comprehensive introduction to federated learning, establishing the conceptual foundation upon which the subsequent chapters will build. We begin by presenting a rigorous definition of federated learning, followed by an exploration of its historical development and technological evolution. The chapter then examines the relationships between federated learning and adjacent fields such as distributed machine learning and privacy-preserving computation. We further analyze the unique value proposition of federated learning in various application scenarios, including privacy protection, data silo breaking, and edge computing. Finally, we provide a roadmap of the entire book, preparing readers for the deep dive into federated learning technologies that follows.

2 联邦学习的定义与核心特征

2.1 形式化定义

The formal definition of federated learning extends beyond its colloquial description as "bringing models to data." A precise mathematical formulation reveals the intricate interplay between model optimization, privacy constraints, and communication efficiency that defines the field. Let us consider a federated learning system comprising $N$ clients, denoted as $\mathcal{N} = \{1, 2, \ldots, N\}$. Each client $i$ possesses a local dataset $\mathcal{D}_i$ consisting of $n_i$ data samples, where the total dataset is $\mathcal{D} = \bigcup_{i=1}^{N} \mathcal{D}_i$ with $|\mathcal{D}| = \sum_{i=1}^{N} n_i$. The objective of federated learning is to learn a global model parameter $\mathbf{w}^*$ that minimizes the empirical risk over the entire dataset:

$$\mathbf{w}^* = \arg\min_{\mathbf{w} \in \mathbb{R}^d} \sum_{i=1}^{N} \frac{n_i}{|\mathcal{D}|} F_i(\mathbf{w}) \tag{1.1}$$

where $F_i(\mathbf{w}) = \frac{1}{n_i} \sum_{(\mathbf{x}_j, y_j) \in \mathcal{D}_i} \ell(\mathbf{w}; \mathbf{x}_j, y_j)$ is the local empirical risk function for client $i$, and $\ell(\cdot)$ is a loss function appropriate to the learning task.

The federated optimization problem (1.1) is complicated by three fundamental constraints that distinguish it from conventional distributed optimization:

Data Locality Constraint. The raw data $\mathcal{D}_i$ must remain on client $i$ throughout the training process. Clients can only share model updates—typically gradients or model parameters—with a central server or among themselves, but never expose their training samples directly.

Communication Constraint. Communication bandwidth, particularly in wireless networks, is typically orders of magnitude slower than local computation. Federated learning algorithms must therefore minimize communication rounds while maintaining model convergence and accuracy.

Statistical Heterogeneity. The local datasets $\mathcal{D}_i$ are typically non-identically distributed (non-IID) across clients. The distribution $P_i(\mathbf{x}, y)$ may vary significantly from the global distribution $P(\mathbf{x}, y)$, leading to what is known as the "statistical heterogeneity problem" in federated learning.

These constraints collectively define the unique challenges and opportunities in federated learning research. The FedAvg algorithm, proposed by McMahan et al. (2017), addresses these challenges through an iterative procedure that alternates between local computation and communication aggregation.

2.2 核心特征分析

联邦学习作为一种新兴的机器学习范式，其核心特征可以从技术、隐私和应用三个维度进行深入分析。这些特征相互关联，共同构成了联邦学习区别于传统机器学习方法的本质属性。

去中心化协作机制构成了联邦学习的技术基础。在传统的集中式机器学习中，所有数据必须汇聚到一个中心节点进行模型训练。这种架构不仅带来了单点故障风险，还造成了数据通信的瓶颈。相比之下，联邦学习采用去中心化的协作机制，允许多个参与方在不暴露原始数据的前提下共同优化模型。中心服务器仅负责模型参数的聚合，而不直接访问或存储任何参与方的训练数据。这种架构设计从根本上改变了机器学习的计算范式，使得跨机构、跨地域的协同建模成为可能。

隐私保护内嵌机制是联邦学习区别于其他分布式学习方法的显著特征。联邦学习并非简单地依赖加密技术来实现隐私保护，而是将隐私保护理念深度嵌入到算法的每一个环节。具体而言，联邦学习的隐私保护机制包含多个层次：数据始终保持在本地，这是最根本的隐私保护措施；参与方仅需传输模型参数或梯度，而非原始数据，这大幅减少了敏感信息的暴露面；可选地引入差分隐私（Differential Privacy）技术，通过向模型更新添加噪声来提供数学可证明的隐私保证；此外，还可以结合安全多方计算（Secure Multi-Party Computation, MPC）和同态加密（Homomorphic Encryption, HE）来进一步增强安全性。

通信高效性设计反映了联邦学习对现实网络环境的适应性考虑。由于联邦学习系统中的客户端通常是移动设备或边缘节点，其网络带宽和稳定性往往难以保证。因此，联邦学习算法必须在通信效率上做出精心设计。实践表明，在同等模型性能条件下，将通信次数减少一个数量级所带来的价值，远超将本地计算速度提升一个数量级。这一认识促使研究者们开发出多种通信压缩技术，包括梯度稀疏化、量化、Sketching等方法，显著降低了联邦学习系统的通信开销。

个性化定制能力是联邦学习在应对异构性问题时展现出的独特优势。现实世界的联邦学习系统面临严峻的数据异构性挑战：不同参与方的数据分布可能存在显著差异，这种非独立同分布（Non-IID）的数据特性会导致全局模型在某些参与方上的性能严重下降。传统的集中式学习方法难以有效处理这一问题，而联邦学习通过引入个性化机制——如联邦多任务学习、联邦元学习、以及基于知识蒸馏的个性化方法——能够为每个参与方训练出适配其本地数据分布的个性化模型，同时保留协作学习带来的性能提升。

2.3 联邦学习系统架构

联邦学习系统的架构设计遵循经典的客户端-服务器范式，但在具体实现上呈现出丰富的变体形态。理解这一架构对于把握联邦学习的技术本质至关重要。

```mermaid graph TB subgraph "客户端层 Client Layer" C1[客户端 1<br/>数据 D₁] C2[客户端 2<br/>数据 D₂] CN[客户端 N<br/>数据 Dₙ] end

subgraph "通信层 Communication Layer" COM[安全通信通道<br/>加密传输] end

subgraph "服务器层 Server Layer" COORD[协调服务器<br/>聚合算法] SEC[安全计算模块] MON[监控与调度] end

C1 -->|本地训练| COM C2 -->|本地训练| COM CN -->|本地训练| COM

COM --> COORD COORD --> SEC COORD --> MON

COORD -->|全局模型| COM ```

Figure 1.1: 联邦学习系统架构示意图 / Federated Learning System Architecture

如 Figure 1.1 所示，一个典型的联邦学习系统包含三个核心层次：客户端层、通信层和服务器层。客户端层由 $N$ 个参与方组成，每个参与方持有本地数据集 $\mathcal{D}_i$ 和本地计算资源。客户端负责执行本地模型训练，根据接收到的全局模型参数 $\mathbf{w}^{(t)}$ 进行梯度计算或模型更新，并将结果上传至服务器。通信层负责在客户端和服务器之间传输模型参数、控制信息等数据。该层通常采用加密协议来保障传输安全性，可选的加密技术包括传输层安全协议（TLS）、端到端加密（E2EE）等。服务器层承担模型聚合、任务调度和系统监控等核心功能。协调服务器执行聚合算法（如 FedAvg），将来自多个客户端的本地更新整合为新的全局模型，并分发给各客户端参与下一轮训练。

值得注意的是，联邦学习的系统架构并非一成不变。根据应用场景和隐私需求的差异，实际系统可能采用不同的拓扑结构。集中式架构（Centralized Architecture）采用单一中心服务器进行模型聚合，这是当前最广泛采用的架构形式。然而，该架构存在中心节点单点故障和性能瓶颈问题。去中心化架构（Decentralized Architecture）通过点对点（Peer-to-Peer, P2P）通信替代中心聚合，每个客户端直接与其他客户端交换模型更新，避免了对中心服务器的依赖。这种架构具有更好的容错性和可扩展性，但通信协议的设计更为复杂。垂直联邦学习架构（Vertical Federated Learning Architecture）则针对数据垂直切分场景设计，不同参与方持有同一批样本的不同特征，聚合过程需要特殊的对齐和保护机制。

3 发展历程与研究演进

3.1 起源与早期发展

The genesis of federated learning can be traced back to the growing tension between the insatiable appetite of machine learning algorithms for data and the increasingly stringent regulations on data privacy. While the term "federated learning" was formally coined by Google researchers in 2016-2017, the underlying concepts found their roots in earlier research on distributed optimization and privacy-preserving data mining.

The publication of the FedAvg algorithm by McMahan et al. in 2017 marked a watershed moment in the history of federated learning. The algorithm introduced a remarkably simple yet effective approach: clients perform multiple epochs of local stochastic gradient descent (SGD) on their data, and the server performs a weighted avera