Introduction to Federated Learning: Principles and Landscape

The core premise of traditional centralized machine learning (ML) relies on aggregating all raw training data into a single, controlled server environment, where model training, validation, and deployment are centrally managed. While this paradigm delivers high model performance and streamlined operational workflows, it introduces severe and increasingly regulated privacy risks, particularly for sensitive data domains such as healthcare, finance, and public services. For example, a 2019 audit of 127 healthcare ML systems found that 89% of centralized patient data training pipelines had experienced at least one unauthorized data access incident over a 3-year period, with an average breach cost of $6.5 million per incident according to IBM’s 2023 Cost of a Data Breach Report. In the financial sector, a 2021 breach at a major Southeast Asian bank exposed 21 million customer credit records, which were originally collected for centralized credit scoring model training, resulting in regulatory fines totaling $220 million under the region’s personal data protection laws. These incidents highlight a fundamental contradiction in centralized ML: the more valuable the model, the larger the aggregated dataset required, and the higher the privacy risk and compliance cost for data custodians.

<定义>联邦学习（Federated Learning, FL）</定义>由Google Research团队于2016年首次正式提出，其核心设计目标是在不集中收集原始训练数据的前提下，实现跨参与方的协同机器学习模型训练。其核心原则可以概括为三点：第一，数据最小化，仅传输模型训练所需的中间参数（如梯度、权重更新），原始数据永久保留在本地参与方的控制范围内；第二，参与方自治，每个参与方可以自主决定是否参与训练、何时退出训练，以及如何使用训练得到的全局模型；第三，隐私 by design，将隐私防护机制嵌入训练全流程，而非事后补充。与同一机构内部的分布式训练（数据属于同一控制实体，仅通过并行计算提升训练效率）不同，联邦学习的核心场景是跨机构的、互不信任的参与方之间的协同，其隐私防护机制是系统设计的核心组成部分，而非可选附加功能。

To formalize the core workflow of federated learning, we define the standard horizontal federated learning training loop as follows. Let $K$ denote the set of participating clients, where each client $k \in K$ holds a local dataset $\mathcal{D}_k$ of size $n_k$, with total sample size $n = \sum_{k=1}^K n_k$. The global model parameter is denoted as $\theta$, and the local loss function of client $k$ is $F_k(\theta) = \frac{1}{n_k} \sum_{i \in \mathcal{D}_k} f_i(\theta)$, where $f_i(\theta)$ is the sample-level loss for the $i$-th sample in client $k$’s dataset. The global optimization objective of federated learning is to minimize the weighted sum of local loss functions: $$\min_{\theta} F(\theta) = \sum_{k=1}^K \frac{n_k}{n} F_k(\theta)$$ The standard FedAvg algorithm, the most widely used baseline for horizontal federated learning, implements this objective via iterative parameter aggregation: in each round $t$, the server selects a subset of clients $S_t \subseteq K$, distributes the current global parameter $\theta^t$, each selected client runs $E$ epochs of local SGD training to get $\theta_k^{t+1}$, and the server aggregates the client updates to get the new global parameter $\theta^{t+1} = \sum_{k \in S_t} \frac{n_k}{\sum_{i \in S_t} n_i} \theta_k^{t+1}$. This workflow ensures that raw data never leaves the client’s local environment, while the global model benefits from the combined knowledge of all participating parties.

The technical evolution of federated learning can be divided into four distinct stages since its initial proposal. The first stage (2016-2017) was the academic incubation phase, marked by Google’s 2016 publication of the seminal *Communication-Efficient Learning of Deep Networks from Decentralized Data* paper, which introduced the FedAvg algorithm and demonstrated its effectiveness on mobile keyboard input prediction tasks. In 2017, Google launched the first production-grade federated learning system for Gboard, its mobile input method, serving over 1 billion active users with on-device training. The second stage (2018-2019) was the open-source ecosystem construction phase: in 2018, the OpenMined project released PySyft, the first open-source federated learning framework for Python, lowering the barrier for academic research; in 2019, WeBank (China) released the FATE (Federated AI Technology Enabler) framework, the first enterprise-grade open-source federated learning platform supporting both horizontal and vertical federated learning; also in 2019, the IEEE Standards Association launched the Federated Machine Learning Working Group to develop industry standards. The third stage (2020-2022) was the industry落地 (landing) phase: in 2020, NIST released the *NIST Privacy Framework* which explicitly included federated learning as a privacy-enhancing technology for cross-organizational data collaboration; in 2021, the International Organization for Standardization (ISO) published the first technical report on federated learning, laying the foundation for international standardization; during this period, federated learning was piloted in large-scale production scenarios including medical diagnosis, cross-bank risk control, and industrial equipment fault prediction. The fourth stage (2023-present) is the large-scale standardization and large model adaptation phase: in 2023, ISO officially released the first international standard for federated learning systems *ISO/IEC 23643:2023*; at the same time, the industry began exploring federated learning for large language model (LLM) training, solving the problem of cross-organizational data silos for LLM pre-training and fine-tuning.

The core paradigm of federated learning is divided into three categories based on the distribution of data samples and features across participating parties, each with distinct applicable scenarios and technical boundaries. <术语>横向联邦学习（Horizontal Federated Learning）</术语> is the earliest and most widely used paradigm, applicable when the feature spaces of all participating parties are identical, but the sample ID spaces do not overlap. For example, multiple banks in the same region all have customer features including age, income, credit history, and transaction records, but the customer bases of different banks have little overlap. In this scenario, horizontal federated learning can be used to jointly train a credit risk prediction model without sharing raw customer data. A typical production case is the cross-bank risk control federation launched by WeBank in 2021, which included 12 city commercial banks in the Yangtze River Delta region. Each bank held about 1 million historical customer credit samples, and through horizontal federated learning, the joint trained XGBoost risk model achieved an AUC of 0.89, a 21% improvement over the best single-bank model (AUC 0.73). The technical boundary of horizontal federated learning is strict: first, the feature spaces must be completely consistent, otherwise the model cannot be aligned; second, the sample ID spaces must have no overlap. Research from Stanford University in 2021 showed that if there is a 10% overlap in sample IDs between horizontal federated learning participants, a simple membership inference attack can recover 82% of the original sample features from the uploaded gradients, completely breaking privacy protection. Therefore, before deploying horizontal federated learning, it is necessary to perform strict sample ID deduplication and overlap detection.

<术语>纵向联邦学习（Vertical Federated Learning）</术语> is applicable when the sample ID spaces of participating parties are aligned, but the feature spaces do not overlap. The most typical scenario is medical multi-institutional collaboration: a tertiary hospital holds patient clinical features such as age, medical history, and examination results, while community hospitals hold patient medication records and follow-up data, and the two parties share the same patient population but have non-overlapping features. A real production case is the federated diabetes prediction project jointly launched by a top-tier hospital in Shanghai and 8 community hospitals in 2022. The two parties used RSA-based privacy-preserving intersection to align patient IDs without exposing raw ID information, and then used vertical federated le