The problem centralized learning cannot solve
Picture a consortium of banks that all process UPI transactions — say State Bank of India, HDFC, and ICICI — each seeing several billion rupees of payment activity a month. A fraud-detection model trained on the pooled transaction history of all three banks would almost certainly outperform any single bank's model: fraud rings rarely restrict themselves to one bank, and a pattern too rare to detect in Bank A's data alone might be obvious once you have Bank C's examples too. The obvious approach — copy every bank's transaction ledger to one server and train a model on the union — is the approach every one of these institutions is legally and competitively forbidden from taking. RBI data-localization and customer-privacy norms restrict how transaction data can move between institutions, and no bank wants a rival reading its ledger even if regulation allowed it.
This is precisely the situation federated learning (FL) was built for: many parties, each holding data too sensitive or too regulated to centralize, who nonetheless want the statistical power of training on everyone's data combined. The insight that makes it possible is a reversal of the usual machine-learning pipeline. Instead of moving data to the model, FL moves the model to the data. Each bank keeps its ledger exactly where it is, trains a copy of a shared model on its own transactions, and sends back only the trained model's numbers — weights, not rows. A coordinating server averages those numbers into an improved shared model and sends it back out for another round. No transaction ever leaves the institution that owns it.
From data-to-model to model-to-data: first principles
You already know the standard supervised-learning loop from your deep-learning coursework: gather a dataset, define a loss function, run gradient descent to find weights that minimize it. Federated learning keeps every one of those pieces — same loss functions, same gradient descent, same neural network or linear model architectures — and changes exactly one structural fact: the dataset is partitioned across K clients (call them C1 … CK, here the three banks), no client can see another client's partition, and there is a central server that can see none of the raw partitions either.
Three properties of this setting matter enormously for how the algorithm must behave, and each has a direct counterpart in the HTTP client-server model you have already studied:
Statistical heterogeneity (non-IID data). A bank serving mostly corporate accounts sees a very different transaction distribution than one serving mostly small retail UPI payments. Unlike the mini-batches you shuffle randomly out of one training set, each client's local data is not a representative sample of the global distribution — it is systematically skewed.
Expensive, unreliable communication. Sending a full model's weights over a network round-trip is far more costly than a single forward pass on local hardware. In the cross-device setting (millions of phones, like Google's Gboard keyboard-prediction system, the original production use case for this algorithm), most clients might be offline or on metered connections at any given round. In the cross-silo setting we are using — a handful of powerful, reliably-online institutions like banks or hospitals — the constraint is lighter but not absent, since coordinating training across three IT departments still means every round has friction and latency that a single in-house GPU cluster would not.
No trust in either direction. The server cannot be handed raw data even if a client wanted to, and a client cannot be assumed to send an honest, uncorrupted update. This is the part with no analogue in your DBMS or DSA coursework, and it is why FL is a genuinely distinct subfield rather than "distributed training with extra steps."
The FedAvg algorithm
The algorithm almost all federated learning systems build on is Federated Averaging (FedAvg), introduced by McMahan and colleagues in 2017. One communication round t proceeds as follows:
1. The server holds a current global weight vector w_t and broadcasts it to a set of participating clients.
2. Each client k, holding n_k local examples, initializes its local model at w_t and runs E epochs of ordinary gradient descent (or SGD, if it further splits its local data into mini-batches) using only its own data, producing a locally updated weight w_t^k.
3. Each client sends w_t^k back to the server — a vector of numbers the same shape as the model, never a data row.
4. The server aggregates by a weighted average, proportional to how much data each client trained on:
w(t+1) = Σ_k (n_k / n) · w_t^k, where n = Σ_k n_k
The weighting by n_k / n rather than a plain 1/K average is not cosmetic. As the worked example below proves by direct computation, when each client runs exactly one full-batch gradient step (E = 1), this weighted average is mathematically identical to what you would get by pooling every client's data on one machine and taking a single centralized gradient step. Weighting protects that equivalence; an unweighted average would let a client with 5 data points sway the model exactly as much as one with 5,000.
Worked example: one round of FedAvg by hand
Take a deliberately tiny linear model so every arithmetic step is checkable: a fraud score predictor f(x) = w·x with a single weight w (no bias term), where x is some anomaly score NPCI-style risk engines already compute per transaction and y is the true fraud label. The loss is mean squared error, L = (1/n) Σ (w·x_i − y_i)², so the gradient is dL/dw = (2/n) Σ (w·x_i − y_i)·x_i. Three banks hold three tiny local datasets, and the global weight starts at w_0 = 1.0 with learning rate η = 0.05.
Bank A (n_A = 2): points (x, y) = (2, 3), (4, 5).
Predictions at w = 1.0: 2, 4. Errors (pred − y): −1, −1.
Σ(error·x) = (−1·2) + (−1·4) = −6.
Gradient = (2/2)·(−6) = −6.0.
Local update: w_A = 1.0 − 0.05·(−6) = 1.0 + 0.30 = 1.30.
Bank B (n_B = 3): points (1, 1), (2, 2), (3, 3) — a perfect y = x pattern.
Predictions at w = 1.0: 1, 2, 3. Errors: 0, 0, 0. Gradient = 0.
Local update: w_B = 1.0 − 0.05·0 = 1.00.
Bank C (n_C = 5): points (2, 1), (2, 1), (4, 2), (4, 2), (5, 1).
Predictions at w = 1.0: 2, 2, 4, 4, 5. Errors: 1, 1, 2, 2, 4.
Σ(error·x) = (1·2)+(1·2)+(2·4)+(2·4)+(4·5) = 2+2+8+8+20 = 40.
Gradient = (2/5)·40 = 16.0.
Local update: w_C = 1.0 − 0.05·16 = 1.0 − 0.80 = 0.20.
The server has never seen any of these six (x, y) pairs — only the three numbers 1.30, 1.00, 0.20, tagged with each bank's sample count. It aggregates:
n = 2 + 3 + 5 = 10
w(1) = (2/10)(1.30) + (3/10)(1.00) + (5/10)(0.20)
= 0.26 + 0.30 + 0.10 = 0.66
Now check the claimed equivalence: if instead all ten points had been pooled onto one machine and a single centralized gradient step taken, what would happen? Sum the three banks' unnormalized error terms — Σ(error·x) — directly: −6 (A) + 0 (B) + 40 (C) = 34. Centralized gradient = (2/10)·34 = 6.8. Centralized update: w = 1.0 − 0.05·6.8 = 1.0 − 0.34 = 0.66. It matches exactly. This is not a coincidence — it follows algebraically from the fact that a sum of per-client gradients, each already scaled by n_k, is just the pooled-dataset gradient rearranged; weighting the local updates by n_k/n before averaging is what reconstructs it. This equivalence is the entire justification for why FedAvg with E = 1 is a correct, unbiased distributed implementation of ordinary gradient descent, not an approximation of it.
Verifying with code
The same computation, run rather than hand-traced, should reproduce every number above exactly:
import numpy as np
# Local datasets (anomaly_score, is_fraud) held privately by each bank
bank_data = {
"Bank A": (np.array([2, 4]), np.array([3, 5])),
"Bank B": (np.array([1, 2, 3]), np.array([1, 2, 3])),
"Bank C": (np.array([2, 2, 4, 4, 5]), np.array([1, 1, 2, 2, 1])),
}
def local_gradient(w, x, y):
n = len(x)
error = w * x - y
return (2 / n) * np.sum(error * x)
def local_update(w, x, y, lr):
return w - lr * local_gradient(w, x, y)
w_global = 1.0
lr = 0.05
local_weights, sizes = {}, {}
for bank, (x, y) in bank_data.items():
sizes[bank] = len(x)
local_weights[bank] = local_update(w_global, x, y, lr)
print(f"{bank}: n={sizes[bank]}, grad={local_gradient(w_global, x, y):.2f}, "
f"w_local={local_weights[bank]:.2f}")
n_total = sum(sizes.values())
w_new = sum(sizes[b] / n_total * local_weights[b] for b in bank_data)
print(f"n_total={n_total}, aggregated global weight w1={w_new:.4f}")Tracing it: local_gradient for Bank A computes error = [1·2−3, 1·4−5] = [−1,−1], then (2/2)·Σ(error·x) = 1·(−2−4) = −6.00, so w_local = 1.0 − 0.05·(−6) = 1.30. Bank B's error vector is all zeros so gradient prints as 0.00 and w_local=1.00. Bank C's error vector is [1,1,2,2,4], giving gradient 16.00 and w_local=0.20. The final line computes n_total = 10 and w_new = 0.2·1.30 + 0.3·1.00 + 0.5·0.20 = 0.6600, printed as w1=0.6600 — matching the hand derivation to the last digit.
Why more local epochs is not free: client drift and non-IID data
The clean equivalence proved above — FedAvg with E = 1 equals one step of centralized gradient descent — is exactly what you would want if communication were free, because you could just run E = 1 every round and get provably identical behaviour to training on one machine. Communication is not free, though, especially in cross-device FL with millions of intermittently-connected clients, so real systems set E > 1: each client takes several local gradient steps before reporting back, cutting the number of network round-trips needed to reach a given accuracy.
The cost is that the equivalence breaks the moment E exceeds 1. After a client's first local step, its weight has already moved along a path shaped entirely by its own data. If Bank A's transactions are mostly high-value corporate transfers and Bank C's are mostly small retail payments, their local loss surfaces point in different directions, and by the second, third, or fifth local step each bank's model has been pulled further toward a "local optimum" that fits its own skewed slice of the world and increasingly diverges from what a model trained on the pooled data would look like. Averaging five specialists that have each drifted toward their own corner of weight-space is not the same as averaging five clients that each took one honest step toward the global optimum. This phenomenon is called client drift, and it is the central open problem in FL research on heterogeneous (non-IID) data. Two well-known fixes are FedProx (Li et al., 2020), which adds a penalty term to each client's local loss that discourages it from moving too far from the broadcast weight w_t, and SCAFFOLD (Karimireddy et al., 2020), which has the server and each client track "control variates" that correct the local gradient direction toward what the true global gradient would have been. Both trade some implementation complexity for keeping FedAvg's guarantees intact when E > 1.
This heterogeneity axis is also why FL literature distinguishes cross-silo FL (our bank/hospital example: a handful of powerful, reliably-online institutions, each with a large local dataset) from cross-device FL (millions of phones or IoT devices, each with a tiny, highly non-IID dataset, and most offline at any given moment, as in Google's original Gboard next-word-prediction deployment). The algorithm is the same; the engineering constraints around client sampling, dropout tolerance, and communication compression differ enormously between the two regimes.
Common misconception: "federated learning automatically guarantees privacy"
The single most common error a student makes on first meeting FL is assuming that because raw data never leaves the client, privacy is solved. It is not. The weight vector w_t^k that Bank A sends back was computed from Bank A's actual transactions, and a sufficiently determined analysis of that vector can leak information about them. In the worked example, Bank B's local update (w_B = 1.00, unchanged from w_0) by itself reveals that Bank B's local gradient was exactly zero — meaning its data already fit the model perfectly, a real fact about its dataset extracted without ever seeing a single transaction. At the scale of a real neural network, this kind of leakage is far more dangerous: gradient-inversion attacks (Zhu, Liu, and Han, "Deep Leakage from Gradients," 2019) show that under some conditions an attacker holding a single client's gradient update can reconstruct individual training images or text near-exactly, and membership-inference attacks can determine whether a specific record was in a client's training set at all, purely from how the model's outputs shift.
What FL actually guarantees is narrower and worth stating precisely: raw data never leaves its owner's infrastructure, and the server's computation is structurally restricted to operating on aggregated model parameters rather than individual records. That is a meaningful and often legally significant guarantee — it satisfies many data-localization requirements outright — but it is not the same as a cryptographic or information-theoretic privacy guarantee against a server or eavesdropper analyzing what those parameters reveal. Two techniques close that gap, and production FL systems typically combine both. Secure aggregation (Bonawitz et al., 2017) uses cryptographic secret-sharing so that the server can only ever recover the sum of masked client updates once enough clients have responded — no individual client's update is ever visible to the server, even transiently. Differential privacy adds carefully calibrated random noise to each client's update before it is sent (clipping the update's magnitude, then adding Gaussian noise), giving a quantifiable bound, expressed as a privacy budget ε, on how much any single training example could have changed the published result — at the cost of some model accuracy. Federated learning without either of these is "data minimization," a real and useful property; calling it "privacy-preserving" without them overstates what has actually been proven.
Active recall
Attempt each question before reading its answer.
- If Bank B in the worked example had held n_B = 30 transactions instead of 3 (ten times the data, same zero gradient), what would the round-1 aggregated weight w(1) become?
- Why does FedAvg weight each client's local update by n_k/n instead of taking a plain average of the K local weights?
- What happens to the exact equivalence between FedAvg and centralized gradient descent once each client runs E = 5 local epochs instead of E = 1, and what is this problem called?
- A hospital consortium of five hospitals has local dataset sizes n = (50, 50, 50, 50, 800). Name one risk of this imbalance and one way to mitigate it.
- True or false: secure aggregation prevents the server from ever learning anything about individual client updates. Justify your answer.
- In the worked example, why does Bank C's local update pull the aggregated weight down so much more than Bank A's pulls it up, even though Bank A's gradient magnitude (6) is closer to Bank C's (16) than to Bank B's (0)?
Answers.
1. n_total = 2 + 30 + 5 = 37. w(1) = (2·1.30 + 30·1.00 + 5·0.20) / 37 = (2.6 + 30 + 1.0) / 37 = 33.6 / 37 ≈ 0.9081. Bank B's much larger dataset, even with a zero gradient, now dominates the aggregate and pulls the result close to its unchanged weight of 1.00.
2. Weighting by n_k/n is what makes the aggregate mathematically equal to the gradient of the full pooled dataset (proven directly in the worked example, where both routes gave exactly 0.66). An unweighted 1/K average would let a client with very little data sway the global model exactly as much as a client with orders of magnitude more, which is both statistically wrong (it does not reconstruct the pooled gradient) and unfair to whoever contributed the most information.
3. The equivalence breaks: after the first local step, each client's trajectory is shaped only by its own (possibly non-IID) data, so five clients each drift toward their own local optimum before being averaged, rather than each taking five honest steps toward the shared global optimum. This is called client drift, and is mitigated by algorithms such as FedProx and SCAFFOLD.
4. Risk: the model will be dominated by the 800-record hospital's weight (800/1000 = 80% of the aggregate), likely overfitting to its patient demographics, equipment calibration, or local protocols and underperforming for the four smaller hospitals — a fairness problem, not just an accuracy one. Mitigation: use a fairness-aware aggregation scheme (such as q-FedAvg) that reweights the objective to favor uniform improvement across clients rather than pure data-size weighting, or cap any single client's effective weight in the average.
5. True, within its stated threat model: the cryptographic masking is constructed so the server can only decode the sum across enough participating clients, never an individual term, as long as it does not collude with all-but-one of the clients. It is not an unconditional guarantee — a server that colludes with K−1 of K clients can isolate the remaining client's update by subtraction, and secure aggregation says nothing about what the final aggregated model itself might still reveal through gradient-inversion or membership-inference analysis across many rounds.
6. Both size and gradient magnitude matter, and Bank C has the larger of both. Bank C's weight in the average (5/10 = 50%) is more than double Bank A's (2/10 = 20%), and its per-example errors were also proportionally larger (a gradient of 16 versus −6). Its contribution to the final weighted sum, 0.5 × 0.20 = 0.10, and its distance moved from w_0 (0.80) both outweigh Bank A's 0.2 × 1.30 = 0.26 and its distance moved (0.30) — the product of client weight and per-client movement is what determines each bank's real influence on the round, not gradient magnitude alone.
Think About It
Think about this: How would you explain federated learning: distributed privacy-preserving training to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Practice Exercises
Now it is time to practice! Complete these challenges to solidify your understanding:
- Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
- Exercise 2: Find a real-world example where federated learning: distributed privacy-preserving training is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
- Exercise 3: Create a mind-map connecting federated learning: distributed privacy-preserving training to at least 3 other topics you have studied.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind federated learning: distributed privacy-preserving training, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.