It is 8:40 a.m. on a Mumbai local train. A commuter opens Gboard to type a message. Her phone is at 23% battery, tethered to mobile data, screen brightly lit, being actively used. Somewhere in a Google data center, a federated learning server is running a training round for the next-word-prediction model that will suggest her next word tonight. Her phone will not participate in that round, or in any round today. It fails three separate eligibility checks at once: it is not charging, it is not idle, and it is not on an unmetered network. Nine hours later, the same phone is plugged into a wall socket at home, screen off, connected to home Wi-Fi. Within minutes it is selected into a round, downloads a global model checkpoint, runs a few steps of local training on the words she typed that day, uploads a compressed update, and disconnects. Nobody at Google ever saw her keystrokes. But just as important to the system that made this work: nobody scheduled her phone to train at 8:40 a.m., because doing so would have drained her battery, competed with her foreground app for CPU, and cost her mobile data. That scheduling decision, repeated across roughly a billion Android devices, is the actual engineering problem this chapter is about.
Two sibling chapters in this course cover federated learning from other angles: one works through FedAvg's convergence behavior under non-IID client data, another covers the attacks (model inversion, gradient leakage, poisoning) that federated systems must defend against. Neither asks the question this chapter asks: given that FedAvg is the algorithm and privacy is the goal, how do you actually run this on a fleet of consumer phones you do not control, that are online unpredictably, on batteries you cannot drain, over networks you do not own? Google's Gboard deployment is the best-documented answer to that question in the public literature, described across three papers you should know by name: Hard et al. (2018), Federated Learning for Mobile Keyboard Prediction; Bonawitz et al. (2019), Towards Federated Learning at Scale: System Design (SysML, now MLSys); and Bonawitz et al. (2017), Practical Secure Aggregation for Privacy-Preserving Machine Learning (ACM CCS). This chapter walks through the system those papers describe.
Cross-device is a different engineering regime than cross-silo
Federated learning splits into two regimes that share an algorithm but not an operating environment. Cross-silo federated learning, the kind a consortium of five hospitals might run to train a shared diagnostic model, involves a handful of participants, each a well-provisioned server, each reliably online, each holding a large and relatively stable local dataset. The hard problems there are statistical (how skewed is each hospital's patient population) and organizational (data-sharing agreements, audit trails). Cross-device federated learning, the kind Gboard runs, involves millions of participants, each an underpowered, battery-constrained, intermittently-connected phone, each holding a tiny and constantly-changing slice of data (today's typed words). A single device might appear in a training round once a week, if that. The hard problems here are almost entirely systems problems: which devices do you even get to hear from, how do you keep a training round from being blocked forever by phones that vanish mid-round, and how do you fit a meaningful model update through a mobile uplink without the user noticing.
Bonawitz et al. (2019) describe the production architecture built to solve this. The key insight is that participation cannot be requested, it must be volunteered by devices that check in when conditions are favorable and are then screened for eligibility. This inverts the usual distributed-training mental model: instead of a parameter server assigning work to workers, an FL server maintains a live population of currently-connected, currently-eligible devices and samples from whoever happens to be available at the moment a round opens.
The eligibility gate and the round lifecycle
A device becomes eligible to participate only when it satisfies, simultaneously: the device is idle (screen off, not in active use), the device is charging, and the device is connected to an unmetered network (Wi-Fi, not mobile data). A minimum battery level and a check that the device has been idle for some minimum duration are typically added on top. Every one of these conditions exists to make federated training invisible to the user: no perceptible battery drain, no data-plan cost, no competition with foreground app performance. This is the opposite design philosophy from a data-center training job, where you provision the hardware specifically to be maximally utilized; here the entire system is engineered around a hardware fleet you must not inconvenience.
Bonawitz et al. describe the resulting architecture as a small set of always-on server processes: a Selector, which maintains a pool of live connections to eligible, checked-in devices and decides which subset to invite into an active round, and a Coordinator, which drives a round through three phases:
Selection: the FL server determines it wants a round with roughly some target cohort size and asks the Selector to invite that many eligible devices to connect. Because a meaningful fraction of invited devices will drop out before finishing (screen turns on, charger unplugged, connectivity lost), the server systematically over-selects, inviting more devices than the target so the round still completes on time even after dropout.
Configuration: each connected device downloads the current global model checkpoint plus a training plan (learning rate, number of local epochs, batch size) and begins local training on its own on-device data, never leaving the device.
Reporting: devices that finish local training within the round's deadline upload their update; devices that do not (still training, or that dropped an eligibility condition mid-round) are simply excluded from that round's aggregate, with no retry and no penalty. Once enough updates have arrived or the deadline passes, the server closes the round, aggregates whatever arrived, and updates the global model.
This deadline-and-drop design is the load-bearing decision of the whole system. A traditional distributed-training job cannot tolerate a worker silently vanishing mid-step without some recovery protocol; a federated round treats vanishing as the expected case and is built to make progress anyway, because a phone's owner picking it up mid-round is far more common, and far less controllable, than a data-center node failing.
Diagram: one Gboard-style federated round
The application: what actually trains on the phone
Hard et al. (2018) describe the concrete model this pipeline trains: a compact recurrent network for next-word prediction in Gboard, a coupled input-forget-gate (CIFG) variant of an LSTM, chosen specifically because it is small enough to ship to a phone and cheap enough to fine-tune in the few seconds of idle-charging time a round allows. The training signal is entirely on-device: as the user types, completed words become labeled training examples (the preceding words as context, the completed word as target) that never leave the device. Only the resulting weight update, not any typed sentence, is what gets uploaded. The paper reports that the federated-trained model, evaluated through live A/B comparison against a model trained the conventional way on logged, centrally-collected data, was competitive with the centrally-trained baseline on next-word-prediction quality, demonstrating that the privacy-preserving pipeline is not merely a compliance exercise, it is a viable production training method.
Worked example: why the update has to be compressed
The eligibility gate exists because charging and idle time are precious; the unmetered-network requirement exists because upload bandwidth is precious too. Konečný et al. (2016), in a companion paper to FedAvg titled Federated Learning: Strategies for Improving Communication Efficiency (NeurIPS Workshop on Private Multi-Party Machine Learning), introduce two compression techniques for exactly this reason: structured updates, where the client restricts its update to a low-dimensional or sparse structure agreed with the server in advance, and sketched updates, where a full update is computed and then probabilistically compressed before sending. We work through the structured-update case, the simpler of the two, with real numbers.
Suppose an on-device LSTM update touches 300,000 parameters, stored as 4-byte floats. The full dense update is:
300,000 params × 4 bytes = 1,200,000 bytes = 1.2 MBA structured update restricts the client to updating only a random 10% of those parameters, a sparsity mask generated from a seed the server sends at the start of the round (so the server, knowing the seed, knows exactly which 30,000 positions the values correspond to, and only those values need to be transmitted):
0.10 × 300,000 = 30,000 nonzero values
30,000 × 4 bytes + 4-byte seed = 120,004 bytes ≈ 0.12 MB
compression ratio = 1,200,000 / 120,004 ≈ 10.0×Here is the mechanism at small scale, executed and checked rather than asserted:
import numpy as np
def structured_update(delta, sparsity, seed):
rng = np.random.default_rng(seed)
mask = rng.random(delta.shape) < sparsity
nonzero_values = delta[mask]
return nonzero_values, mask, int(mask.sum())
delta = np.array([0.02, -0.01, 0.15, 0.04, -0.08,
0.03, 0.11, -0.02, 0.05, 0.07])
values, mask, kept = structured_update(delta, sparsity=0.3, seed=42)
print(mask.tolist())
print(kept, values.tolist())
print(delta.nbytes, values.nbytes + 4)Run against 10 parameters with a target sparsity of 0.3, this produces [False, False, False, False, True, False, False, False, True, False], keeping 2 of the 10 slots (indices 4 and 8) with values [-0.08, 0.05], and byte counts 80 (full) versus 20 (compressed). The demo's delta array is left at NumPy's default dtype, float64 (8 bytes per value) — which is why those byte counts are 80 and 20 rather than the 40-and-12-byte figures an explicit dtype=np.float32 array would print (2 kept values × 4 bytes + a 4-byte seed = 12); the chapter's 300,000-parameter estimates above use the 4-byte float32 convention that production Gboard weights actually use, so it is the toy demo's ~4× compression ratio, not its raw byte counts, that carries over to production scale. Note that 2 kept is not exactly 3 (10 × 0.3): with only 10 draws, a Bernoulli mask does not hit its expectation precisely. At 300,000 parameters the law of large numbers takes over and the realized fraction lands almost exactly on 10%, which is why the 30,000-value estimate above is safe to use at production scale even though the 10-parameter demo undershoots it.
Compression matters because upload speed, not download speed, is the binding constraint on most mobile connections. At a constrained mobile uplink of 1 Mbps (125,000 bytes/s), the full update takes 1,200,000 / 125,000 = 9.6 seconds to upload; the compressed update takes 120,004 / 125,000 ≈ 0.96 seconds. On a home Wi-Fi uplink of 12 Mbps (1,500,000 bytes/s), the full update takes 0.8 seconds and the compressed one about 0.08 seconds. Multiply by a cohort: if a round needs roughly 200 devices to report, the uncompressed aggregate upload volume is 200 × 1.2 MB = 240 MB; compressed, it is 200 × 0.12 MB = 24 MB. That order-of-magnitude reduction is what makes it feasible to run this over consumer connections at all without a user noticing a dent in their data plan or their battery, which is precisely what the eligibility gate is trying to protect in the first place. Compression and eligibility are solving the same underlying resource-scarcity problem from two different angles: one shrinks the update, the other restricts when updates are even attempted.
Secure aggregation: privacy that costs bandwidth too
Sending a compressed update is not yet a privacy guarantee by itself: an update from a single device, even a small one, is still that device's own information, visible in the clear to the server that receives it. Bonawitz et al. (2017) close this gap with a secure aggregation protocol built specifically for the cross-device setting. Before any model update is sent, devices in the same cohort run a lightweight key-agreement step so that each device can generate a pairwise random mask shared with every other device in the cohort. Each device then adds its own share of these masks to its real update before uploading. Individually, every uploaded value looks like noise. Summed across the whole cohort, the masks are constructed so that they cancel out exactly, leaving the server with nothing but the true sum of the real updates, never any individual device's contribution. The protocol is also built to tolerate the same dropout problem the round lifecycle already has to handle: devices that vanish mid-protocol do not prevent the survivors' masks from cancelling correctly.
This buys real privacy, but it is not free: the key-agreement and mask-generation steps require every device in the cohort to exchange messages related to cohort size before the round's actual model traffic even begins, so the communication cost of a secure-aggregation round scales with the cohort size N, not only with the model's size. This is a genuine three-way systems tradeoff a deployment engineer has to navigate: a larger cohort gives a statistically better FedAvg average (less noisy, less prone to overfitting one device's idiosyncratic typing), but it also means a larger, slower secure-aggregation handshake and a longer window in which dropout can occur, which is exactly why the Selector over-selects rather than targeting a minimal cohort and hoping everyone finishes.
A misconception worth killing
Students who have only seen federated learning described abstractly, as "training happens where the data lives instead of on a central server," often assume that a Gboard-style deployment means every phone with the app installed contributes to every training round. That is false, and the falseness is the entire point of this chapter: at any given moment, well under 2% of the opted-in population is simultaneously idle, charging, and on unmetered Wi-Fi, and only a further subset of that is actually invited and completes in time. A round's cohort is a small, time-of-day-biased, connectivity-biased sample, not a census. That has a direct statistical consequence the security-focused sibling chapter does not need to dwell on but this one does: the population that trains the model skews toward whoever is asleep next to a charger on home Wi-Fi at 11 p.m., which is not demographically identical to the full user base. Bonawitz et al. (2019) and follow-up work on this system explicitly flag that eligibility-driven sampling bias, not just non-IID data per se, is a first-class deployment concern, separate from and additional to the statistical non-IID problem covered elsewhere in this course.
Active recall
Attempt each question before reading its answer.
Q1. Why does the FL server over-select more devices than its target cohort size, rather than simply inviting exactly the target number?
Q2. A device satisfies idle + charging + unmetered Wi-Fi at the start of a round, is invited, downloads the model, and starts local training. Midway through, its owner picks it up and starts typing. What happens to that device's contribution to the round, and why does the system tolerate this without retrying?
Q3. With 300,000 parameters at 4 bytes each and a structured-update sparsity of 5% instead of 10%, what is the new compressed update size in bytes, and what compression ratio does that represent relative to the full 1.2 MB update?
Q4. What specifically does the FL server see after secure aggregation completes for a round, and what does it not see?
Q5. Suppose Gboard's eligibility policy is tightened to require idle AND charging AND unmetered Wi-Fi AND battery ≥ 90% (adding the battery condition to the original three), where independently, illustratively, P(idle) = 0.30, P(charging) = 0.15, P(unmetered Wi-Fi) = 0.40, and P(battery ≥ 90%) = 0.20. Out of a population of 5,000,000 opted-in devices, how many are instantaneously eligible before and after this change, and trace at least two downstream systems consequences beyond just "fewer devices."
Q6. Why is cross-silo federated learning (e.g., a hospital consortium) not subject to the same over-selection and structured-update-compression engineering that Gboard needs?
A1. Because a meaningful fraction of invited devices will drop out before the round's deadline (the user picks up the phone, unplugs the charger, walks out of Wi-Fi range), inviting exactly the target number risks finishing a round with too few survivors to produce a good FedAvg average, or missing the deadline while waiting for stragglers. Over-selection is a buffer against dropout, not a bug in the selection logic.
A2. The screen turning on breaks the idle condition, so the device is no longer eligible; it either aborts its local training or simply fails to upload before the round's reporting deadline. The server does not wait for it or retry it: the round closes on schedule using whichever subset of the (over-selected) cohort did finish. This is only tolerable because FedAvg's weighted average is fairly robust to which subset of an already-large, already-random cohort actually reports, and because a device that missed this round remains eligible for a future one, so nothing is permanently lost, only deferred.
A3. 5% of 300,000 = 15,000 nonzero values. At 4 bytes each plus a 4-byte seed: 15,000 × 4 + 4 = 60,004 bytes ≈ 0.06 MB. Relative to the full 1,200,000-byte update, the ratio is 1,200,000 / 60,004 ≈ 20.0×, twice the compression of the 10% case, because halving the sparsity fraction halves the number of transmitted values while the fixed 4-byte seed overhead stays negligible either way. The tradeoff is that a sparser update also carries less signal per round, typically requiring more rounds to converge, so this is a bandwidth-versus-convergence-speed knob, not a free lunch.
A4. The server sees only the aggregate sum of the masked updates across the whole cohort, which, because the pairwise masks are constructed to cancel exactly, equals the true sum of the cohort's real (unmasked) updates. It never sees any individual device's own update, masked or otherwise, and cannot isolate one device's contribution from the sum, since the sum is the smallest unit of information the protocol reveals.
A5. Before the battery condition: eligible fraction = 0.30 × 0.15 × 0.40 = 0.018 (1.8%), giving 5,000,000 × 0.018 = 90,000 instantaneously eligible devices. After adding P(battery ≥ 90%) = 0.20: eligible fraction = 0.018 × 0.20 = 0.0036 (0.36%), giving 5,000,000 × 0.0036 = 18,000 devices, a 5× drop in the instantaneously eligible pool. Consequences beyond raw count: (1) the Selector must invite proportionally more devices, or the pace-steering logic must widen the round's time window, to still assemble a cohort of the same target size, which slows how frequently the global model can be updated; (2) a battery-health filter correlates with device age and model tier, since older or heavily-used batteries degrade faster and rarely stay above 90% while charging is still in progress, so the training population skews toward users with newer phones, which in turn correlates with income and geography, introducing a demographic sampling bias in exactly the direction the misconception section above warned about, now made worse, not just smaller.
A6. A hospital consortium has a handful of well-provisioned, reliably-online, institutionally-controlled servers rather than millions of unpredictable consumer phones, so there is no dropout problem at the scale that motivates over-selection, and no battery or metered-data constraint that motivates shrinking the update to a random 10% subset. Cross-silo participants can typically afford to send a full-precision update over a stable institutional network; the engineering effort there goes instead into governance, auditability, and handling the small number of very large, very different local datasets, not into surviving flaky connectivity.
Think About It
Think about this: How would you explain federated learning: distributed privacy-preserving training to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind federated learning: distributed privacy-preserving training, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.