In March 2024 the Government of India approved the IndiaAI Mission with an outlay of ₹10,372 crore, and by 2025 its Compute Capacity pillar had empanelled a shared pool of roughly 18,000 GPUs — a mix of NVIDIA H100s and A100s procured through public-private partnerships and offered to startups, researchers, and academic labs at subsidised hourly rates. The moment that pool exists, a hard question follows immediately: a lab applies for compute to train a new Hindi-English bilingual model. Do you give them 64 GPUs for a month, or 256 GPUs for a week? Either allocation burns roughly the same electricity bill, but they do not produce the same model. This is the problem this chapter is about — not "how do transformers work" (you already know that from Grade 11), but "given that GPU-hours are the scarcest, most expensive resource in modern AI, what is the mathematically correct way to convert a fixed compute budget into a model, and what is the correct policy for splitting a finite cluster among competing claims on it?" These are two different problems — one is applied statistics (scaling laws), the other is systems and mechanism design (governance) — and production AI labs, and now national compute missions, have to solve both simultaneously.
Compute as a governed resource: what a FLOP budget actually measures
Before you can allocate compute you need a unit for it. GPU-hours are the accounting unit, but the physically meaningful quantity is floating-point operations (FLOPs) actually spent on useful arithmetic. For a dense transformer, this has a clean closed-form estimate. In the forward pass, each of the model's N parameters is touched by roughly one multiply-accumulate per token processed — a multiply and an add, so 2 FLOPs per parameter per token, giving forward-pass cost ≈ 2N FLOPs per token. Backpropagation computes gradients with respect to both the activations and the weights, and each of those gradient computations costs about as much as the forward pass itself, so the backward pass costs roughly 4N FLOPs per token. Summed over a training run of D tokens, total training compute is:
C ≈ 6 · N · D (FLOPs)This is the estimate used throughout the scaling-law literature (Kaplan et al., OpenAI, 2020) and it is not just a rule of thumb — you can sanity-check it against a real published run. DeepMind's Chinchilla model has N = 70 billion parameters and was trained on D = 1.4 trillion tokens. Plugging in: C = 6 × 70×10⁹ × 1.4×10¹² ≈ 5.88×10²³ FLOPs, which matches the ≈5.76×10²³ FLOPs the Chinchilla paper (Hoffmann et al., DeepMind, 2022) reports it was trained for, within rounding. A cluster operator does not get to choose C directly — they choose GPU count, wall-clock time, and hope for good hardware utilisation, and C falls out of that choice. The delivered FLOPs/second from a GPU is never its advertised peak: achieved throughput divided by peak throughput is called Model FLOPs Utilisation (MFU), and even well-optimised large-scale training runs typically land at 30–50% MFU because of communication stalls between GPUs, attention's non-matmul overhead, and pipeline bubbles. Any governance decision about "how many GPUs for how long" is therefore really a decision about C, mediated by an MFU you have to estimate, not assume.
From Kaplan to Chinchilla: how much data does a compute budget deserve
Once you know C, you still face a choice: spend it on a bigger model trained on fewer tokens, or a smaller model trained on more tokens. Kaplan et al. (OpenAI, 2020) ran the first systematic study of this trade-off and concluded that, as compute grows, you should grow model size much faster than dataset size — spend most of a bigger budget on more parameters, only a little on more data. Many of the large models trained in 2020–2021, including GPT-3 (175 billion parameters, trained on roughly 300 billion tokens — about 1.7 tokens per parameter), followed this prescription.
Hoffmann et al. (DeepMind, 2022) revisited the question with a method called IsoFLOP profiling: fix a compute budget C, train many models of different sizes N to convergence within that fixed budget (so smaller N automatically gets more tokens D, since D = C/6N), and plot final training loss against N. Each fixed-C curve traces a shallow bowl: too few parameters and the model is too weak to use the data well; too many parameters and it runs out of tokens before it converges; a minimum sits in between. Repeating this at several values of C and connecting the minima gives the compute-optimal frontier, and it does not match Kaplan's prescription — it says N and D should grow at roughly the same rate as compute grows, both scaling close to C^0.5. Applied at Chinchilla's scale, this works out to roughly 20 training tokens per parameter — well above what Kaplan's rule (and GPT-3's actual ratio) prescribed. Hoffmann et al. traced the discrepancy in part to a confound in Kaplan's smaller-scale experiments: models with a learning-rate schedule not properly matched to their training length looked artificially worse at high token counts, biasing the fitted exponents toward favouring parameters over data.
The common misconception this corrects is the instinct that, at a fixed compute budget, the biggest model you can fit is always the best choice — "more parameters = smarter model." It is false as a compute-constrained statement. Chinchilla's own headline result is the proof: DeepMind trained Chinchilla (70B parameters, 1.4T tokens) on essentially the same compute budget as their earlier model Gopher (280B parameters, ≈300B tokens — about 1 token per parameter). Despite having a quarter as many parameters, Chinchilla outperformed Gopher on nearly every benchmark, because Gopher was severely under-trained relative to its size — it had capacity it never got the data to use. A governance board that hands a lab the biggest GPU allocation possible and lets them maximise parameter count for that hardware, without also thinking about how many tokens they can actually feed it in the time granted, is reproducing Gopher's mistake at whatever scale it operates.
Worked example: sizing a model against a real allocation
Suppose a governance board grants a research consortium 64 H100 GPUs for 30 days from the national pool. H100 SXM cards deliver ≈989 TFLOPS of dense BF16 tensor-core throughput at peak; assume, as a realistic engineering estimate for a well-tuned distributed run, 40% MFU. What is the compute-optimal model this allocation should produce?
import math
# Governance allocation
num_gpus = 64
days = 30
peak_flops_gpu = 989.5e12 # NVIDIA H100 SXM, bf16 dense tensor-core peak
mfu = 0.40 # assumed achieved model-FLOPs-utilization
seconds = days * 86400
effective_flops_per_gpu = peak_flops_gpu * mfu
C = effective_flops_per_gpu * num_gpus * seconds
print(f"Compute budget C = {C:.3e} FLOPs")
# Chinchilla-style compute-optimal split: C = 6*N*D, empirically D ~ 20*N
N = math.sqrt(C / 120) # since C = 6*N*(20N) = 120*N^2
D = 20 * N
print(f"Optimal parameters N = {N:.3e}")
print(f"Optimal tokens D = {D:.3e}")
print(f"Check 6*N*D = {6*N*D:.3e}")Tracing it by hand: seconds = 30 × 86400 = 2,592,000. Effective per-GPU throughput is 989.5×10¹² × 0.40 = 3.958×10¹⁴ FLOPs/s. Across 64 GPUs for that duration, C = 3.958×10¹⁴ × 64 × 2.592×10⁶ ≈ 6.566×10²² FLOPs. Solving C = 120N² for N gives N = √(6.566×10²²/120) ≈ 2.339×10¹⁰ — about 23.4 billion parameters — and D = 20N ≈ 4.678×10¹¹, about 468 billion tokens. The code prints exactly these three lines:
Compute budget C = 6.566e+22 FLOPs
Optimal parameters N = 2.339e+10
Optimal tokens D = 4.678e+11
Check 6*N*D = 6.566e+22So this specific allocation — 64 GPUs, 30 days, 40% MFU — is compute-optimally spent on a ≈23-billion-parameter model trained on ≈468 billion tokens, not on the largest model that fits in 64×80GB of HBM. A governance board that instead let the lab train a 70-billion-parameter model (because "bigger sounds more advanced") on the same 30-day, 64-GPU allocation would only be able to afford D = C/(6N) = 6.566×10²²/(6×7×10¹⁰) ≈ 1.56×10¹¹ tokens — about 156 billion, roughly 2.2 tokens per parameter — landing far to the under-trained side of the IsoFLOP bowl, exactly Gopher's mistake, on a smaller cluster.
There is a real constraint this example glosses over: 468 billion good-quality tokens of a specific Indian language pair are not always sitting on disk waiting to be tokenised. If the available corpus tops out at 200 billion tokens, the board faces a genuine trade-off — hold D fixed at 200B and solve N = C/(6D) = 6.566×10²²/(6×2×10¹¹) ≈ 5.47×10¹⁰, i.e. train a 54.7-billion-parameter model at under 4 tokens per parameter (badly under-trained), or keep N at the compute-optimal 23.4B and accept that the run finishes with GPUs sitting idle before the 30-day window is up, because there is not enough data left to feed them. Data scarcity, not GPU scarcity, is often the actual binding constraint in this governance problem, and no amount of additional GPU-hours fixes it.
Governance mechanisms: turning one scarce pool into a schedule
Sizing a model correctly assumes you already know your allocation. Deciding whose job gets which GPUs, and when, is a separate scheduling problem, and it has real published mechanisms behind it. Two matter most for AI training clusters:
Gang scheduling. Synchronous data-parallel training needs every one of its GPUs present at once — each optimisation step ends with an all-reduce across all participating devices, so a job holding 63 of its requested 64 GPUs cannot make progress at all; it is not "63/64ths as fast," it is completely stalled. A scheduler that hands out GPUs one at a time, as it would for ordinary web-service workloads, can leave a large training job perpetually almost-but-not-quite launched while smaller jobs cut in around it — a form of resource fragmentation. Production cluster managers such as Google's Borg (Verma et al., EuroSys 2015) and its successors treat large distributed jobs as atomic units: all requested resources are granted together or the job stays queued, exactly as sketched by "Job A" in the diagram below.
Fairness across heterogeneous jobs. A pool serving both training jobs (GPU- and network-bandwidth-hungry) and inference-serving jobs (GPU-memory- and request-latency-hungry) cannot be split fairly by GPU count alone, because a "fair" GPU share can still be wildly unfair on the resource each job actually bottlenecks on. Ghodsi et al. (NSDI 2011) formalised this as Dominant Resource Fairness: for each tenant, compute the fraction of every resource type (GPU count, host memory, interconnect bandwidth) their job would consume, call the largest such fraction their dominant share, and allocate so that every tenant's dominant share is equalised. This generalises ordinary max-min fairness — which only works for one resource — to a cluster where different jobs are bottlenecked on different things. Inference-serving jobs, unlike training jobs, do not need gang-scheduled whole GPUs at all: NVIDIA's Multi-Instance GPU (MIG) feature lets a governance layer slice one physical H100 into up to seven isolated instances, so many low-traffic tenants can share hardware that a single training job would otherwise monopolise.
One more governance detail belongs here because it silently eats into the MFU assumed in the worked example above: a 30-day, 64-GPU job is long enough that hardware failures and priority preemptions are close to certain, so production training runs checkpoint model and optimiser state every few hundred to few thousand steps. That I/O is pure overhead against the "6ND" ideal — it is why real achieved MFU sits at 30–50% rather than near 100%, and why the governance layer's preemption policy (how often it is allowed to evict a running job for a higher-priority one) directly trades off against how much of the granted compute budget actually reaches the model.
Active recall
1. Using the worked example's numbers, the governance board cuts the allocation from 64 GPUs to 32, but extends the deadline from 30 to 60 days "to compensate." Does the compute-optimal model change?
Answer: No. C is proportional to GPUs × seconds. Halving GPUs and doubling days leaves the product — and therefore C, N, and D — unchanged. The board has traded parallelism for wall-clock time, which affects how fast the answer arrives, not what the compute-optimal answer is.
2. Same 64 GPUs, 30 days, but the engineering team installs a better attention kernel and raises MFU from 40% to 55%. Recompute C, N, and D.
Answer: C scales linearly with MFU: C_new = 6.566×10²² × (55/40) ≈ 9.03×10²² FLOPs. Since N ∝ √C, N_new = √(9.03×10²²/120) ≈ 2.74×10¹⁰ — about 27.4 billion parameters — and D_new = 20N ≈ 5.48×10¹¹, about 548 billion tokens. A 37.5% MFU improvement buys a 17% larger compute-optimal model, because N only grows as the square root of compute — software efficiency gains are valuable but they do not translate one-for-one into model size.
3. Why can a scheduler not simply hand Job A 63 of its 64 requested GPUs while the 64th finishes another task, and let training start early?
Answer: Synchronous data-parallel training performs an all-reduce across every participating GPU at the end of each step; with one GPU missing, that collective operation cannot complete, so the job makes zero progress, not 63/64ths progress. This is exactly why cluster managers like Borg treat such jobs as gang-scheduled atomic units — allocate all requested GPUs together or keep the job queued.
4. Two tenants share a cluster: Tenant X's jobs are GPU-bound (they'd take 80% of GPUs but only 10% of network bandwidth if fully served), Tenant Y's are bandwidth-bound (30% of GPUs, 70% of bandwidth). Under Dominant Resource Fairness, whose dominant resource is which, and what does the scheduler try to equalise?
Answer: X's dominant resource is GPUs (80% > 10%); Y's is bandwidth (70% > 30%). DRF equalises each tenant's dominant share rather than giving both tenants equal GPU counts — it would scale back X's GPU allocation and Y's bandwidth allocation until the fraction each tenant holds of their own bottleneck resource matches, so neither tenant is worse off on the resource that actually constrains them.
5. A lab has a compute budget for a 468-billion-token, compute-optimal run, but the available high-quality corpus for their target language pair is only 200 billion tokens. Using C ≈ 6.566×10²² FLOPs from the worked example, what are the two governance options, and what does each cost?
Answer: Option one: hold D at the available 200B tokens and solve N = C/(6D) = 6.566×10²²/(6×2×10¹¹) ≈ 5.47×10¹⁰ — a 54.7-billion-parameter model at under 4 tokens per parameter, well below the ~20 that Chinchilla-style scaling recommends, so the model is under-trained relative to its size. Option two: keep N at the compute-optimal 23.4 billion parameters and train only on the 200B available tokens; the run finishes using C' = 6×2.339×10¹⁰×2×10¹¹ ≈ 2.81×10²² FLOPs, well under the 6.566×10²² granted, so roughly 57% of the allocated GPU-hours go unused unless the board reallocates them elsewhere. Neither option is free — the real constraint is data availability, not GPU count.
6. Explain, in one sentence, why "give the lab the biggest model that fits in GPU memory" is bad governance policy even when GPU-hours are abundant.
Answer: Memory capacity bounds N but says nothing about D; a model sized to fill memory without a matching token budget lands on the under-trained side of the IsoFLOP curve — Gopher's mistake — so the correct sizing rule is the compute-optimal pair (N, D) derived from the granted FLOPs, not the largest N the hardware can hold.
Think About It
Think about this: How would you explain compute governance and scaling laws: managing ai resource allocation to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Practice Exercises
Now it is time to practice! Complete these challenges to solidify your understanding:
- Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
- Exercise 2: Find a real-world example where compute governance and scaling laws: managing ai resource allocation is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
- Exercise 3: Create a mind-map connecting compute governance and scaling laws: managing ai resource allocation to at least 3 other topics you have studied.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind compute governance and scaling laws: managing ai resource allocation, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.