A transformer encoder layer processes a batch of 16 sequences, each of length 64 tokens, with model dimension d_model = 256 and 8 attention heads. For multi-head self-attention, Q, K, and V are first computed as X·Wq, X·Wk, X·Wv (each of shape [16, 64, 256]) and then reshaped and transposed so each head operates independently, with d_k = d_model / num_heads. What are the shapes of the raw attention score matrix QK^T (before softmax) and the concatenated multi-head output (after attention, before the final output projection Wo), respectively?
Splitting the projections into 8 heads removes the batch axis, leaving score matrices of shape [8, 64, 64] and a concatenated output of [64, 256] before the final projection is applied.
The score matrix QK^T is [16, 8, 64, 64], and the concatenated output across all eight heads is [16, 64, 256], since d_k = 256/8 = 32 and 8x32 recombines exactly into d_model.
Attention scores are [16, 8, 64, 64], but each head retains the full 256-dimensional value vectors, so concatenating the eight heads before the output projection gives [16, 64, 2048].
Because K is transposed along its 32-dimensional head axis rather than the sequence axis, QK^T works out to [16, 8, 32, 32], while the concatenated head output remains [16, 64, 256].
Answer: B. The score matrix QK^T is [16, 8, 64, 64], and the concatenated output across all eight heads is [16, 64, 256], since d_k = 256/8 = 32 and 8x32 recombines exactly into d_model.
ExplanationWith d_model = 256 and num_heads = 8, each head's dimension is d_k = 256/8 = 32, so Q, K, and V are reshaped from [16, 64, 256] into [16, 8, 64, 32] — batch, heads, sequence length, per-head dimension. The score matrix is a batched matmul over only the last two axes: Q [16, 8, 64, 32] against K^T [16, 8, 32, 64] gives QK^T of shape [16, 8, 64, 64], meaning every one of the 8 heads in every one of the 16 batch items gets its own 64x64 grid of token-to-token compatibility scores. After softmax and the weighted sum with V, each head still outputs [16, 8, 64, 32]; transposing the head axis back next to the sequence axis and merging it with d_k reassembles exactly [16, 64, 256], recovering d_model — this is the concatenated result handed to Wo, not yet reduced or expanded by it. Treating each head as if it kept the full 256-dimensional value vectors instead of the 32-dimensional split inflates the concatenation to [16, 64, 2048], forgetting that num_heads x d_k must equal d_model by construction. Matching K's transpose to the 32-wide head axis instead of the 64-wide sequence axis confuses which dimension attention actually operates over, shrinking the score matrix to [16, 8, 32, 32] when it should compare every token position against every other token position. Assuming the per-head reshape collapses or drops the batch axis is also incorrect — the batch dimension of 16 is carried through every reshape and transpose untouched.
Question 2 · Self-Attention · hard
In a decoder-only transformer, causal (autoregressive) self-attention restricts each query at position i (0-indexed, i = 0 to n−1) to attend only to key positions j satisfying j ≤ i; every other score is set to −∞ before the softmax is applied. For a single attention head processing a sequence of length n = 10, how many of the 100 entries in the n×n raw attention-score matrix are masked out before softmax?
45 masked entries, since the masked positions form a strict upper triangle above the diagonal, giving n(n-1)/2 entries
55 masked entries, matching the count of positions where the key index is less than or equal to the query index
90 masked entries, since only the positions where query index equals key index are allowed to remain unmasked
50 masked entries, since a triangular mask blocks roughly half of the square attention matrix by symmetry
Answer: A. 45 masked entries, since the masked positions form a strict upper triangle above the diagonal, giving n(n-1)/2 entries
ExplanationBecause causal masking allows query position i to attend to key positions j = 0, 1, ..., i, the number of unmasked (allowed) entries for row i is i + 1. Summing over all rows from i = 0 to 9 gives the total unmasked count: 1 + 2 + 3 + ... + 10 = 55. The full score matrix has n^2 = 10 x 10 = 100 entries, so the masked count is 100 - 55 = 45. This matches the direct combinatorial view: the masked entries are exactly those with j > i, which form a strict upper triangle of the square matrix, containing n(n-1)/2 = 10 x 9 / 2 = 45 entries. Confusing which triangle is blocked leads to reporting the unmasked count of 55 instead of the masked count. Assuming attention is only permitted along the diagonal (each token attending solely to itself) overcounts the mask, producing n^2 - n = 90. Assuming the triangular split divides the matrix exactly in half ignores that the diagonal itself is unmasked, so the two triangles are not equal in size and the true split is 45 masked versus 55 unmasked, not 50-50.
Question 3 · Distributed Training · hard
A cluster of 4 GPUs performs data-parallel synchronous SGD to train a model with 50 million parameters (4 bytes each, so each GPU's local gradient occupies 200 MB). Each GPU processes a local mini-batch of 32 samples before gradients are synchronized using ring all-reduce across the 4 GPUs. What is the effective global batch size for this training step, and how does the resulting per-GPU communication volume for ring all-reduce behave as the number of GPUs N is increased?
With 4 GPUs each contributing a local batch of 32, the effective global batch size is 128, and each GPU's ring all-reduce traffic works out to 2(N−1)/N × 200 MB = 300 MB — a volume that stays bounded near twice the gradient size as more GPUs join, rather than growing linearly with N.
Multiplying the 4 GPUs by the local batch of 32 gives an effective global batch size of 128, and because each GPU exchanges its full gradient directly with every other GPU during ring all-reduce, per-GPU traffic equals (N−1) × 200 MB = 600 MB, which scales linearly with the number of GPUs.
Because synchronous SGD only advances once every GPU finishes its own forward-backward pass, the global batch size stays at the per-GPU value of 32 regardless of how many GPUs participate, since synchronization does not combine samples across devices.
Since 4 GPUs each process 32 samples before synchronizing, the effective global batch size is 128, and ring all-reduce keeps total per-GPU traffic fixed at 200 MB no matter how large the gradient vector grows, because the ring topology transmits only one fixed-size chunk per GPU per synchronization round.
Answer: A. With 4 GPUs each contributing a local batch of 32, the effective global batch size is 128, and each GPU's ring all-reduce traffic works out to 2(N−1)/N × 200 MB = 300 MB — a volume that stays bounded near twice the gradient size as more GPUs join, rather than growing linearly with N.
ExplanationData parallelism replicates the model on every GPU and splits the mini-batch across devices, so the effective global batch is the per-GPU batch multiplied by the number of GPUs: 4 × 32 = 128 samples per synchronization step, not the per-GPU batch alone. For gradient synchronization, ring all-reduce arranges the N=4 GPUs in a logical ring and performs the exchange in two phases — scatter-reduce followed by all-gather — each requiring N−1 steps where every GPU sends and receives a 1/N-sized chunk of the gradient. Summing the data moved through both phases gives a total per-GPU communication volume of 2(N−1)/N × S, where S is the gradient size. With S = 200 MB (50 million parameters × 4 bytes) and N = 4, this equals 2(3)/4 × 200 MB = 300 MB. Crucially, as N grows, (N−1)/N approaches 1, so this volume converges to a fixed ceiling of 2S — it does not keep climbing linearly with the GPU count the way a naive scheme would, where every GPU exchanges its full gradient with every other GPU, or all gradients funnel through a single parameter server. This bounded, N-independent communication cost per GPU is exactly why ring all-reduce scales efficiently to large GPU clusters.
Question 4 · Model Compression · hard
A convolutional neural network has 10 million parameters, each initially stored as a 32-bit (4-byte) floating-point number, giving an original size of 40 MB (using 1 MB = 10⁶ bytes). Two compression steps are then applied in sequence: first, magnitude-based pruning permanently removes the 75% of weights with the smallest absolute value, keeping the rest; second, each surviving weight is quantized from a 32-bit float down to an 8-bit (1-byte) integer. What are the final model size and the overall compression ratio (original size divided by final size) after both steps?
Pruning alone determines the outcome here: since quantization only affects computation and not stored file size, the final size stays 10 MB and the overall compression ratio is 4x.
Interpreting 'prune 75%' as removing only the smallest 25% of weights (keeping 75%) gives 7.5 million surviving weights, a final size of 7.5 MB, and a compression ratio near 5.3x.
Eight-bit integer quantization needs 2 bytes to preserve sign and precision, so the surviving weights occupy a final size of 5 MB with a compression ratio of 8x.
Pruning leaves 2.5 million surviving weights, and storing each in a single byte yields a final size of 2.5 MB, an overall compression ratio of 16x.
Answer: D. Pruning leaves 2.5 million surviving weights, and storing each in a single byte yields a final size of 2.5 MB, an overall compression ratio of 16x.
ExplanationPruning removes 75% of the 10 million weights, leaving 25% x 10,000,000 = 2,500,000 weights. Quantizing each surviving weight from 4 bytes (32-bit float) down to 1 byte (8-bit integer) gives a final size of 2,500,000 x 1 byte = 2,500,000 bytes = 2.5 MB. The original size was 10,000,000 x 4 bytes = 40,000,000 bytes = 40 MB, so the overall compression ratio is 40 MB / 2.5 MB = 16x — the compressed model occupies just 6.25% of the original storage. Treating quantization as irrelevant to file size confuses using lower precision for computation with using lower precision for storage; shrinking the byte-width per weight is exactly what reduces the stored file. Reading 'prune 75%' as a retention rate rather than a removal rate inflates the surviving weight count from 2.5 million to 7.5 million. Claiming 8-bit integers need 2 bytes ignores that 8 bits is, by definition, exactly 1 byte, not 2.
Question 5 · Multimodal AI · hard
CLIP is trained with a batch of 64 image-caption pairs. In each training step, the image encoder and text encoder each produce one embedding per input, and the model computes cosine similarity between every image embedding and every text embedding to form an N×N similarity matrix that feeds the symmetric InfoNCE contrastive loss, where only the diagonal entries are true matching pairs. With N = 64, how many negative (non-matching) image-caption pairs does this single training step generate, and what is the time complexity of building the full similarity matrix as a function of the batch size N?
The batch produces 4,032 negative pairs (N² − N), and building the full similarity matrix runs in O(N²) time.
All 4,096 entries of the matrix (N²) count as negatives, since the loss treats every pair independently, giving O(N²) time.
Only 63 negatives exist per anchor image, so the similarity computation scales as O(N) rather than quadratically.
There are 4,032 negative pairs (N² − N), but the matrix computation is O(N³) because each cosine similarity involves an embedding-dimension inner product.
Answer: A. The batch produces 4,032 negative pairs (N² − N), and building the full similarity matrix runs in O(N²) time.
ExplanationWith batch size N = 64, CLIP builds an N×N cosine-similarity matrix between all image and text embeddings, giving 64² = 4,096 total entries. Of these, the N = 64 diagonal entries are the true matching image-caption pairs (the positives used in the InfoNCE loss), leaving N² − N = 4,096 − 64 = 4,032 off-diagonal entries as negatives — every image is contrasted against every caption from every other example in the batch. Computing this matrix costs O(N²) as a function of batch size: the number of pairwise similarities grows quadratically with N, even though each individual cosine similarity is an inner product over the fixed embedding dimension d, which is a constant factor that does not scale with N. This quadratic cost in batch size, rather than a per-image O(N) count or a cubic O(N³) blow-up, is exactly why very large batches (OpenAI trained CLIP with batch size 32,768) make each training step memory- and compute-intensive, and why contrastive pretraining is typically distributed across many accelerators.
Question 6 · Multimodal AI · hard
A Vision Transformer (ViT) processes an image of size 384×384 pixels by dividing it into non-overlapping patches of size 32×32, then linearly projects each flattened patch into an embedding dimension of 1024 before feeding the sequence into the transformer encoder. What are the total number of patches and the final input sequence length (including the learnable [CLS] token) that the encoder actually receives?
The 384×384 image splits into a 12×12 grid of 32×32 patches, yielding 144 total patches and a final sequence length of 145 after prepending the [CLS] token.
Dividing 384 by 32 directly gives 12 patches along one dimension, so the encoder receives a sequence length of 13 after prepending the [CLS] token.
A 12×12 grid of non-overlapping patches produces 144 patches, but the sequence length remains 144 since ViT does not use a classification token for image classification.
Counting each of the 3 RGB channels as a separate patch token across the 12×12 grid yields 432 patches, so the sequence length becomes 433 after prepending the [CLS] token.
Answer: A. The 384×384 image splits into a 12×12 grid of 32×32 patches, yielding 144 total patches and a final sequence length of 145 after prepending the [CLS] token.
ExplanationBecause the image is two-dimensional, patches tile both height and width, so the grid size is (384/32) x (384/32) = 12 x 12 = 144 patches - not just 12, which would only account for one row of tiling. Following the BERT-style convention ViT adopts, a single learnable [CLS] token is prepended to the sequence of patch embeddings to serve as the pooled representation used for classification, so the encoder's actual input length is 144 + 1 = 145 tokens. The embedding dimension of 1024 is a separate hyperparameter: each 32x32x3 patch is first flattened into a 3072-dim vector (32*32*3) and then linearly projected into the 1024-dim embedding space by a learned weight matrix - this projection changes the width of each token's representation, but it does not change how many tokens exist. In particular, the 3 color channels are already folded into each patch's flattened vector before projection, so channels never multiply the patch count; there is exactly one embedding vector per spatial patch, giving 144 patch tokens plus the [CLS] token for a total sequence length of 145.
Question 7 · Advanced RL · hard
In a policy-gradient training run using Generalized Advantage Estimation (GAE), an episode visits states s_0, s_1, s_2, s_3, where s_3 is terminal so V(s_3) = 0. The discount factor is γ = 0.9 and the GAE parameter is λ = 0.8. The rewards are r_0 = 2, r_1 = 1, r_2 = 3, and the critic's value estimates are V(s_0) = 1, V(s_1) = 1.5, V(s_2) = 1. Using A_t = Σ_{l=0}^∞ (γλ)^l δ_{t+l}, where δ_t = r_t + γV(s_{t+1}) − V(s_t), what is A_0, rounded to two decimal places?
Using λ = 1 collapses GAE to the Monte Carlo advantage G_0 − V(s_0), giving A_0 ≈ 4.33 from the full discounted return G_0 = r_0 + γr_1 + γ²r_2.
Weighting the TD errors by powers of λ alone instead of powers of γλ (δ_0 + λδ_1 + λ²δ_2) yields A_0 ≈ 3.95.
Stopping the geometric sum after the first term, as if λ = 0, leaves only δ_0, giving A_0 ≈ 2.35.
Answer: C. Summing δ_0 + (γλ)δ_1 + (γλ)²δ_2 with δ_0 = 2.35, δ_1 = 0.40, and δ_2 = 2.00 gives A_0 ≈ 3.67.
ExplanationGAE defines the advantage as an exponentially-weighted sum of TD errors, A_t = Σ_{l=0}^∞ (γλ)^l δ_{t+l}, where each δ_t = r_t + γV(s_{t+1}) − V(s_t) measures how much better the actual next-step outcome was than the critic's estimate. With γ = 0.9 and V(s_3) = 0 at the terminal state, the three TD errors are δ_0 = 2 + 0.9(1.5) − 1 = 2.35, δ_1 = 1 + 0.9(1) − 1.5 = 0.40, and δ_2 = 3 + 0.9(0) − 1 = 2.00. GAE discounts each successive TD error by the product γλ = 0.9 × 0.8 = 0.72, not by λ alone, so A_0 = δ_0 + (γλ)δ_1 + (γλ)²δ_2 = 2.35 + 0.72(0.40) + 0.5184(2.00) = 2.35 + 0.288 + 1.0368 ≈ 3.67. Weighting by λ alone instead of γλ mixes up the two decay mechanisms and produces a different total (3.95). Setting λ = 1 turns GAE into the plain Monte Carlo advantage, discarding the bootstrapped value estimates and inflating the result to 4.33. Truncating the sum after the first TD error, as if λ = 0, ignores the two later steps entirely and understates the advantage at 2.35. The λ = 0.8 setting used here sits between these extremes, blending short-horizon TD estimates with longer-horizon return information to control the bias-variance tradeoff.
Question 8 · Federated Learning · hard
Implement federated averaging (FedAvg) algorithm where multiple clients (devices) train local models on decentralized data. In each round: (1) server sends global model to K selected clients, (2) clients train locally for E=5 epochs on their 600-sample dataset with batch_size=32, (3) clients send updated weights back to server, (4) server averages weights. How many gradient updates (iterations) does each client perform per round?
Iterations per client = 5 × ceil(600/32) = 5 × 19 = 95 gradient updates per round.
Iterations per client = 1; no local iterations occur in FedAvg between communication rounds.
Iterations per client = 600; each client treats every individual sample as one full iteration.
Iterations unbounded; the actual count depends only on when local training converges, not on the fixed epoch count E.
Answer: A. Iterations per client = 5 × ceil(600/32) = 5 × 19 = 95 gradient updates per round.
ExplanationIn FedAvg, each client performs local training for E=5 epochs on its own 600-sample dataset using batch_size=32. The number of mini-batch iterations per epoch is ceil(600/32) = 19, since the last mini-batch contains fewer than 32 samples but still counts as one iteration. Total gradient updates per round is therefore E × iterations_per_epoch = 5 × 19 = 95. This local batching is what makes federated learning communication-efficient: each client completes many gradient steps on-device before the server needs to hear from it again, in contrast to naive distributed SGD, which would require a network round after every single mini-batch update.
Question 9 · MLOps · hard
A team wants to speed up inference from a 13B-parameter target LLM using speculative decoding. A small draft model proposes k = 4 tokens per round, taking 15 ms to generate each token (60 ms total for the round's draft phase). The target model then verifies all 4 draft tokens together in a single forward pass taking 90 ms — the same wall-clock cost as generating just one token with the target model alone, because LLM decoding is memory-bandwidth-bound rather than compute-bound, so a few extra tokens in the same pass add negligible cost. Assuming every drafted token is accepted (the best-case scenario), what is the speedup in tokens-per-second compared to running the target model alone with standard autoregressive decoding?
Roughly 2.4x, since one round produces 4 tokens in 150 ms total (60 ms draft + 90 ms verify), giving 37.5 ms per token versus the target model's standalone 90 ms per token
Exactly 4x, because k tokens are produced per verification pass and the 60 ms drafting cost can be ignored since it runs on a much smaller, faster model
Speculative decoding is actually slower here, about 0.6x, since dividing the full 150 ms round time by a single token yields 150 ms per token, worse than the 90 ms baseline
Throughput scales to about 6x, because the 90 ms verification pass overlaps completely with the next round's 60 ms drafting phase, leaving only the 15 ms per-token draft cost as the limit
Answer: A. Roughly 2.4x, since one round produces 4 tokens in 150 ms total (60 ms draft + 90 ms verify), giving 37.5 ms per token versus the target model's standalone 90 ms per token
ExplanationIn one speculative-decoding round the draft model spends 4 x 15 ms = 60 ms proposing tokens, and the target model spends 90 ms verifying all 4 in a single forward pass — a flat cost because autoregressive decoding is bottlenecked by loading model weights from memory at each step, not by the extra compute of processing a few more tokens within that same pass. With every draft token accepted, the round yields 4 tokens in 60 + 90 = 150 ms, or 37.5 ms per token on average. Standard autoregressive decoding with the target model alone produces one token per 90 ms forward pass. The speedup is therefore 90 ms / 37.5 ms = 2.4x. Claiming an exact 4x speedup wrongly treats the 60 ms drafting phase as free just because it runs on a smaller model — it still adds to the round's wall-clock time. Claiming a 0.6x slowdown comes from dividing the full 150 ms round by one token instead of by the four tokens actually produced in that round. Claiming a 6x speedup assumes the verification pass can run concurrently with the following round's drafting phase, but the algorithm as described is sequential: verification must wait for all k draft tokens to exist before it can check them, so no such overlap is possible.
Question 10 · Parameter-Efficient Fine-Tuning · hard
Given that a transformer model with 175 billion parameters is being fine-tuned using LoRA with rank r=16 on a downstream classification task, and the original attention weight matrix W_q has dimensions 12288×12288, how would you calculate the total number of trainable parameters introduced by applying LoRA to all query and value projection matrices across 96 attention layers?
The total would be 96 × (12288 × 16) = 18,874,368, because LoRA only adds a single low-rank matrix to each layer without the complementary decomposition factor
This equals 96 × 16 × 16 = 24,576, because LoRA only introduces parameters proportional to the square of the rank value across all layers
The total would be 2 × 96 × (12288 × 12288) = 28,991,029,248, because LoRA still requires updating the full weight matrices but with a rank constraint applied during training
The total trainable parameters would be 96 × 2 × (12288 × 16 + 16 × 12288) = 75,497,472, because LoRA decomposes each weight matrix into two low-rank matrices A and B, applied to both Q and V projections across all layers
Answer: D. The total trainable parameters would be 96 × 2 × (12288 × 16 + 16 × 12288) = 75,497,472, because LoRA decomposes each weight matrix into two low-rank matrices A and B, applied to both Q and V projections across all layers
ExplanationFirst, loRA (Low-Rank Adaptation) works by decomposing the weight update ΔW into two matrices A (d×r) and B (r×d), where d=12288 and r=16. Therefore each LoRA module adds 2×d×r = 2×12288×16 = 393,216 parameters. Then, applied to both Q and V projections across 96 layers: 2 × 96 × 393,216 = 75,497,472. This is approximately 0.04% of the original 175B parameters, which is why LoRA is so parameter-efficient — because it exploits the low intrinsic dimensionality of the adaptation task.
Question 11 · Vision Transformer Architecture · hard
Suppose you are implementing a Vision Transformer (ViT) and need to encode spatial information for 16×16 image patches from a 224×224 input image. If you use learnable positional embeddings with a dimension of 768, what calculation determines whether positional interpolation would be necessary when fine-tuning on a downstream task with 384×384 input images?
You would need positional interpolation because the number of patches increases from 196 (14×14) to 576 (24×24), which exceeds the 196 learned positions from pre-training; therefore you must interpolate the existing position embeddings to cover the new spatial range
You would not need interpolation because positional embeddings are task-agnostic and automatically adapt to any input resolution without modification or retraining
Interpolation becomes necessary when input resolution increases by more than 50%, which 384/224 ≈ 1.71× exceeds, requiring 2D bilinear interpolation of the embedding matrix
The 768-dimensional embedding space is sufficient to encode any resolution without interpolation because high-dimensional spaces can represent arbitrary spatial information implicitly
Answer: A. You would need positional interpolation because the number of patches increases from 196 (14×14) to 576 (24×24), which exceeds the 196 learned positions from pre-training; therefore you must interpolate the existing position embeddings to cover the new spatial range
ExplanationFirst, vision Transformers divide images into fixed-size patches. A 224×224 image with 16×16 patches creates 196 patches (14×14 grid = 196), while a 384×384 image creates 576 patches (24×24 grid = 576). Then, pre-training learns position embeddings for exactly 196 positions. During fine-tuning on 384×384 images, you have 576 positions but only 196 learned embeddings. Therefore, you must interpolate the learned position embeddings to estimate embeddings for the new 576 positions. This is typically done using 2D bilinear interpolation on the reshaped embedding matrix. Finally, without interpolation, you would have undefined embeddings for the 380 additional patches.
Question 12 · RLHF and Alignment · hard
In a PPO-based RLHF fine-tuning run, a batch of responses has an average reward-model score r = 2.4 and an average KL divergence from the reference (SFT) model of KL = 0.15 nats per token, so the PPO objective is J = r − β·KL with β = 0.1. A teammate argues that doubling β to 0.2 will "halve the KL penalty's contribution to J," letting the policy chase reward more freely. Evaluating the arithmetic and its consequence for the policy, which statement is correct?
The teammate has it backwards: doubling β raises the penalty term from β·KL = 0.015 to 0.030, so J falls from 2.385 to 2.370, and the objective now punishes divergence from the reference model more heavily, pulling the policy closer to it rather than freeing it to chase reward.
The teammate is correct: raising β from 0.1 to 0.2 lowers the penalty term from 0.015 to 0.0075, so J rises to 2.3925, giving the policy more room to increase reward before the KL term intervenes.
Before comparing the two β values, the KL divergence must be converted from nats to bits (0.15 × log2(e) ≈ 0.216 bits), which changes the original penalty to 0.0216 rather than 0.015, making the teammate's proposed doubling numerically irrelevant.
Because the reward term r = 2.4 is more than a hundred times larger than either penalty value, changing β from 0.1 to 0.2 alters J by less than 0.001 and therefore produces no detectable difference in how strongly the policy is constrained toward the reference model.
Answer: A. The teammate has it backwards: doubling β raises the penalty term from β·KL = 0.015 to 0.030, so J falls from 2.385 to 2.370, and the objective now punishes divergence from the reference model more heavily, pulling the policy closer to it rather than freeing it to chase reward.
ExplanationIn the PPO objective J = r − β·KL, the penalty term scales linearly with β for a fixed KL divergence: at β=0.1, the penalty is 0.1 × 0.15 = 0.015, giving J = 2.4 − 0.015 = 2.385. Doubling β to 0.2 does not halve this penalty — it doubles it, since penalty = β × KL is linear in β with KL held fixed: 0.2 × 0.15 = 0.030, giving J = 2.4 − 0.030 = 2.370. A larger penalty subtracts more from the reward-driven objective, which discourages the policy from drifting away from the reference (SFT) model rather than freeing it to chase reward — the opposite of what the teammate claims. Converting the KL value from nats to bits doesn't change this relationship, since rescaling units affects both β=0.1 and β=0.2 cases symmetrically and doesn't alter which one produces the larger penalty. And while the 0.015 change in J is small relative to r = 2.4, it is not negligible in absolute terms (0.015 out of a 0.015 baseline penalty is a 100% increase) — the KL coefficient directly and proportionally governs how tightly the trust region constrains policy updates in PPO-style RLHF, regardless of the reward term's absolute size.
Question 13 · Gradient Checkpointing for Memory · hard
Implement gradient checkpointing in transformer training to reduce memory usage. If a transformer has 96 layers, and naive backpropagation requires storing activations of all 96 layers (each 2 GB) = 192 GB memory, while gradient checkpointing recomputes activations on-the-fly, how would you calculate memory savings and the computational overhead?
Memory savings are minimal (10-15%) because backward pass still requires full activation storage.
No memory is saved by gradient checkpointing because it must store checkpoints anyway.
This technique is only useful for extremely deep models (>1000 layers).
Answer: C. Gradient checkpointing stores only √96 ≈ 10 checkpoints + current layer activations = 10 × 2GB + 2GB = 22 GB (88% savings).
ExplanationFirst, gradient checkpointing trades memory for computation. Standard backprop stores all L layer activations: 96 layers × 2GB = 192 GB. Then, with checkpointing: divide into √L ≈ 10 segments. Store activations only at segment boundaries: 10 checkpoints × 2GB + current layer × 2GB ≈ 22 GB total. During backward pass, for each segment, recompute forward pass through all layers in that segment, then backprop. This is why gradient checkpointing is critical for training large models: it reduces memory from 192 GB to about 22 GB, an 88% saving, at the cost of one extra forward pass per segment during the backward pass.
Question 14 · LoRA and Parameter-Efficient Fine-Tuning · hard
Consider the following scenario and evaluate: You fine-tune a 7B parameter LLM using LoRA (Low-Rank Adaptation) with rank r=16 on a single 24GB GPU. The original model has a weight matrix W ∈ ℝ^(4096×4096). How many trainable parameters does LoRA add for this one matrix?
131,072 parameters — LoRA decomposes the update as ΔW = BA where B ∈ ℝ^(4096×16) and A ∈ ℝ^(16×4096), totaling 4096×16 + 16×4096 = 65536 + 65536 = 131,072 trainable parameters
LoRA trains 16,777,216 parameters — the full weight matrix W is still updated during backpropagation, and the low-rank decomposition only affects how gradients are projected, not the total parameter count
16 parameters — the rank r=16 means only 16 scalar values are added as adapters
4,096 parameters — LoRA adds one trainable scalar per row of the original weight matrix
Answer: A. 131,072 parameters — LoRA decomposes the update as ΔW = BA where B ∈ ℝ^(4096×16) and A ∈ ℝ^(16×4096), totaling 4096×16 + 16×4096 = 65536 + 65536 = 131,072 trainable parameters
ExplanationFirst, LoRA (Low-Rank Adaptation) freezes the original weight matrix W and introduces trainable low-rank decomposition: instead of updating W directly, LoRA adds ΔW = BA where B ∈ ℝ^(d×r) and A ∈ ℝ^(r×d). For W(4096×4096) with rank r=16: B has 4096×16 = 65,536 parameters, A has 16×4096 = 65,536 parameters, total 131,072 trainable parameters per matrix. Then, LoRA demonstrates that parameter efficiency and performance can coexist—fundamental for large model fine-tuning. Consequently, full fine-tuning of this single matrix would require updating all 4096×4096 = 16,777,216 parameters, so LoRA's 131,072 trainable parameters are only about 0.78% of that count — which is exactly why rank-16 adapters make fine-tuning a 7B-parameter model feasible on a single 24GB GPU.
Question 15 · Transformer Architecture · hard
A transformer's multi-head self-attention block uses hidden dimension d_model = 768 split evenly across h = 12 heads, so each head has dimension d_k = d_model / h = 64. The block has four weight matrices — query, key, value, and output projections — each with no bias terms. What is the total number of parameters in this block, and how does that total change if h is increased from 12 to 16 while d_model stays at 768?
The four projection matrices are each effectively 768 × 768 in size because a head's individual 768 × 64 slice is one of h such slices that together tile the full 768 × 768 matrix, giving 4 × 768 × 768 = 2,359,296 parameters total; since h × d_k always equals d_model by definition, this total is exactly the same at h = 16 as it is at h = 12.
Only the query, key, and value matrices contribute parameters since the output projection is optional in the base attention formulation, giving 3 × 768 × 768 = 1,769,472 parameters total; like the other three matrices, this count does not depend on h, so it remains 1,769,472 when h rises to 16.
Each of the h heads requires its own full-sized 768 × 768 matrix for all four projections, giving 12 × 4 × 768 × 768 = 28,311,552 parameters at h = 12; because more heads mean more of these full matrices, the total would rise even further to 16 × 4 × 768 × 768 = 37,748,736 once h is increased to 16.
Each projection matrix maps the 768-dimensional input down to a single head's 64-dimensional output, so the four matrices contribute 4 × 768 × 64 = 196,608 parameters at h = 12; raising h to 16 shrinks each head's dimension to d_k = 48, so the total falls to 4 × 768 × 48 = 147,456 parameters.
Answer: A. The four projection matrices are each effectively 768 × 768 in size because a head's individual 768 × 64 slice is one of h such slices that together tile the full 768 × 768 matrix, giving 4 × 768 × 768 = 2,359,296 parameters total; since h × d_k always equals d_model by definition, this total is exactly the same at h = 16 as it is at h = 12.
ExplanationIn multi-head attention, the query, key, and value weight matrices are each sized d_model × d_model, not d_model × d_k, because the h separate d_model × d_k slices used by each head are packed side by side to fill the full d_model × d_model matrix — since h × d_k = d_model by definition of d_k = d_model / h. The output projection matrix is also d_model × d_model, mapping the concatenated heads back to the model dimension. That gives four matrices of 768 × 768 = 589,824 parameters each, for a total of 4 × 589,824 = 2,359,296 parameters. Because h always cancels out of the product h × d_model × d_k = d_model × d_model, this total does not change when h increases from 12 to 16; it stays 2,359,296 either way. The number of heads only changes how the fixed-size projections are split into pieces for the attention computation, not how many weights those projections contain.
Question 16 · RLHF and LLM Alignment · hard
A language model with N = 7 billion parameters is undergoing RLHF alignment. Standard training-compute approximations treat one forward pass as costing 2N FLOPs per token and a full forward-plus-backward training step as costing 6N FLOPs per token. The SFT stage runs one full forward-plus-backward step over a batch of 100,000 tokens using the policy model. The subsequent PPO stage processes the SAME 100,000-token batch by running a full forward-plus-backward update on the policy model, plus one forward-only pass through the frozen reference model (to compute the KL penalty) and one forward-only pass through the reward model (to score completions); neither the reference model nor the reward model is backpropagated through. What is the total FLOPs required for the PPO step, and by what factor does it exceed the FLOPs required for the SFT step?
Because the PPO stage backpropagates through the reference model and reward model as well as the policy model, all three models each cost 6N×tokens = 4.2×10¹⁵ FLOPs, giving a PPO total of 1.26×10¹⁶ FLOPs — three times the 4.2×10¹⁵ FLOPs used by the SFT step.
Since the reference model and reward model are frozen and only produce a single score per sequence, the PPO stage runs three forward-only passes at 2N×tokens = 1.4×10¹⁵ FLOPs each, giving a PPO total of 4.2×10¹⁵ FLOPs — exactly the same compute as the SFT step.
The policy update still costs 6N×tokens = 4.2×10¹⁵ FLOPs, and the two additional forward-only passes through the reference and reward models each add 2N×tokens = 1.4×10¹⁵ FLOPs, bringing the PPO total to 7.0×10¹⁵ FLOPs — five-thirds (about 1.67 times) the 4.2×10¹⁵ FLOPs used by the SFT step.
Treating the SFT step itself as a single forward pass gives 2N×tokens = 1.4×10¹⁵ FLOPs, so the correctly computed PPO total of 7.0×10¹⁵ FLOPs is five times as expensive as the SFT step.
Answer: C. The policy update still costs 6N×tokens = 4.2×10¹⁵ FLOPs, and the two additional forward-only passes through the reference and reward models each add 2N×tokens = 1.4×10¹⁵ FLOPs, bringing the PPO total to 7.0×10¹⁵ FLOPs — five-thirds (about 1.67 times) the 4.2×10¹⁵ FLOPs used by the SFT step.
ExplanationUsing the standard scaling-law approximation, a full forward-plus-backward training step costs 6N FLOPs per token, since the backward pass costs roughly twice the forward pass (2N forward + 4N backward = 6N total). For N = 7×10⁹ parameters and a 100,000-token batch, the SFT step costs 6 × 7×10⁹ × 10⁵ = 4.2×10¹⁵ FLOPs. In the PPO stage, the policy model still undergoes a full forward-plus-backward update, contributing the same 4.2×10¹⁵ FLOPs. The reference model and reward model, however, are frozen — no gradients are computed through them — so each contributes only a forward pass at 2N×tokens = 2 × 7×10⁹ × 10⁵ = 1.4×10¹⁵ FLOPs. Adding the three contributions gives 4.2×10¹⁵ + 1.4×10¹⁵ + 1.4×10¹⁵ = 7.0×10¹⁵ FLOPs for the PPO step. Dividing by the SFT cost, 7.0×10¹⁵ / 4.2×10¹⁵ = 5/3 ≈ 1.67, so the PPO step requires about two-thirds more compute than SFT alone. This is a real cost driver behind why RLHF pipelines are markedly more expensive than the SFT stage that precedes them, even though only one of the three models involved — the policy — is actually being trained; the reference and reward models add pure inference overhead with no matching gradient cost.
Question 17 · Model Compression · hard
A magnitude-based pruning experiment removes 70 percent of a trained CNN's weights, keeping the 30 percent with the largest absolute value. Baseline accuracy is 94.0 percent. Immediately after pruning, accuracy drops to 90.0 percent. After fine-tuning the pruned network for a few more epochs, accuracy rises to 93.2 percent. Compute the accuracy lost to pruning, the accuracy recovered through fine-tuning, and the recovery efficiency (recovered divided by lost), then determine whether this result alone confirms the lottery ticket hypothesis's specific claim about the pruned subnetwork?
The accuracy lost is 4.0 percentage points and the accuracy recovered is 3.2 percentage points, giving a recovery efficiency of 80 percent; this supports the broader observation that sparse subnetworks can nearly match dense-network accuracy after fine-tuning, but it does not confirm the lottery ticket hypothesis's specific claim, since that requires retraining the pruned mask from the network's original random initialization rather than fine-tuning already-trained weights.
Dividing the 3.2-point recovery by the 94.0 percent baseline accuracy yields a recovery efficiency of roughly 3.4 percent, which is too low to support any claim that magnitude-based pruning preserves functionally important weights.
Since the accuracy lost equals 4.0 percentage points and the accuracy recovered equals 3.2 percentage points, the resulting 80 percent recovery efficiency directly confirms the lottery ticket hypothesis, because fine-tuning the pruned weights is equivalent to retraining the winning ticket from its original initialization.
Subtracting the pruned accuracy from the fine-tuned accuracy gives a loss of 3.2 percentage points, so the recovery efficiency exceeds 100 percent, indicating the pruned subnetwork now outperforms the original dense network.
Answer: A. The accuracy lost is 4.0 percentage points and the accuracy recovered is 3.2 percentage points, giving a recovery efficiency of 80 percent; this supports the broader observation that sparse subnetworks can nearly match dense-network accuracy after fine-tuning, but it does not confirm the lottery ticket hypothesis's specific claim, since that requires retraining the pruned mask from the network's original random initialization rather than fine-tuning already-trained weights.
ExplanationAccuracy lost to pruning is 94.0% minus 90.0%, or 4.0 percentage points, and accuracy recovered via fine-tuning is 93.2% minus 90.0%, or 3.2 percentage points, giving a recovery efficiency of 3.2 divided by 4.0, which is 80%. This 80% recovery shows fine-tuning restores most of the lost performance, supporting the general finding that sparse subnetworks can approach dense-network accuracy. But the lottery ticket hypothesis makes a narrower claim: a winning-ticket subnetwork, when reset to its original random initialization and retrained from scratch, can match the dense network's accuracy. Because this experiment fine-tunes the already-trained pruned weights rather than resetting to the original initialization, the 80% recovery efficiency alone does not verify that specific claim. One distractor divides the recovered accuracy by the baseline accuracy instead of by the accuracy actually lost, producing a meaningless 3.4% figure. Another treats fine-tuning of already-trained weights as equivalent to the lottery ticket hypothesis's reset-and-retrain procedure, which conflates two different experimental protocols. A third pairs the wrong two accuracy values when computing the loss, arriving at an impossible recovery efficiency above 100 percent.
Question 18 · Scaling Laws · hard
A research lab has a fixed compute budget of 6×10²³ FLOPs to pretrain a transformer language model. Using the Chinchilla compute-optimal relationship C≈6ND (where N is the parameter count and D is the number of training tokens) together with the empirical compute-optimal finding that the training-token-to-parameter ratio should be approximately 20:1 (D≈20N), what parameter count and token count should the lab choose to train compute-optimally on this budget?
Solving parameters times tokens equal to the compute budget directly, then applying the twenty-to-one token ratio afterward, yields about 173 billion parameters trained on about 3.5 trillion tokens — a calculation that drops the factor of six from the FLOPs relationship.
Applying the twenty-to-one ratio with parameters as the larger quantity flips the intended allocation, giving about 1.4 trillion parameters trained on only about 70 billion tokens — a severely token-starved model.
Substituting the empirical twenty-to-one token-per-parameter ratio into the compute-optimal FLOPs equation and solving for N gives about 70 billion parameters trained on about 1.4 trillion tokens, matching the published Chinchilla-70B configuration.
Treating compute-optimal scaling as requiring parameter count and token count to match exactly, while ignoring the twenty-to-one empirical ratio, gives about 316 billion parameters trained on about 316 billion tokens.
Answer: C. Substituting the empirical twenty-to-one token-per-parameter ratio into the compute-optimal FLOPs equation and solving for N gives about 70 billion parameters trained on about 1.4 trillion tokens, matching the published Chinchilla-70B configuration.
ExplanationSubstituting the compute-optimal ratio D≈20N into C≈6ND gives C≈6N(20N)=120N². Solving for N with C=6×10²³ FLOPs: N²=C/120=6×10²³/120=5×10²¹, so N=√(5×10²¹)≈7.1×10¹⁰, roughly 70 billion parameters. Then D=20N≈1.414×10¹²≈1.4 trillion tokens — matching the actual compute-optimal Chinchilla-70B configuration reported by Hoffmann et al. (2022). Equating parameters and tokens directly, as if compute-optimal training required N≈D, ignores the empirical 20:1 ratio entirely and produces a smaller, mismatched model of about 316 billion parameters on only 316 billion tokens rather than the correct allocation. Applying the ratio in reverse, with the parameter count twenty times the token count, produces a severely token-starved model of about 1.4 trillion parameters trained on only about 70 billion tokens — the opposite of what the empirical scaling data supports. Dropping the factor of six from the FLOPs formula and solving parameters × tokens = C directly before applying the ratio instead yields about 173 billion parameters and 3.5 trillion tokens, overstating both quantities because the compute cost per parameter per token was undercounted by a factor of six.
Question 19 · Explainable AI (SHAP, LIME, attention viz) · hard
Evaluate the interpretability of neural network predictions using SHAP (SHapley Additive exPlanations) values where each feature contributes additively to the model output. Given a model with 50 input features, calculating SHAP values requires approximately 2^50 coalition evaluations without approximation. Determine the computational feasibility and compare against LIME (Local Interpretable Model-agnostic Explanations) which requires K=100 perturbed samples, analyzing which method provides more reliable feature importance estimates and why?
SHAP requires 2^50≈10¹⁵ evaluations, computationally infeasible without approximation.
SHAP with 2^50 evaluations is trivial since modern GPUs handle 10¹⁵ operations in milliseconds, so SHAP is always preferable to LIME which provides only local, sample-based approximations.
LIME with K=100 samples is computationally equivalent to SHAP since both approximate feature importance.
Both SHAP and LIME require approximately equal computation when optimized, so practitioners should use LIME exclusively because it's simpler to implement with fewer hyperparameters to tune.
Answer: A. SHAP requires 2^50≈10¹⁵ evaluations, computationally infeasible without approximation.
ExplanationSHAP values are based on Shapley values from coalitional game theory, requiring summation over all 2^n feature coalitions — for n=50, this is 2^50 ≈ 1.125×10¹⁵ evaluations, which is computationally prohibitive to compute exactly. Approximations exist: TreeSHAP (polynomial for tree models), KernelSHAP (Monte Carlo, roughly 1,000 samples), and DeepSHAP (a neural-network fast-track). LIME instead fits a local linear surrogate model using only K=100 perturbed samples, so it never touches the 2^50 coalition space at all — far cheaper, but only locally faithful rather than a theoretically exact global attribution. This correctly balances the computational infeasibility of exact SHAP against the practical, approximation-based alternatives used in real systems.
Question 20 · LoRA and Parameter-Efficient Fine-Tuning · hard
A pretrained transformer encoder (similar in scale to BERT-base) has hidden dimension d_model = 768, 12 transformer layers, and approximately 110 million total parameters. A team fine-tunes it using LoRA with rank r = 8, injecting trainable low-rank matrices A (r × d_model) and B (d_model × r) into only the query and value projection matrices of every layer, while every original weight stays frozen. Given that each adapted projection needs both an A matrix and a B matrix, and that both the query and value projections are adapted in each of the 12 layers, what is the total number of additional trainable parameters LoRA introduces, and what fraction of the model's 110 million total parameters does this represent?
LoRA reconstructs each adapted weight matrix as a full 768×768 update ΔW = BA for both the query and value projections in every layer, adding 2×768×768×12 = 14,155,776 parameters, about 12.9% of the 110 million total — only a modest reduction because the effective update is still full rank.
LoRA adds r×d_model parameters for each of the query and value projections per layer, giving 2×8×768×12 = 147,456 additional parameters, roughly 0.13% of the 110 million total parameters.
LoRA adds 2×(r×d_model) parameters for each of matrices A and B per adapted projection, and with both query and value projections adapted in every layer this totals 4×r×d_model×layers = 4×8×768×12 = 294,912 additional parameters, about 0.27% of the 110 million total — a reduction exceeding 99.7% — because pretrained weights already encode general representations and task adaptation needs only a low-rank correction within that space, not a full-rank rewrite.
LoRA adds 4×r×d_model parameters per layer but must also adapt the key projection alongside query and value, giving 6×8×768×12 = 442,368 additional parameters, about 0.40% of the 110 million total.
Answer: C. LoRA adds 2×(r×d_model) parameters for each of matrices A and B per adapted projection, and with both query and value projections adapted in every layer this totals 4×r×d_model×layers = 4×8×768×12 = 294,912 additional parameters, about 0.27% of the 110 million total — a reduction exceeding 99.7% — because pretrained weights already encode general representations and task adaptation needs only a low-rank correction within that space, not a full-rank rewrite.
ExplanationEach LoRA-adapted projection uses two matrices: A with shape r × d_model and B with shape d_model × r, contributing r×d_model parameters each, so 2×(r×d_model) = 2×(8×768) = 12,288 trainable parameters per adapted matrix — never the original matrix's full d_model×d_model, since A and B are stored and multiplied on the fly rather than merged into an explicit dense update. With both the query and value projections adapted in every layer, that is 2×12,288 = 24,576 parameters per layer, and across all 12 layers the total is 4×r×d_model×layers = 4×8×768×12 = 294,912 additional trainable parameters. Against the model's 110 million total parameters, that is 294,912/110,000,000 ≈ 0.27%, meaning more than 99.7% of the original parameters stay frozen. Performance holds up despite this because the pretrained weights already encode broad, general-purpose representations from large-scale pretraining; the core LoRA insight is that the *change* needed to specialize a model for a downstream task has low intrinsic rank — the useful update lives in a small subspace, so a rank-8 correction can capture it without the model needing to relearn its general knowledge, which also guards against catastrophic forgetting and overfitting on small fine-tuning datasets. Treating the update as if it must be materialized as a full 768×768 matrix per projection overstates the parameter count 48-fold, since that ignores the entire point of factoring the update into two skinny matrices; counting only one of A or B per projection undercounts it by exactly half; and silently adapting the key projection, which the setup explicitly excludes, inflates the count by assuming a wider adaptation scope than was specified.