In a GAN, the discriminator outputs D(x) = 0.8 for a real image and D(G(z)) = 0.3 for a generated image. Using the standard GAN loss, what is the generator's loss for this sample?
0.357 — generator loss = ln(1 - D(G(z))) = ln(0.7) = -0.357, so loss = 0.357
0.700 — generator loss equals 1 minus the discriminator's output on fake: 1 - D(G(z)) = 1 - 0.3 = 0.7
0.800 — generator loss uses the discriminator's score on real images
Answer: A. 1.204 — generator loss = -ln(D(G(z))) = -ln(0.3) = 1.204 (non-saturating formulation)
ExplanationThe original GAN minimax formulation has the generator minimizing ln(1 - D(G(z))). However, this saturates when D(G(z)) is small (early training). The practical non-saturating alternative has the generator MAXIMIZING ln(D(G(z))), or equivalently minimizing -ln(D(G(z))). With D(G(z)) = 0.3: Generator loss = -ln(0.3) = -(-1.204) = 1.204. The generator wants D(G(z)) → 1.0 (fool the discriminator). At D(G(z))=0.3, the loss is high (1.204), pushing the generator to produce more convincing images. If D(G(z))=0.9, the loss would be -ln(0.9) = 0.105, indicating the discriminator is almost fooled. This produces a high loss because the negative log of a small probability (0.3) yields a large penalty, driving the generator to improve its outputs.
Question 62 · Q-Learning Update Rule · hard
Consider a Q-learning scenario with discount factor gamma=0.9 where the agent is in state s, takes action a, receives reward r=10, and transitions to state s'. The current Q-table has Q(s,a)=5 and max_a' Q(s',a')=8. With learning rate alpha=0.1, what is the updated Q(s,a)?
7.2 — using only the discounted next-state value, gamma*max Q(s',a') = 0.9*8 = 7.2, treated (incorrectly) as the updated Q(s,a) itself.
ExplanationQ-learning update: Q(s,a) ← Q(s,a) + alpha * (TD_target - Q(s,a)). TD_target = r + gamma * max_a' Q(s', a') = 10 + 0.9 * 8 = 10 + 7.2 = 17.2. TD_error = TD_target - Q(s,a) = 17.2 - 5 = 12.2. Update: Q(s,a) = 5 + 0.1 * 12.2 = 5 + 1.22 = 6.22. The Q-value moves 10% of the way from 5 toward the target 17.2. With alpha=0.1, the update is conservative — many iterations are needed for Q(s,a) to converge to 17.2. A common mistake is treating the discounted next-state value alone (0.9*8 = 7.2) as the answer, or updating using only the reward term (5 + 0.1*10 = 6.0) while ignoring the discounted future value — both skip the full TD-target-minus-current-estimate structure of the Bellman update.
Question 63 · Adam Optimizer Internals · hard
You use Adam optimizer with beta1=0.9, beta2=0.999, epsilon=1e-8. At step t=1, the gradient is g_1=4.0. What are the bias-corrected first moment (m_hat) and second moment (v_hat)?
ExplanationAt t=1, initializing m_0=0, v_0=0: First moment: m_1 = beta1*m_0 + (1-beta1)*g_1 = 0.9*0 + 0.1*4.0 = 0.4. Second moment: v_1 = beta2*v_0 + (1-beta2)*g_1^2 = 0.999*0 + 0.001*16.0 = 0.016. These raw estimates are biased toward zero (initialized at 0). Bias correction: m_hat = m_1/(1-beta1^t) = 0.4/(1-0.9) = 0.4/0.1 = 4.0. v_hat = v_1/(1-beta2^t) = 0.016/(1-0.999) = 0.016/0.001 = 16.0. The correction is massive at t=1 (10x for m, 1000x for v) because the exponential averages haven't warmed up yet. The parameter update would be: alpha * m_hat / (sqrt(v_hat) + epsilon) = alpha * 4.0 / (4.0 + 1e-8) ≈ alpha.
Question 64 · BERT Pre-Training Strategy · hard
In BERT's masked language model pre-training, 15% of tokens are selected for masking. Of those selected tokens, 80% are replaced with [MASK], 10% with a random token, and 10% remain unchanged. If a sentence has 100 tokens, how many tokens are replaced with [MASK], random, and unchanged respectively?
Exactly 12 get the [MASK] symbol, about 1.5 become random words, about 1.5 stay original — from the 15 chosen: 80%=12, 10%=1.5, 10%=1.5
All 80 words masked, 10 randomized, 10 kept — applying the 80/10/10 split directly to the full sentence length of 100
Every one of the 15 chosen words becomes [MASK] — no randomization or preservation step occurs at all
The 12/1.5/1.5 split is correct but exclusively during the fine-tuning phase, never during initial self-supervised learning
Answer: A. Exactly 12 get the [MASK] symbol, about 1.5 become random words, about 1.5 stay original — from the 15 chosen: 80%=12, 10%=1.5, 10%=1.5
ExplanationBERT's MLM strategy: Step 1: Select 15% of tokens. For 100 tokens: 15 selected. Step 2: Of these 15 selected tokens: 80% → [MASK]: 0.8 * 15 = 12 tokens replaced with [MASK]. 10% → random: 0.1 * 15 = 1.5 tokens replaced with random vocabulary words. 10% → unchanged: 0.1 * 15 = 1.5 tokens left as-is. The 80/10/10 split is deliberate: if ALL selected tokens became [MASK], the model would learn that [MASK] tokens are the only ones to predict, creating a train-inference mismatch (no [MASK] tokens at inference). The random replacement prevents the model from assuming unmasked tokens are always correct. The unchanged portion teaches the model to represent ALL tokens well, not just masked ones.
Question 65 · Language Model Perplexity · hard
You compute the perplexity of a language model on a test set of 4 tokens with predicted probabilities [0.5, 0.25, 0.5, 0.125] for the actual next tokens. What is the perplexity?
0.34 — average probability: (0.5+0.25+0.5+0.125)/4 = 0.34
8.0 — perplexity = 1/min(p_i) = 1/0.125 = 8
Answer: B. 3.36 — PPL = exp(-(1/N)*sum(ln(p_i))) = exp(-(1/4)*(ln0.5+ln0.25+ln0.5+ln0.125)) which equals 2^1.75 = 3.36
ExplanationPerplexity = exp(-(1/N) * sum(ln(p_i))). Using log base 2 equivalently: PPL = 2^H where H = -(1/N)*sum(log2(p_i)). log2(0.5) = -1. log2(0.25) = -2. log2(0.5) = -1. log2(0.125) = -3. Sum = -1 + -2 + -1 + -3 = -7. H = -(1/4)*(-7) = 7/4 = 1.75 bits. PPL = 2^1.75 = 2^1 * 2^0.75 = 2 * 1.682 = 3.36. Interpretation: the model is as uncertain as if choosing uniformly among ~3.36 options at each step. Lower perplexity is better — a perfect model predicting p=1.0 for every token would have PPL=1. Taking a plain arithmetic mean of the reciprocal probabilities (16/4 = 4.0) is a common mistake: true perplexity requires the geometric mean of the reciprocals, obtained by exponentiating the average negative log-probability, not a simple arithmetic average. Averaging the raw probabilities (0.34) or using only the minimum probability (8.0) are likewise not how perplexity is defined.
Question 66 · L2 Regularization and Weight Decay · hard
You apply L2 regularization (weight decay lambda=0.01) to a weight w=2.0. The gradient of the loss with respect to w is dL/dw=0.5. With learning rate alpha=0.1, what is the updated weight?
1.948 — total gradient = dL/dw + lambda*w = 0.5 + 0.01*2.0 = 0.52, new w = 2.0 - 0.1*0.52 = 1.948
1.950 — w = w - alpha*dL/dw = 2.0 - 0.1*0.5 = 1.95, regularization has no effect on updates
1.800 — w = w - alpha*(dL/dw + lambda) = 2.0 - 0.1*(0.5+0.01*100) = 2.0 - 0.2
1.948 — but the weight should also be clipped to [0, 1] range after L2 regularization
Answer: A. 1.948 — total gradient = dL/dw + lambda*w = 0.5 + 0.01*2.0 = 0.52, new w = 2.0 - 0.1*0.52 = 1.948
ExplanationWith L2 regularization, the total loss becomes L_total = L_original + (lambda/2)*sum(w_i^2). The gradient of the regularization term with respect to w is lambda*w. Total gradient: dL_total/dw = dL/dw + lambda*w = 0.5 + 0.01*2.0 = 0.5 + 0.02 = 0.52. Update: w_new = w - alpha * 0.52 = 2.0 - 0.1*0.52 = 2.0 - 0.052 = 1.948. The L2 term adds a small push toward zero (0.02 added to gradient), which is why it is called weight decay — weights shrink slightly each step beyond what the loss gradient alone would cause. This discourages large weights, acting as a form of regularization.
Question 67 · ResNet Skip Connections · hard
In a residual network (ResNet), a residual block computes y = F(x) + x where F is a 2-layer CNN. If x has shape 64x64x128 but F(x) has shape 32x32x256 (due to stride=2 and more filters), what must be done to make the skip connection work?
Apply a 1x1 convolution with stride=2 and 256 filters to x, producing 32x32x256 to match F(x) — this is the projection shortcut
Pad x with zeros to match 32x32x256 — add 128 zero channels and subsample spatially
Skip connections are impossible when dimensions don't match — use a plain network instead
Interpolate F(x) back up to 64x64x128 to match x — upsample the residual
Answer: A. Apply a 1x1 convolution with stride=2 and 256 filters to x, producing 32x32x256 to match F(x) — this is the projection shortcut
ExplanationWhen the skip connection dimensions don't match (spatial or channel), ResNet uses a projection shortcut: a 1x1 convolution that simultaneously (1) changes the number of channels from 128 to 256, and (2) applies stride=2 to reduce spatial dimensions from 64x64 to 32x32. The 1x1 conv has 256 filters, each of size 1x1x128, with stride=2. Parameters: 256*128*1*1 + 256 = 32,768 + 256 = 33,024. Now both branches produce 32x32x256 tensors that can be added element-wise: y = F(x) + W_s*x where W_s is the projection. The original ResNet paper (He et al. 2015) showed that identity shortcuts (when dimensions match) + projection shortcuts (when they don't) outperform plain networks on ImageNet. This produces the solution because the 1x1 convolution simultaneously transforms both spatial resolution and channel depth, enabling the skip connection to match dimensions for element-wise addition.
Question 68 · Gradient Clipping · hard
You train a model with gradient clipping at max_norm=1.0. The gradient vector for one step is g = [3, 4] (L2 norm = 5). After clipping, what is the gradient used for the update?
[0.6, 0.8] — the gradient is scaled to unit norm: g_clipped = g * (max_norm / ||g||) = [3, 4] * (1/5) = [0.6, 0.8]
[1, 1] — each component is individually clipped to max_norm=1.0
[3, 4] — the gradient is unchanged because we clip norms, not values
[0.3, 0.4] — the gradient is divided by its norm squared: [3/25, 4/25]
Answer: A. [0.6, 0.8] — the gradient is scaled to unit norm: g_clipped = g * (max_norm / ||g||) = [3, 4] * (1/5) = [0.6, 0.8]
ExplanationGradient clipping by norm: if ||g||_2 > max_norm, scale g by max_norm/||g||_2. ||g||_2 = sqrt(3^2 + 4^2) = sqrt(9 + 16) = sqrt(25) = 5. Since 5 > 1.0 (max_norm), we clip: g_clipped = g * (1.0/5) = [3*0.2, 4*0.2] = [0.6, 0.8]. Verify: ||g_clipped||_2 = sqrt(0.36 + 0.64) = sqrt(1.0) = 1.0. The clipped gradient has the same DIRECTION as the original (both point toward [3,4]) but its magnitude is capped at 1.0. This prevents gradient explosions in deep networks (especially RNNs/Transformers) where gradients can grow exponentially through many layers. Note: this is different from per-component clipping (option B), which would change the gradient direction.
Question 69 · Self-Attention Computation · hard
In a transformer with d_model=512 and 8 attention heads, each head has d_k = d_v = 64. For a sequence of length 10, what is the shape of the attention weight matrix for ONE head, and what is the computational complexity of computing it?
Attention weights shape: (10, 10) — computed as softmax(QK^T / sqrt(64)) where Q and K are both (10, 64). QK^T is (10,64)×(64,10) = (10,10). Complexity: O(n^2 × d_k) = O(100 × 64) = O(6400) per head
Shape: (64, 64) — the attention matrix has dimensions d_k × d_k, independent of sequence length
Shape: (10, 512) — attention weights have the full d_model dimension, not reduced by head count
Shape: (10, 10) but complexity is O(n × d_k) = O(640) — the matrix multiply is linear in sequence length
Answer: A. Attention weights shape: (10, 10) — computed as softmax(QK^T / sqrt(64)) where Q and K are both (10, 64). QK^T is (10,64)×(64,10) = (10,10). Complexity: O(n^2 × d_k) = O(100 × 64) = O(6400) per head
ExplanationFor one head: Q, K are (seq_len, d_k) = (10, 64). Attention = softmax(QK^T/sqrt(d_k)). QK^T matrix multiply: (10,64)×(64,10) = (10,10). Each entry is the dot product of a query and key vector. The 10×10 matrix represents how much each of the 10 positions attends to every other position. Complexity of the matmul is O(n²×d_k) — quadratic in sequence length, which is why long sequences are expensive: doubling n from 10 to 20 would roughly quadruple the matmul cost to about O(400×64)=O(25600), while doubling d_k only doubles it. This quadratic-in-n, linear-in-d_k scaling is the specific reason self-attention becomes the dominant cost as context length grows, and it holds identically for each of the 8 heads since d_k=64 is fixed per head regardless of head count.
Question 70 · GAN Loss Function · hard
In a GAN, the discriminator D outputs D(x)=0.9 for a real image and D(G(z))=0.3 for a generated image. Using the standard GAN loss L_D = -[log(D(x)) + log(1-D(G(z)))] and the values ln(0.9)=-0.105, ln(0.7)=-0.357, ln(0.3)=-1.204, what is the discriminator loss?
ExplanationStandard GAN discriminator loss: L_D = -E[log D(x)] - E[log(1-D(G(z)))]. For one real sample D(x)=0.9 and one fake D(G(z))=0.3: L_D = -[ln(0.9) + ln(1-0.3)] = -[ln(0.9) + ln(0.7)] = -[(-0.105)+(-0.357)] = 0.462. Lower loss means better discrimination. A perfect discriminator (D(x)=1, D(G(z))=0) would have L_D = 0; here L_D=0.462 reflects that the discriminator gives the real image only a moderately high score (0.9, not 1.0) and still assigns the fake image a nontrivial probability (0.3) of being real, so it has not yet converged to perfect separation of the two classes.
Question 71 · BERT MLM Strategy · hard
In BERT masked language modeling, 15% of tokens are selected. Of those, 80% are replaced with [MASK], 10% with a random token, 10% kept unchanged. In a sequence of 100 tokens, how many tokens are [MASK], how many are random replacements, and how many are unchanged but still predicted?
15 selected total: 12 replaced with [MASK], 1.5 (≈1-2) random replacements, 1.5 (≈1-2) unchanged. The model must predict all 15, not just the [MASK] tokens, which prevents the model from learning that [MASK] signals prediction
15 replaced with [MASK], 10 random, 10 unchanged = 35 total modified tokens; the percentages apply to all 100 tokens
80 tokens become [MASK], 10 random, 10 unchanged; the percentages apply to all 100 tokens directly
15 replaced with [MASK], 0 random, 0 unchanged; only [MASK] tokens exist in practice
Answer: A. 15 selected total: 12 replaced with [MASK], 1.5 (≈1-2) random replacements, 1.5 (≈1-2) unchanged. The model must predict all 15, not just the [MASK] tokens, which prevents the model from learning that [MASK] signals prediction
Explanation15% of 100 = 15 tokens selected for prediction. Of those 15: 80% → [MASK] = 12 tokens. 10% → random token = 1.5 ≈ 1-2 tokens. 10% → unchanged = 1.5 ≈ 1-2 tokens. The model predicts all 15 positions. The random+unchanged tokens prevent the model from relying on the [MASK] token as a signal, since sometimes it must predict tokens that look normal.
Question 72 · Perplexity · hard
You compute perplexity for a language model using the formula PPL = exp(-1/N × sum(ln P(wi))). On a test set of 4 tokens, the model assigns probabilities: P(w1)=0.5, P(w2)=0.25, P(w3)=0.5, P(w4)=0.25. What is the perplexity?
Sum of log probs = ln(0.5)+ln(0.25)+ln(0.5)+ln(0.25) = -0.693+(-1.386)+(-0.693)+(-1.386) = -4.158. Average = -4.158/4 = -1.040. PPL = exp(1.040) ≈ 2.828
PPL = 1/(0.5×0.25×0.5×0.25) = 1/0.015625 = 64; perplexity is the reciprocal of the joint probability
PPL = (0.5+0.25+0.5+0.25)/4 = 0.375; perplexity is the average probability
PPL = 4 × exp(-0.5-0.25-0.5-0.25) = 4 × exp(-1.5) ≈ 0.893; multiplying N by the exponential of raw probabilities
Answer: A. Sum of log probs = ln(0.5)+ln(0.25)+ln(0.5)+ln(0.25) = -0.693+(-1.386)+(-0.693)+(-1.386) = -4.158. Average = -4.158/4 = -1.040. PPL = exp(1.040) ≈ 2.828
ExplanationPPL = exp(-(1/N)Σln P(wi)). ln(0.5) = -0.693, ln(0.25) = -1.386. Sum = 2(-0.693) + 2(-1.386) = -1.386 - 2.772 = -4.158. Average = -4.158/4 = -1.0395. PPL = exp(1.0395) ≈ 2.828. Equivalently, PPL = (P(w1)×...×P(w4))^(-1/4) = (0.5×0.25×0.5×0.25)^(-0.25) = (0.015625)^(-0.25) = (1/64)^(-0.25) = 64^0.25 = 2.828. Lower perplexity means the model assigns higher probability to the observed sequence, so a PPL of 2.828 reflects moderately confident predictions on this test set.
Question 73 · L2 Regularization · hard
Consider the following scenario: In L2 regularization, the loss becomes L_total = L_data + (lambda/2) × ||w||^2. If w = [3, 4] and lambda = 0.01, what is the regularization term, and what is the gradient of the regularization with respect to w?
||w||^2 = 3^2 + 4^2 = 25. Reg term = 0.01/2 × 25 = 0.125. Gradient = lambda × w = [0.01×3, 0.01×4] = [0.03, 0.04]; the regularization pushes weights toward zero
||w||^2 = sqrt(9+16) = 5. Reg term = 0.01/2 × 5 = 0.025. Gradient = [0.005, 0.005]; using L2 norm instead of squared L2 norm
Reg term = 0.01 × (3+4) = 0.07. Gradient = [0.01, 0.01]; this is L1 regularization, not L2
||w||^2 = 25. Reg term = 0.01 × 25 = 0.25. Gradient = [0.06, 0.08]; missing the 1/2 factor
Answer: A. ||w||^2 = 3^2 + 4^2 = 25. Reg term = 0.01/2 × 25 = 0.125. Gradient = lambda × w = [0.01×3, 0.01×4] = [0.03, 0.04]; the regularization pushes weights toward zero
Explanation||w||^2 = 3² + 4² = 9 + 16 = 25. L_reg = (λ/2)||w||² = (0.01/2)(25) = 0.125. The 1/2 factor makes the gradient cleaner: d/dw[(λ/2)||w||²] = λw. So gradient = [0.01×3, 0.01×4] = [0.03, 0.04]. This gradient acts as "weight decay" — subtracting a fraction of the weight at each step, shrinking weights toward zero to prevent overfitting.
Question 74 · ResNet Skip Connections · hard
In a ResNet skip connection, the output is H(x) = F(x) + x, where F(x) is the residual learned by the block. If the input x has values [2.0, -1.0, 3.0] and the block computes F(x) = [0.1, -0.3, 0.2], what is the output? Why does this help with gradient flow?
H(x) = [2.0+0.1, -1.0+(-0.3), 3.0+0.2] = [2.1, -1.3, 3.2]. The skip connection ensures dH/dx = dF/dx + I, so gradients always have a component of 1 flowing backward, preventing vanishing gradients even in very deep networks
H(x) = [0.1, -0.3, 0.2] — the skip connection replaces x with F(x), not adds them
H(x) = [2.0, -1.0, 3.0] — if F(x) is small, the skip connection ignores it entirely
H(x) = [2.1, -1.3, 3.2] but gradient flow is dH/dx = dF/dx only; the identity path does not carry gradients
Answer: A. H(x) = [2.0+0.1, -1.0+(-0.3), 3.0+0.2] = [2.1, -1.3, 3.2]. The skip connection ensures dH/dx = dF/dx + I, so gradients always have a component of 1 flowing backward, preventing vanishing gradients even in very deep networks
ExplanationH(x) = F(x) + x = [0.1+2.0, -0.3+(-1.0), 0.2+3.0] = [2.1, -1.3, 3.2]. The key insight: dH/dx = dF/dx + I (identity matrix). Even if dF/dx vanishes (gradients through conv layers decay), the +I term guarantees a gradient of 1 flows directly back. This is why ResNets can train 100+ layers while plain networks fail at 20+. The network only needs to learn the residual F(x) = H(x)-x, which is often small — close to zero when the identity mapping is already near-optimal for that block.
Question 75 · Causal Attention Mask · hard
In a causal (autoregressive) attention mask for a sequence of length 4, what does the mask matrix look like, and why is it necessary? Analyze the computation step by step and determine the exact numerical answer?
Mask = [[1,0,0,0],[1,1,0,0],[1,1,1,0],[1,1,1,1]] where 1 = attend, 0 = blocked. Position i can only attend to positions 0 through i. This prevents the model from seeing future tokens during training, matching the autoregressive generation constraint at inference
Mask = [[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1]] — causal models use full attention; the autoregressive constraint only applies during generation
Mask = [[1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1]] — each position only attends to itself in causal attention
Mask = [[0,1,1,1],[0,0,1,1],[0,0,0,1],[0,0,0,0]] — each position attends to future tokens only, as it already knows its own value
Answer: A. Mask = [[1,0,0,0],[1,1,0,0],[1,1,1,0],[1,1,1,1]] where 1 = attend, 0 = blocked. Position i can only attend to positions 0 through i. This prevents the model from seeing future tokens during training, matching the autoregressive generation constraint at inference
ExplanationThe lower-triangular mask ensures position i attends only to positions ≤ i. Row 0: [1,0,0,0] (token 0 sees only itself). Row 1: [1,1,0,0] (token 1 sees tokens 0-1). Row 2: [1,1,1,0] (tokens 0-2). Row 3: [1,1,1,1] (all tokens). Masked positions get -inf before softmax, zeroing their attention weights. Without this, during training the model could cheat by looking at the token it is supposed to predict.
Question 76 · Vision Transformer Architecture · hard
In a Vision Transformer (ViT), a 224×224×3 image is split into non-overlapping patches of size 16×16. Each patch is flattened and linearly projected to dimension 768. How many patch tokens are produced, what is the flattened patch size before projection, and how many parameters does the projection layer have?
Patches: (224/16)² = 14² = 196 tokens. Each patch: 16×16×3 = 768 pixels flattened. Projection: 768→768 means W is 768×768 = 589,824 weights + 768 bias = 590,592. Coincidentally, patch size equals projection dimension here
Patches: 224/16 = 14 tokens total (not squared) because patches are 1D. Projection params = 14×768 = 10,752
Patches: 196 tokens. Projection has zero parameters because patches are just reshaped, not learned
Answer: A. Patches: (224/16)² = 14² = 196 tokens. Each patch: 16×16×3 = 768 pixels flattened. Projection: 768→768 means W is 768×768 = 589,824 weights + 768 bias = 590,592. Coincidentally, patch size equals projection dimension here
ExplanationImage 224×224 divided into 16×16 patches: 224/16 = 14 patches per side, 14×14 = 196 total patches (tokens). Each patch has 16×16 pixels × 3 channels = 768 values when flattened. The linear projection maps 768-dim → 768-dim, so W has 768×768 = 589,824 parameters + 768 bias = 590,592 total. The input and output dimensions being equal (768) is a design choice in ViT-Base. A [CLS] token is prepended, making the total sequence 197 tokens fed to the transformer encoder.
Question 77 · PPO Clipped Objective · hard
Consider the following scenario and analyze the result: In Proximal Policy Optimization (PPO), the clipped objective is L = min(r_t × A_t, clip(r_t, 1-eps, 1+eps) × A_t) where r_t = pi_new/pi_old. If eps=0.2, r_t=1.5, and advantage A_t=2.0, what is the clipped loss value?
Unclipped: r_t × A_t = 1.5 × 2.0 = 3.0. Clipped r_t: clip(1.5, 0.8, 1.2) = 1.2. Clipped term: 1.2 × 2.0 = 2.4. L = min(3.0, 2.4) = 2.4. The clip prevents the policy from changing too much in a single update by capping the probability ratio at 1.2
L = 1.5 × 2.0 = 3.0; clipping is only applied when the advantage is negative
L = clip(1.5×2.0, 0.8, 1.2) = clip(3.0, 0.8, 1.2) = 1.2; the clip is applied to the final product
L = max(3.0, 2.4) = 3.0; PPO takes the maximum to encourage exploration
Answer: A. Unclipped: r_t × A_t = 1.5 × 2.0 = 3.0. Clipped r_t: clip(1.5, 0.8, 1.2) = 1.2. Clipped term: 1.2 × 2.0 = 2.4. L = min(3.0, 2.4) = 2.4. The clip prevents the policy from changing too much in a single update by capping the probability ratio at 1.2
ExplanationPPO clips the probability ratio r_t, not the final loss. r_t = 1.5 means the new policy is 50% more likely to take this action than the old policy. Unclipped objective: 1.5 × 2.0 = 3.0. Clipped r_t: clip(1.5, 1-0.2, 1+0.2) = clip(1.5, 0.8, 1.2) = 1.2 (capped at upper bound). Clipped objective: 1.2 × 2.0 = 2.4. L = min(3.0, 2.4) = 2.4. The min ensures we use the more conservative estimate. Since A_t > 0 (good action), PPO prevents over-exploitation by limiting how much we can increase this action's probability.
Question 78 · VAE Reparameterization · hard
In a Variational Autoencoder (VAE), the encoder outputs mu=[1.0, 0.0] and log_var=[-2.0, 0.0] for a latent sample, and the reparameterization trick z = mu + exp(0.5 × log_var) ⊙ epsilon is applied with epsilon=[1.0, -1.0]. What is z, and what is the KL divergence term −0.5 × Σ(1 + log_var − mu² − exp(log_var)) for this sample?
The correct reparameterization gives z = mu + std ⊙ epsilon = [1.368, -1.0] using std = exp(0.5×log_var) = [0.368, 1.0], but the KL divergence is KL = -0.5 × Σ(1 + log_var − mu²) = -0.5 × [(1−2−1) + (1+0−0)] = -0.5 × (−1) = 0.5
Since log_var already represents the variance term, z = mu + log_var ⊙ epsilon = [1.0 + (−2.0)×1.0, 0.0 + 0.0×(−1.0)] = [−1.0, 0.0], and the KL divergence evaluates to KL = -0.5 × Σ(1 + log_var − mu² − exp(log_var)) = 1.068
z = mu + epsilon = [1.0+1.0, 0.0+(−1.0)] = [2.0, −1.0], because the reparameterization trick simply adds the noise vector epsilon to the mean without scaling by the standard deviation, giving KL = -0.5 × Σ(1 + log_var − mu²) = 0.5
ExplanationReparameterization writes z = mu + std ⊙ epsilon, where std = exp(0.5 × log_var) — never log_var directly, and never a plain sum of mu and epsilon, since that would discard the encoder's learned variance entirely. Here std = exp(0.5 × [-2.0, 0.0]) = exp([-1.0, 0.0]) = [0.368, 1.0], so z = [1.0 + 0.368×1.0, 0.0 + 1.0×(-1.0)] = [1.368, -1.0]. The KL divergence between the approximate posterior N(mu, exp(log_var)) and the standard normal prior N(0,1) is KL = -0.5 × Σ(1 + log_var − mu² − exp(log_var)), and it depends only on mu and log_var — never on the sampled epsilon or z, since it measures how far the whole learned distribution sits from the prior, not where one sample landed. Per dimension: the first gives 1 + (-2.0) − 1.0² − exp(-2.0) = 1 − 2 − 1 − 0.135 = -2.135, and the second gives 1 + 0 − 0² − exp(0) = 1 + 0 − 0 − 1 = 0. Summing gives -2.135, so KL = -0.5 × (-2.135) = 1.068. Dropping the exp(log_var) term from the formula understates the penalty (yielding 0.5 instead of 1.068), and treating log_var as though it were already the standard deviation skips the required exponentiation step entirely, producing a z with the wrong scale and sign pattern.
Question 79 · Knowledge Distillation Loss · hard
Consider the following scenario and analyze the result: In knowledge distillation, the student loss is L = alpha×T²×KL(soft_teacher || soft_student) + (1-alpha)×CE(hard_labels, student). If temperature T=4, alpha=0.7, the soft loss is 0.5, and the hard loss is 1.2, what is the total student loss?
L = 0.7 × 4² × 0.5 + (1-0.7) × 1.2 = 0.7 × 16 × 0.5 + 0.3 × 1.2 = 5.6 + 0.36 = 5.96. The T² factor compensates for the reduced gradient magnitude when using soft targets at high temperature, because softmax(z/T) has gradients scaled by 1/T²
L = 0.7 × 0.5 + 0.3 × 1.2 = 0.35 + 0.36 = 0.71; temperature does not appear in the loss formula
L = 4 × (0.7 × 0.5 + 0.3 × 1.2) = 4 × 0.71 = 2.84; multiply the entire loss by T, not T²
L = 0.7 × 0.5 + 0.3 × 1.2 × 16 = 0.35 + 5.76 = 6.11; T² multiplies the hard loss instead of the soft loss
Answer: A. L = 0.7 × 4² × 0.5 + (1-0.7) × 1.2 = 0.7 × 16 × 0.5 + 0.3 × 1.2 = 5.6 + 0.36 = 5.96. The T² factor compensates for the reduced gradient magnitude when using soft targets at high temperature, because softmax(z/T) has gradients scaled by 1/T²
ExplanationThe distillation loss formula: L = α×T²×L_soft + (1-α)×L_hard. With α=0.7, T=4: L = 0.7 × 16 × 0.5 + 0.3 × 1.2 = 5.6 + 0.36 = 5.96. The T² scaling is crucial: when computing softmax at temperature T, the gradients are scaled down by 1/T², so multiplying by T² restores them to the correct magnitude. Without T², the soft loss contribution would be negligible at high temperatures. Alpha=0.7 means the student learns 70% from the teacher's soft predictions and 30% from the ground truth labels.
Question 80 · Diffusion Model Forward Process · hard
In a diffusion model (DDPM), the forward process adds noise at each step: x_t = sqrt(alpha_bar_t) × x_0 + sqrt(1 - alpha_bar_t) × epsilon. If alpha_bar_t = 0.01 at the final timestep (t=1000), what fraction of the original signal remains, and what does x_1000 approximately look like?
Signal component: sqrt(0.01) = 0.1, so only 10% of x_0's magnitude remains. Noise component: sqrt(1-0.01) = sqrt(0.99) = 0.995, so 99.5% is noise. x_1000 ≈ 0.1×x_0 + 0.995×epsilon, which is essentially pure Gaussian noise. The reverse process then learns to denoise step by step from this near-pure noise back to data
50% signal and 50% noise because alpha_bar divides equally between signal and noise at the final step
100% signal remains because alpha_bar_t = 0.01 means only 1% noise is added at the final step
x_1000 = x_0 always; the forward process is reversed before the final step so no information is lost
Answer: A. Signal component: sqrt(0.01) = 0.1, so only 10% of x_0's magnitude remains. Noise component: sqrt(1-0.01) = sqrt(0.99) = 0.995, so 99.5% is noise. x_1000 ≈ 0.1×x_0 + 0.995×epsilon, which is essentially pure Gaussian noise. The reverse process then learns to denoise step by step from this near-pure noise back to data
ExplanationAt timestep t=1000 with alpha_bar_t=0.01: x_t = sqrt(0.01)×x_0 + sqrt(0.99)×epsilon = 0.1×x_0 + 0.995×epsilon. The original signal is attenuated to just 10% of its magnitude while noise dominates at 99.5%. This makes x_1000 nearly indistinguishable from pure N(0,1) noise. The entire generative process works by training a neural network to predict epsilon (the noise) at each step, then iteratively removing it: x_{t-1} = f(x_t, t). Starting from pure noise and denoising through 1000 steps produces a clean sample from the data distribution.