Grade 12 AI & Computer Science Practice Questions — Set 9

ExplanationFollow the variance-propagation argument He et al. used to derive Kaiming initialization. For layer l, the pre-activation is y_l = W_l x_l, where x_l = ReLU(y_{l-1}). With weights i.i.d., zero-mean, variance Var[W], and independent of the inputs: Var[y_l] = n_in · Var[W] · E[x_l²] Note E[x_l²] is used, not Var[x_l], because x_l = ReLU(y_{l-1}) is not zero-mean. If y_{l-1} is symmetric around zero, ReLU sets the negative half to exactly 0 and passes the positive half through unchanged, so: E[x_l²] = E[ReLU(y_{l-1})²] = (1/2)·E[y_{l-1}²] = (1/2)·Var[y_{l-1}] Substituting back: Var[y_l] = n_in · Var[W] · (1/2) · Var[y_{l-1}]. To keep the variance stable across layers (Var[y_l] = Var[y_{l-1}]), the coefficient must equal 1: n_in · Var[W] · (1/2) = 1 ⟹ Var[W] = 2/n_in With n_in = 512: Var[W] = 2/512 = 1/256 = 0.00390625, so σ = √(1/256) = 1/16 = 0.0625. This is exactly the He/Kaiming rule, and the factor of 2 (versus Xavier's factor of 1, or equivalently 2/(n_in+n_out) when n_in = n_out) exists purely because ReLU deletes half the signal's second moment — Xavier's derivation assumes a roughly linear, symmetric activation (like tanh near zero) with no such deletion, so it under-scales the weights for ReLU networks and biases the network toward vanishing activations in deep stacks. For comparison, Xavier here gives σ = 1/√512 ≈ 0.0442, and indeed 0.0625 / 0.0442 ≈ √2, matching the well-known result that Kaiming's std is √2 times Xavier's. The remaining choices trace common errors: one drops the square root and reports the variance (0.0039) as if it were the standard deviation; one assumes ReLU's clipping has no net effect on variance and reuses Xavier's value unchanged; and one double-applies the correction factor (treating forward- and backward-pass preservation as requiring separate, multiplicative factor-of-2 corrections) even though He initialization only ever uses a single factor of 2 tied to one fan mode (fan_in or fan_out), not both at once.

ExplanationWith P = 8 stages, P − 1 = 7 is the fixed "fill-and-drain" bubble every stage sits through while the first microbatch propagates through the pipeline and the last one drains back out. That 7-unit idle cost does not shrink as M grows, so its impact must be measured against the *entire* flush length (M + P − 1), not against M alone. Set up the inequality using the given formula: 7/(M + 7) ≤ 0.10 Multiply both sides by (M + 7), which is positive: 7 ≤ 0.10(M + 7) = 0.1M + 0.7 Subtract 0.7 from both sides: 6.3 ≤ 0.1M Divide by 0.1: M ≥ 63 Check the boundary directly, since M must be a whole number. At M = 63: bubble fraction = 7/(63 + 7) = 7/70 = 0.10 exactly — this meets "at or below 10%." At M = 62: bubble fraction = 7/(62 + 7) = 7/69 ≈ 0.1014, i.e., 10.14% — this exceeds 10%. So 63 is the smallest microbatch count that satisfies the target, confirming M ≥ 63 is tight. The distractors trace two real misconceptions and a rounding slip. 70 comes from using the loose asymptotic form (P − 1)/M ≤ 0.10 (giving 7/M ≤ 0.10, M ≥ 70) instead of the exact formula — this ignores that the flush's wall-clock length is (M + P − 1), not M, so it overstates how many microbatches are needed. 73 comes from mistakenly putting P (= 8) in the numerator instead of P − 1 (= 7), treating every stage as fully idle during fill/drain rather than recognizing that only P − 1 stage-slots are ever empty at once. 62 is the boundary value with the inequality direction flipped — rounding 62.3 down instead of up, which leaves the bubble fraction just above the 10% target. This is exactly why production systems like GPipe and PipeDream push toward many small microbatches: the fixed (P − 1) bubble cost gets amortized over more useful work as M increases, driving the idle fraction toward zero even though the absolute bubble time per flush stays constant at (P − 1)·t = 35 ms.

ExplanationLoss scaling multiplies the scalar loss L by S before calling backward(); by the chain rule, d(S·L)/dw = S · dL/dw, so every gradient in the graph gets scaled by the same factor S. Here the gradient's true magnitude is 2^-24, and the target is FP16's smallest normal value, 2^-14. Solving S · 2^-24 = 2^-14 gives S = 2^-14 / 2^-24 = 2^10 = 1024. Scaling the loss by 1024 moves this gradient from the extreme edge of the subnormal range — where its mantissa field held only a single nonzero bit (0000000001), giving essentially zero relative precision — into the normal range, where the implicit leading 1 plus the full 10-bit mantissa restore roughly three decimal digits of relative precision. After the backward pass, the FP32 optimizer divides the accumulated gradients by 1024 again before the weight update, so the final update matches what full FP32 precision would have produced; the scale factor only widens precision during the FP16 computation itself. A factor of 16384 (2^14) overshoots to 2^-10 — a valid normal number, but not the smallest one requested. A factor of 2^24 pushes the gradient all the way to 1.0, which is unnecessarily aggressive: since FP16's largest representable magnitude is only about 65504, such a large uniform scale risks pushing other, bigger gradients in the same batch into overflow (inf). A fixed scale of 128 — a common early default before dynamic loss scaling was standard — leaves this particular gradient at 128 × 2^-24 = 2^-17, still well inside the subnormal range and still short on precision, showing why a single hardcoded constant cannot safely cover every gradient's dynamic range.

ExplanationThe clean way to compute E[X] for this stopping process is the identity E[X] = Σ_{m=0}^{∞} P(X ≥ m+1), valid for any non-negative integer-valued random variable. Here X ≥ m+1 happens exactly when the first m proposed draft tokens are all accepted — nothing about position m+1 matters yet — so P(X ≥ m+1) = α^m for m = 0, 1, 2, 3, 4 (the m = 0 term is just P(X ≥ 1) = 1, since a round always finalizes at least one token; the m = 4 term, α^4, is the branch where all 4 draft tokens survive and the guaranteed bonus token still follows). So E[X] = Σ_{m=0}^{4} 0.8^m = 1 + 0.8 + 0.64 + 0.512 + 0.4096, which is a finite geometric progression with first term a = 1, common ratio r = 0.8, and n = γ + 1 = 5 terms. Applying the standard GP sum formula S_n = a(1 − r^n)/(1 − r): E[X] = (1 − 0.8^5)/(1 − 0.8) = (1 − 0.32768)/0.2 = 0.67232/0.2 = 3.3616 ≈ 3.36 tokens per round. This number is the entire point of speculative decoding: a single expensive target-model forward pass, which costs about the same whether it verifies 1 token or γ+1 tokens in parallel, now yields 3.36 finalized tokens on average instead of 1 — a raw ~3.36x reduction in the number of sequential target-model calls needed per generated token (before subtracting the extra compute spent running the cheap draft model γ times per round). Multiplying γ by α (4 × 0.8 = 3.20) is a tempting shortcut because it lands close to the right answer, but the reasoning is wrong: it treats the 4 draft tokens as if their acceptances are independently averaged, when in fact the process stops dead at the first rejection, and it silently ignores the corrective/bonus token that fires on literally every round without exception — two errors that happen to nearly cancel numerically, which is exactly what makes this distractor dangerous. Assuming γ + 1 = 5.00 implicitly sets α = 1, erasing the very reason a target model needs to verify anything — if the draft model were always correct there would be no acceptance/rejection step at all, and speculative decoding would just be running the small model alone. Building the geometric series correctly but stopping at (1 − 0.8^4)/(1 − 0.8) ≈ 2.95 counts only the tokens accepted before the first rejection and forgets that the branch where all 4 tokens are accepted (probability 0.8^4 = 0.4096) still contributes one further bonus token, which is exactly the missing m = 4 term in the sum.

ExplanationStart with the dot product and magnitudes needed for cosine similarity. For Q=(3,4) and A=(6,8): Q·A = (3)(6)+(4)(8) = 18+32 = 50, |Q| = √(9+16) = 5, and |A| = √(36+64) = 10, so cos θ = 50/(5×10) = 1.0 — a perfect match, which makes sense because A = 2Q, so A points in exactly the same direction as Q. For Q and B=(4,3): Q·B = (3)(4)+(4)(3) = 24, |B| = √(16+9) = 5, giving cos θ = 24/25 = 0.96. Since 1.0 > 0.96, the cosine-similarity index returns Document A as the top match. Now compute the Euclidean distances the same query would face under a distance-based index: |Q−A| = √((3−6)²+(4−8)²) = √(9+16) = 5.0, while |Q−B| = √((3−4)²+(4−3)²) = √(1+1) ≈ 1.41. Document B is more than three times closer to Q in raw distance, even though it is less aligned in direction. Switching the ranking metric therefore flips the winner from A to B. This divergence is exactly why production vector databases default to cosine similarity (or equivalently, L2-normalize embeddings before using dot-product/Euclidean search): a smaller angle between two vectors does not guarantee a smaller Euclidean distance between them, because distance also depends on how far each vector's magnitude sits from the other's — Document A's magnitude (10) is twice Q's (5), which inflates its Euclidean distance despite zero angular difference. Cosine similarity strips magnitude out of the comparison entirely, so it is not "biased" toward either larger- or smaller-magnitude embeddings — it measures direction alone, which is what a well-trained embedding model uses to encode semantic meaning.

ExplanationSince every embedding is unit-normalized, cosine similarity collapses to a plain dot product, and each 768-dimensional dot product costs 2 × 768 = 1,536 FLOPs (768 multiplications plus 768 additions — a multiply-add pair per dimension). Scanning all 10,000,000 product embeddings therefore costs 10,000,000 × 1,536 = 1.536 × 10¹⁰ FLOPs total. At a sustained throughput of 4 × 10⁹ FLOPs/second, that scan takes 1.536×10¹⁰ ÷ 4×10⁹ = 3.84 seconds = 3,840 ms. Measured against the 50 ms latency budget, that is 3,840 ÷ 50 = 76.8 times too slow. This gap is exactly why production semantic search never runs brute-force cosine similarity at this scale: approximate nearest-neighbour structures such as HNSW (graph-based greedy search) or IVF-PQ (clustering plus quantization) trade a small, tunable amount of recall for query complexity that grows roughly as O(log N) instead of O(N·d), bringing a 3.84-second exhaustive scan down into the low-millisecond range without touching the underlying embeddings. The wrong answers correspond to real errors students make on this calculation: dropping the multiply-add factor of 2 gives 38.4×; misreading the corpus size as 1 million instead of 10 million gives 7.68×; and forgetting that each comparison itself costs 768 FLOPs (treating throughput as "dot products per second" rather than "FLOPs per second") makes the scan look like it finishes in 2.5 ms when it actually takes over three and a half seconds.

ExplanationXie et al.'s Bayesian view treats each in-context demonstration as evidence about a hidden "concept" the model is trying to infer purely through its forward pass — no weights change, but its effective belief over concepts still updates via Bayes' rule applied inside the network's computation. With a per-example likelihood ratio of λ = 3 in favour of Concept A and 4 conditionally independent demonstrations, the posterior odds compound multiplicatively, not additively: odds(A:B) = prior odds × λ^k = 1 × 3⁴ = 1 × 81 = 81. Converting odds to a probability: P(A) = 81 / (81 + 1) = 81/82 ≈ 0.9878, i.e., about 98.8%. This exponential concentration is exactly why adding a few well-chosen demonstrations can sharply shift an LLM's behaviour even though its parameters never move: each additional consistent example multiplies — rather than adds to — the odds in favour of the correct task concept. Summing the per-shot ratios instead of multiplying them, treating the raw compounded value 81 as if it were already a percentage, or assuming inference-time prompting cannot move the model's effective belief at all, are the three most common errors when reasoning about this mechanism.

ExplanationWrite q·k = Σ_{i=1}^{64} q_i k_i. Each term q_i k_i has mean E[q_i]E[k_i] = 0 by independence of q_i and k_i, and since E[q_i²] = E[k_i²] = 1, Var(q_i k_i) = E[q_i² k_i²] − 0² = E[q_i²]E[k_i²] = 1·1 = 1. The 64 terms are mutually independent (across dimensions), so variances add: Var(q·k) = Σ_{i=1}^{64} Var(q_i k_i) = 64 × 1 = 64, giving standard deviation √64 = 8. A raw score with typical magnitude of several units (versus the ±1 scale of a single term) sits far out on softmax's exponential curve: softmax(x)_j = exp(x_j) / Σ_l exp(x_l) is dominated by whichever logit is largest once logits differ by a few units, so the output collapses toward a one-hot vector and the gradient of softmax with respect to the non-maximal logits shrinks toward zero. This is precisely the vanishing-gradient regime that motivated the √d_k scaling in "Attention Is All You Need." Dividing every score by √d_k = 8 rescales the dot product back to unit variance — the same ±1 scale each individual term already had — so logits stay near zero, where softmax's Jacobian is well-conditioned and gradients propagate normally. The distractors fail for distinct reasons: summing variances does not itself give the standard deviation (variance and standard deviation are conflated when 64 is used directly instead of √64); √d_k scaling is a variance-control mechanism, not an L2-style regularizer, and it has nothing to do with penalizing weight magnitudes; and nothing in the setup normalizes q or k to unit length — assuming i.i.d. unit-variance components across 64 dimensions is in fact incompatible with a unit-norm vector, whose squared length would need to equal exactly 1 rather than have an expected value of 64.

ExplanationTreating h as a generic function of all 8 variables throws away its structure and forces a shallow network to cover an 8-dimensional domain: N_shallow = ε^-8 = (0.1)^-8 = 10^8 units. The compositional network instead mirrors the tree: leaves x1..x8 feed into f1, f2, f3, f4 (2 inputs each), whose outputs feed into f5, f6 (2 inputs each), whose outputs feed into f7 (2 inputs) — a full binary tree with 8 leaves always has exactly 7 internal nodes (each merge of two children reduces the leaf count by one, and 8 leaves need 7 merges to reach a single root). Every node function has only 2 inputs, so each needs only ε^-2 = (0.1)^-2 = 100 units, giving N_deep = 7 × 100 = 700 units. The ratio is 10^8 / 700 ≈ 142,857 ≈ 1.4 × 10^5. This is the core mechanism behind why deep, hierarchically structured networks beat shallow ones on compositional targets: the exponential curse of dimensionality ε^-d is replaced by a sum of small, fixed-size subproblems whose count grows only linearly with the number of variables (here d − 1 = 7 nodes), turning an exponential requirement into a tractable, near-linear one. Indian students can see the same idea in a UPI fraud-detection pipeline, where a transaction's risk score is built by composing small sub-models — a device-check, a velocity-check, a merchant-category-check — instead of training one giant model directly on every raw feature at once.

ExplanationThis is a direct application of Bayes' theorem to a medical AI screening tool, and it shows why even a highly accurate model can have a low positive predictive value (PPV) when disease prevalence is low. Let TB denote "has active tuberculosis" and + denote "AI flags the X-ray as positive." We are given P(TB) = 0.02 (prevalence), sensitivity P(+|TB) = 0.95, and specificity P(-|no TB) = 0.90, so the false-positive rate is P(+|no TB) = 1 - 0.90 = 0.10. By the law of total probability, P(+) = P(+|TB)·P(TB) + P(+|no TB)·P(no TB) = (0.95)(0.02) + (0.10)(0.98) = 0.019 + 0.098 = 0.117. Applying Bayes' theorem, P(TB|+) = P(+|TB)·P(TB) / P(+) = 0.019 / 0.117 ≈ 0.1624, i.e., about 16.2%. So even though the AI tool correctly flags 95% of true TB cases and correctly clears 90% of healthy individuals, only about 16.2% of all positive flags in this 2%-prevalence population actually correspond to real TB, because the healthy majority (98% of the population) generates almost five times as many false positives (9.8 per 100 screened) as the diseased minority generates true positives (1.9 per 100 screened). This is exactly why programmes using AI chest X-ray triage tools (such as qXR under India's National TB Elimination Programme) treat an AI-positive result only as a pre-screening flag, requiring confirmation with a molecular test like CBNAAT/GeneXpert before treatment begins. Sensitivity and specificity describe the test's behaviour conditioned on true disease status, not the reverse; how much to trust a single positive result depends critically on the base rate (prevalence), a distinction known as the base-rate fallacy when it is ignored.

ExplanationBuild the confusion matrix for the "infested" class first: TP = 80 (infested, correctly flagged), FN = 20 (infested, missed), FP = 45 (healthy, wrongly flagged), TN = 855 (healthy, correctly cleared) — these four sum to 1000, matching the field trial. Precision asks: of the leaves the model flagged as infested, how many actually were? Precision = TP/(TP+FP) = 80/(80+45) = 80/125 = 0.64. Recall asks: of the leaves that were actually infested, how many did the model catch? Recall = TP/(TP+FN) = 80/(80+20) = 80/100 = 0.80. The F1-score is defined as the harmonic mean of precision and recall, not their arithmetic mean: F1 = 2·P·R/(P+R) = 2(0.64)(0.80)/(0.64+0.80) = 1.024/1.44 ≈ 0.711, i.e., F1 ≈ 0.71. The harmonic mean is used deliberately because it penalizes imbalance between precision and recall more heavily than a simple average — a model can't hide a weak precision behind a strong recall (or vice versa) the way an arithmetic mean of 0.64 and 0.80 (= 0.72) would let it. That is why averaging P and R directly gives the wrong figure. Accuracy — (TP+TN)/total = (80+855)/1000 = 0.935 — is misleading here precisely because of class imbalance: 900 of the 1000 leaves are healthy, so a model could score ~90% accuracy by predicting "healthy" almost every time while still missing most real infestations. This is exactly why F1, not accuracy, is the standard reporting metric for imbalanced detection tasks like pest identification. Recall alone (0.80) also isn't the F1-score. It's true that in pest management a false negative (an infestation left untreated) is often costlier than a false positive (a healthy leaf sprayed unnecessarily) — but that cost asymmetry is a reason to weight recall more heavily using a metric like F-beta with beta > 1, not a reason to call recall itself the F1-score, which by definition must incorporate precision too.

ExplanationThe score gap is r_A − r_B = 2.3 − 0.7 = 1.6, so P(A ≻ B) = σ(1.6) = 1/(1 + e^(−1.6)). Since e^1.6 ≈ 4.953, e^(−1.6) ≈ 0.202, giving P(A ≻ B) ≈ 1/1.202 ≈ 0.83 — the preference model favors the safe, dispute-portal reply over the PIN-harvesting reply roughly 83% of the time it samples this pair. This matches how Constitutional AI's RL phase actually works: the harmlessness comparisons used to train the preference model are AI-generated — a separate model applies the written constitution via chain-of-thought critique to judge which reply is safer — while the helpfulness comparisons feeding the same preference model still come from human-labeled data collected the standard RLHF way. These two feedback streams are combined into one mixed preference model, which then supplies the reward signal for PPO. So Constitutional AI does not remove human labelers from the loop; it narrows their role by handing specifically the harmlessness judgments to an AI critic bound by explicit written principles, which is what makes the "AI Feedback" in RLAIF distinct from ordinary RLHF.

ExplanationDPO turns pairwise human preferences directly into a classification loss on the policy itself, without ever training a separate reward model. Starting from the KL-regularized RL objective max_π E[r(x,y)] − β·KL(π‖π_ref), the closed-form optimal policy implies an implicit reward r(x,y) = β log(π_θ(y|x)/π_ref(y|x)) + β log Z(x), where Z(x) is a partition function depending only on the prompt x. Since the Bradley-Terry preference model only ever needs the difference r(x,y_w) − r(x,y_l), the Z(x) terms cancel, leaving L = −log σ(β[log(π_θ(y_w|x)/π_ref(y_w|x)) − log(π_θ(y_l|x)/π_ref(y_l|x))]). Plugging in the numbers: the chosen response's log-ratio is log(0.4/0.1) = log 4 ≈ 1.3863, and the rejected response's log-ratio is log(0.05/0.2) = log 0.25 ≈ −1.3863. The margin is β times their difference: 0.5 × (1.3863 − (−1.3863)) = 0.5 × 2.7726 = 1.3863 = log 4. Passing this through the logistic function gives σ(log 4) = 4/(1+4) = 0.8, so the loss is −log(0.8) ≈ 0.223 nats. Notice that although π_θ assigns y_l a fairly low absolute probability (0.05), what actually drives the loss is how much that probability moved relative to π_ref — it dropped by a factor of 4 from π_ref's 0.2, exactly mirroring the chosen response's 4x rise from 0.1 to 0.4. That relative, reference-anchored comparison is the whole point of the KL term, and it's why DPO's gradient cannot be computed from π_θ's probabilities alone. The 0.303-nats figure comes from treating the margin as β times the raw policy log-ratio only, 0.5 × (log 0.4 − log 0.05) = 0.5 × log 8 ≈ 1.040, which silently drops π_ref out of the computation and mistakes DPO for a plain likelihood-ranking loss with no anchor to the reference policy. The 0.061-nats figure uses the correct log-ratio margin, log 16 ≈ 2.773, but forgets to scale it by β, which changes how sharply the sigmoid saturates and understates the loss. The 1.609-nats figure comes from swapping which response is treated as preferred — computing β[(log-ratio of y_l) − (log-ratio of y_w)] = −log 4 instead — which flips the sign of the margin, gives σ(−log 4) = 0.2, and is exactly the error that results from mislabelling which response the annotators actually preferred.

ExplanationPerplexity filtering exploits a structural property of GCG-style adversarial suffixes: because these strings are found by gradient-guided search over token space to maximize the probability of a harmful target completion, they are optimized for attack success, not for fluency — so a well-trained language model assigns them unusually low probability (high negative log-likelihood) per token, unlike natural language. Apply the given definition exactly as stated, PPL(x) = exp((1/N) Σ −ln P(xᵢ)), with N = 6. Sum the six negative log-likelihoods: 1.2 + 0.9 + 5.8 + 6.1 + 5.5 + 6.3 = 25.8 nats. Divide by N to get the mean per-token loss: 25.8 / 6 = 4.3 nats. Exponentiate with base e (since the log-likelihoods were computed in natural-log units, "nats," the inverse of ln is exp, not 2^x): PPL = e^4.3. Using e^4 ≈ 54.60 and e^0.3 ≈ 1.350, PPL ≈ 54.60 × 1.350 ≈ 73.7. Since 73.7 > 50, the filter correctly flags the input as a likely adversarial suffix. The 2^4.3 ≈ 19.7 route mixes up units: base-2 perplexity is only equivalent to this base-e definition if the log-likelihoods were originally measured in bits (log base 2), not nats. Applying 2^x to a natural-log-derived exponent silently shrinks the exponent's effective base and produces a perplexity low enough to slip under the threshold — precisely the failure mode a defender must avoid when wiring a perplexity filter to a model's actual log-probability output format. The e^25.8 route skips the (1/N) normalization entirely. Raw summed negative log-likelihood grows with sequence length regardless of how anomalous each individual token is, so an un-normalized "perplexity" would flag long benign passages just as readily as short adversarial ones — defeating the purpose of a per-token fluency measure. The four-token route silently redefines which tokens count as "the suffix," computing e^(23.7/4) ≈ 374.3 over a different, smaller token set than the N = 6 the problem specifies, and then draws a non-sequitur conclusion: a high perplexity score over the suffix window says nothing about the separately-scored prefix, since perplexity is computed independently over whatever window is chosen, not propagated backward onto unrelated tokens.

ExplanationThe alignment tax is the drop in capability caused by safety fine-tuning: Tax(x) = C(0) − C(x). To find the intensity the deployment policy actually settles on, maximize the combined utility U(x) = S(x) + C(x) = 10x + (100 − x²) = 100 + 10x − x². This is a downward-opening parabola in x, so differentiate and set the derivative to zero: U'(x) = 10 − 2x = 0, giving x* = 5. Since U''(x) = −2 < 0, this is indeed a maximum, and x* = 5 lies inside the allowed range [0, 10], so the boundary is irrelevant here. At this intensity, capability retention is C(5) = 100 − 5² = 100 − 25 = 75, so the alignment tax is Tax(5) = C(0) − C(5) = 100 − 75 = 25 percentage points. The model trades 25 points of baseline task accuracy for the safety level S(5) = 10(5) = 50 that the utility-maximizing policy settles on. Treating the safety score itself as the tax conflates "safety gained" with "capability lost" — they are different axes of the same trade-off. Assuming the policy pushes intensity to the domain's upper bound ignores that U(x) is concave and peaks strictly inside the interval, not at an edge. And assuming the policy maximizes raw capability alone confuses the unconstrained optimum of C(x) with the optimum of the joint objective U(x) that the policy is actually built to maximize — capability-only optimization is exactly the pre-alignment baseline the tax is measured against, not the deployed operating point.

ExplanationDividing similarities by τ before the softmax is exactly what makes CLIP's temperature-scaled contrastive loss (InfoNCE) work: a small τ sharpens the distribution over the batch, turning even a modest similarity gap into a highly confident prediction. Start from the I1 row of S: [0.9, 0.2, 0.1]. Dividing each entry by τ = 0.1 (equivalent to multiplying by 10) gives logits [9, 2, 1]. Applying softmax: exp(9) = 8103.08, exp(2) = 7.389, exp(1) = 2.718, sum = 8113.19 p(T1 | I1) = 8103.08 / 8113.19 = 0.9988 The loss contribution is −ln(0.9988) ≈ 0.0012 nats — a very small loss, because the temperature-sharpened distribution puts almost all its mass on the correct caption T1. The 0.666-nats distractor skips the division by τ entirely and runs softmax on the raw similarities [0.9, 0.2, 0.1]; that ignores where τ actually enters CLIP's formula (inside the softmax, as sim/τ), not as a post-hoc rescaling of the loss. The 1.05-nats distractor inverts the temperature operation: multiplying the similarities by τ = 0.1 instead of dividing by it produces logits [0.09, 0.02, 0.01], which flattens the distribution toward uniform (roughly equal probabilities for all three captions) instead of sharpening it — the opposite of what a small τ is designed to do, and it actually increases the loss above what you'd get with no temperature at all. The 0-nats distractor confuses top-1 accuracy with cross-entropy loss. T1 does have the highest similarity to I1, so the model's ranking is correct, but the loss is only exactly zero if the assigned probability is exactly 1. Since softmax always spreads some probability mass to T2 and T3, the loss stays strictly positive — small, but not zero — which is precisely why gradient signal keeps flowing even once the model is "getting it right."

ExplanationThis is the classical compounding-error (covariate-shift) bound for behavior cloning, due to Ross and Bagnell. Let the first mistake occur at step t. Because ε is small, the probability of matching the expert exactly through steps 1 to t−1 is approximately 1, so the probability that the first mistake happens exactly at step t is approximately ε for every t = 1, …, T. Once that first mistake occurs, the robot enters a state outside its training distribution — one the expert demonstrations never visited — so by the worst-case assumption every one of the remaining T − t + 1 steps (including step t itself) counts as a failure. The expected number of failed steps is therefore E[C] = Σ_{t=1}^{T} ε(T − t + 1) = ε Σ_{k=1}^{T} k = ε · T(T+1)/2. Substituting ε = 0.02 and T = 50: E[C] = 0.02 × (50 × 51)/2 = 0.02 × 1275 = 25.5. So on average about 25.5 of the 50 steps — roughly half the episode — end up as failures, even though the policy is wrong only 2% of the time on the states it saw during training. Compare this with the naive, non-compounding estimate εT = 0.02 × 50 = 1, which just adds up 50 independent 2%-probability failures and predicts about one bad step. The true worst-case expectation is about 25 times larger than that naive linear estimate, because the quadratic T(T+1)/2 term captures something the naive sum ignores: an early mistake doesn't cost one step, it costs every step that follows, since the robot was never shown how to recover once it drifts off the expert's state distribution. This is precisely why pure offline behavior cloning is a fragile way to train a robot foundation-model policy despite low training loss, and why techniques like DAgger (iteratively querying the expert for corrective labels on states the learner itself visits) or recovery-aware architectures (e.g., diffusion-policy action chunking, which resamples a whole action sequence instead of committing to one brittle action) are used in practice — DAgger provably brings the bound back down from O(εT²) to O(εT).

ExplanationThe real actuator obeys exponential first-order dynamics, not the instantaneous response the simulator assumes, so the fraction of commanded torque reached at time t is τ(t)/τ_cmd = 1 − e^(−t/T). Substituting the control period t = 20 ms and the identified hardware time constant T = 50 ms gives t/T = 20/50 = 0.4, so τ(t)/τ_cmd = 1 − e^(−0.4). Since e^(−0.4) ≈ 0.6703, the achieved fraction is 1 − 0.6703 ≈ 0.3297, i.e., about 33%. So every 20 ms the real servo closes only about a third of the gap to its target before a fresh command supersedes it entirely — the actuator is perpetually chasing a moving reference it can never catch, even though the simulator's torque source hit every target exactly and instantly. This is the classic unmodeled-actuator-dynamics source of the reality gap: a policy that looks stable and precise in simulation can become jittery, underpowered, or unstable on hardware purely because the simulator's assumption of instantaneous actuation was never true. The fix is not more training in the flawed simulator but closing the model mismatch itself — via system identification of T followed by adding matching first-order lag to the simulated actuators (or randomizing T across a realistic range during training), or by slowing the control loop to match the hardware's actual bandwidth. The 40% distractor comes from wrongly treating the torque rise as linear (t/T) rather than exponential — first-order lag systems approach their asymptote fastest at first and slower over time, so linear extrapolation overstates early progress incorrectly here by only a little but reflects a fundamentally wrong model. The 67% distractor comes from computing e^(−0.4) correctly but forgetting to subtract it from 1, reporting the fraction still remaining as if it were the fraction achieved. The 92% distractor comes from inverting the exponent's ratio (T/t instead of t/T), which corresponds to asking how far the system would rise if the roles of the control period and the time constant were swapped.

ExplanationThe ACO pheromone-update rule has two components that must be applied in a fixed order, and the order is the entire point of the exercise: evaporation acts on the pheromone trail that already exists, while deposits from the current round's ants are layered on afterward, independently of each other. Step 1 — Evaporation: the existing trail decays by factor (1−ρ) regardless of what happens this round. With ρ = 0.4 and τ(t) = 2.0, this gives (1 − 0.4) × 2.0 = 0.6 × 2.0 = 1.2. Step 2 — Deposits: every ant that used edge A→B this round contributes Q/Lₖ, where a shorter tour length Lₖ produces a larger deposit — this is the mechanism by which ACO reinforces good (short) routes faster than bad ones. Bike 1 (50 km) deposits 100/50 = 2. Bike 2 (25 km) deposits 100/25 = 4. Step 3 — Sum the deposits: because each ant's pheromone contribution is independent and additive, the two deposits stack rather than average: 2 + 4 = 6. Step 4 — Combine: the evaporated base and the summed deposits add together: 1.2 + 6 = 7.2. So τ(t+1) = 7.2 is the only value consistent with the standard update rule. Skipping evaporation entirely gives 8.0, evaporating the combined total instead of just the pre-existing trail gives 4.8, and averaging instead of summing the ants' deposits gives 4.2 — each of these corresponds to a genuine, common implementation mistake, but none matches the rule as stated.

ExplanationBoth expression trees compute exactly y = 2x + 1 at every sampled point (x = 1,2,3,4 give y = 3,5,7,9): tree A directly encodes 2x + 1, while tree B reaches the same function through the redundant identities x + x = 2x and 1 + (x − x) = 1, so both have raw SSE = 0 and are indistinguishable by accuracy alone. Node counts differ sharply: A = (+ (* 2 x) 1) has 5 nodes in total (2 function nodes + and *, plus 3 terminals 2, x, 1); B = (+ (+ x x) (+ 1 (- x x))) has 9 nodes in total (4 function nodes +, +, +, − plus 5 terminals x, x, 1, x, x). Applying the parsimony-pressure formula, adjusted fitness = SSE + 0.1 × nodes, gives A = 0 + 0.1(5) = 0.5 and B = 0 + 0.1(9) = 0.9. Because the tournament here minimizes adjusted fitness, the smaller score wins, so Individual A is selected over the functionally identical but bloated Individual B. This is exactly what parsimony pressure is designed to do in Genetic Programming: once accuracy is tied, it steers selection toward the more compact program and discourages the introns and redundant subtrees that drive code bloat across generations.