Grade 12 AI & Computer Science Practice Questions — Set 11

ExplanationCareful reading of an ablation table means checking whether a reported gap between two means is actually distinguishable from noise, not just comparing the averages themselves. Here the baseline's one-standard-deviation range is 75.8% to 76.6%, and the SparseAttn range is 76.4% to 77.4% -- these ranges overlap between 76.4% and 76.6%. That overlap shows the 0.7-point difference in average accuracy is consistent with the two configurations performing identically and the gap arising purely from which of the 5 random seeds happened to be drawn; a proper paired significance test (comparing the two configurations seed-by-seed) is needed before the abstract's claim of "effectiveness" can be accepted. Insisting that accuracy can never be an appropriate metric ignores that accuracy is a standard, reasonable choice for a balanced classification benchmark, and nothing in the setup indicates a class-imbalance problem that would require switching to F1-score. Judging the module purely by how far 76.9% sits from 100% confuses absolute performance level with the separate question of whether a specific incremental improvement is statistically real -- a model can be highly effective without being anywhere near ceiling performance. And treating multi-seed averaging as a flaw has the methodology backwards: averaging over several runs reduces the influence of noise, while reporting only the single best-performing run would cherry-pick the luckiest seed and systematically overstate the module's true benefit.

ExplanationThe self-critique-and-revision loop works by having the model itself identify problems in its own output against a written set of principles (the constitution), then produce an improved version. This (original, revised) pair is naturally interpretable as a PREFERENCE comparison — the revised version is preferred over the original — generating exactly the kind of pairwise preference data that would normally require a human labeler to produce by comparing two model outputs directly. Because this entire process runs via the model critiquing itself against explicit written principles, NO human ever needs to read and label graphic or harmful content example-by-example for the harmlessness training signal specifically — this is the concrete bottleneck RLAIF relieves, since human review of disturbing content at scale is slow, expensive, and has documented psychological costs for the human labelers involved (helpfulness training, by contrast, in the original Constitutional AI paper, still uses human preference data, since RLAIF was specifically targeted at the harmlessness half of training).

ExplanationThe core FedAvg algorithm aggregates client updates by weighting each client's contribution PROPORTIONALLY to how much local data it trained on, precisely because an update computed from more data is a more reliable, lower-variance estimate than one computed from less data. Computing the correctly weighted average: numerator = (100x0.60)+(50x0.30)+(150x0.80) = 60+15+120 = 195; denominator = 100+50+150 = 300; result = 195/300 = 0.65. A naive UNWEIGHTED average — (0.60+0.30+0.80)/3 ≈ 0.5667 — treats Client 2's comparatively noisy, small-sample update (n=50) as EQUALLY important as Client 3's more reliable, larger-sample update (n=150), which systematically distorts the aggregate away from what the actual combined data would support. This exact weighting scheme is why FedAvg is written the way it is in the original McMahan et al. 2017 paper — get the weighting wrong, and the aggregated global model no longer faithfully represents the union of all clients' data, degrading convergence and final model quality.

ExplanationVariational quantum circuits are trained via a HYBRID quantum-classical loop, not a purely quantum process. For a given set of gate-angle parameters, the circuit is executed (on real quantum hardware or a classical simulator of one), and measuring the resulting quantum state produces a classical output (like a bit string or expectation value) that gets compared against the desired output via a loss function, exactly as in classical ML. The key technical challenge is computing GRADIENTS of this loss with respect to the quantum gate angles, since you can't directly 'backpropagate' through a quantum measurement the way you would through a classical differentiable function — the PARAMETER-SHIFT RULE is a widely-used technique that estimates these gradients by running the SAME circuit twice more, with one gate-angle parameter shifted slightly forward and backward, and computing a finite-difference-like combination of the resulting measurement outcomes. A classical optimizer (like Adam or plain gradient descent) then uses these estimated gradients to update the gate angles, exactly like updating classical neural network weights — this alternating quantum-execution/classical-update loop is what makes VQCs trainable despite operating on fundamentally different (quantum) parameters than a standard neural network.

ExplanationThis example crisply illustrates why causal inference requires more than conditional probability. Atmospheric pressure is a COMMON CAUSE of both the barometer's reading and the actual weather (a storm) — low pressure causes both. OBSERVING a low barometer reading is informative BECAUSE it's evidence of low pressure (which independently raises storm probability) — so P(storm | barometer reads low) is genuinely elevated, reflecting real correlational information flowing backward from the reading to its cause (pressure) and then forward to the other effect (storm). But INTERVENING — physically forcing the barometer's needle to point low, say by jamming it with your finger — does nothing whatsoever to actual atmospheric pressure, which is the only thing that causally affects storms; the do-operator explicitly SEVERS the barometer's normal causal link to its usual cause (pressure) and sets it directly, so this manipulation carries no information about pressure at all, and P(storm | do(barometer forced low)) remains at the storm's base rate, unaffected. This distinction — that observational conditioning can pick up spurious backward-flowing correlation through common causes, while true intervention cannot — is precisely what Pearl's do-calculus formalizes, and it's the mathematical foundation for why 'correlation is not causation' has a rigorous, computable meaning.

ExplanationCorrectly identifying a variable's ROLE in the causal structure is essential before deciding whether to adjust for it — the same statistical action (controlling for a variable) has completely different, sometimes opposite, effects depending on that role. A CONFOUNDER (Z causing both X and Y) creates a spurious 'backdoor path' between X and Y that has nothing to do with X's actual causal effect — controlling for Z blocks this backdoor path, correctly isolating X's true causal effect on Y. A MEDIATOR (Z sitting ON the causal path X->Z->Y) is different in kind: controlling for it BLOCKS part of the actual mechanism through which X affects Y, causing you to underestimate (or entirely miss) X's TOTAL effect — this is the 'over-controlling' or 'bad control' mistake, appropriate only if you specifically want the DIRECT effect of X excluding the Z-mediated pathway, not the total effect. A COLLIDER (Z caused by both X and Y, X->Z<-Y) is the most counterintuitive: X and Y might be entirely independent on their own, but conditioning on (controlling for, or even just selecting your sample based on) their common effect Z can INDUCE a spurious statistical association between X and Y that has no causal basis at all — a well-documented phenomenon called collider bias, which is also the mechanism behind certain selection-bias problems in observational studies.

ExplanationA Transformer's autoregressive generation requires attending back to every previously-generated token at each new step — the KV-cache (storing every past token's key and value vectors) grows by one entry per generated token, so generating the n-th token requires attending over n cached entries, an O(n) cost for that single step; summing this across generating a full sequence of length n gives a total O(n^2) generation cost, and — separately — the memory needed to STORE the growing KV-cache also scales linearly with sequence length, becoming a real practical bottleneck for very long contexts. An SSM's recurrence, by contrast, is designed so that ALL relevant history is compressed into a FIXED-SIZE hidden state (unlike the ever-growing KV-cache) — generating each new token only requires reading and updating this constant-size state, an O(1) cost per step regardless of how many tokens have already been generated, giving O(n) TOTAL cost across a full generation (and constant, not growing, memory usage). This asymptotic advantage is the central practical selling point of SSM architectures like Mamba for very long-context generation, though it comes with its own tradeoffs — most notably, historical difficulty matching Transformers' strength on tasks requiring precise retrieval of specific far-back information (in-context 'needle in a haystack'-style tasks), since compressing all history into a fixed-size state necessarily discards some information a growing KV-cache would have preserved exactly.

ExplanationThe defining functional purpose of cross-attention is to let one sequence (the decoder, generating output) look INTO and retrieve relevant information FROM a different, separately-encoded sequence (the encoder's representation of the source input) — this is exactly what makes translation, summarization, and similar sequence-to-sequence tasks possible in a Transformer. Q represents 'what is the decoder currently looking for, given what it's generated so far and what token it's about to produce next' — this must come from the decoder's own evolving hidden state, since it's inherently about the DECODER's current needs. K and V represent 'what information is actually available to retrieve from', which must come from the ENCODER's output — the fixed representation of the source sequence, computed once and reused at every decoding step. If Q, K, and V were all drawn from the same sequence (as in ordinary self-attention), there would be no mechanism at all for the decoder to access the encoder's representation of the source text — it would just be reasoning about its own partial output in isolation, with zero connection to what it's actually supposed to be translating or summarizing. The cross-sequence Q-vs-K/V split is precisely the architectural feature that bridges the two halves of an encoder-decoder Transformer.

ExplanationFlashAttention's exactness comes from a specific mathematical property of its online-softmax algorithm, not from any approximation or truncation. Standard softmax computation subtracts the maximum score before exponentiating (for numerical stability) and normalizes by the sum of all exponentials — both of these require knowing ALL scores in advance, which is why standard attention materializes the full score matrix first. FlashAttention instead processes keys/values in small blocks, maintaining a RUNNING maximum and running sum as it goes; critically, whenever a new block reveals a score larger than any seen before, ALL previously-accumulated partial results (both the running sum and the running weighted-value-sum) get RESCALED by an exact correction factor, exp(old_max - new_max), which precisely re-bases everything computed so far to be consistent with the new, larger maximum. This rescaling is not an approximation — it's an algebraically EXACT correction, derivable directly from the mathematical definition of softmax, that guarantees the final accumulated result, after all blocks have been processed, is bit-for-bit (up to ordinary floating-point rounding) identical to what standard batch softmax attention would have computed on the same inputs, despite never holding the full N x N matrix in memory simultaneously. This is precisely why the FlashAttention paper's title specifically emphasizes 'Exact Attention', distinguishing it from genuinely approximate methods like sparse or low-rank attention variants.

ExplanationDPO's key theoretical insight is a clever algebraic substitution. RLHF's objective (maximize expected reward, subject to a KL-divergence penalty keeping the policy close to a reference model) has a known closed-form solution: the OPTIMAL policy under this objective can be expressed directly as a function of the reward and the reference policy. DPO's derivation runs this relationship in REVERSE — instead of first learning a reward function and then finding the policy that optimizes it, DPO algebraically solves for what the REWARD must be, GIVEN a policy, using that same closed-form relationship. Substituting this reward-in-terms-of-policy expression directly into the Bradley-Terry preference model (which normally computes preference probabilities from reward differences) produces a preference probability expressed entirely in terms of the POLICY'S own output probabilities — with the explicit reward function having algebraically cancelled out of the equation entirely. The practical result is a simple, stable, classification-style loss (essentially: increase the policy's relative probability of the preferred response over the dispreferred one, weighted appropriately) that can be optimized directly via standard supervised-learning-style gradient descent on preference-pair data, completely bypassing the need to train a separate reward model or run the more complex, less stable PPO reinforcement learning loop that standard RLHF requires.

ExplanationGradient descent's core requirement is a differentiable objective function, from which useful gradients can be computed via backpropagation — this is exactly what makes it so efficient for standard neural network training, where the loss is a smooth, differentiable function of the weights. But many real-world objectives genuinely lack this property: a reward signal from a physical robot's real-world performance, an objective that depends on a discrete decision or a non-differentiable simulator step, or a black-box system where you can only query 'given these parameters, what score do I get' without any analytical formula to differentiate — in these cases, there simply IS no usable gradient for gradient descent to follow, regardless of how cleverly it's implemented. CMA-ES sidesteps this entirely: it only requires the ability to EVALUATE the objective for any given parameter setting (a black-box function call), then uses a population of sampled parameter vectors and their relative fitness/performance to iteratively update a multivariate search distribution (adapting both its mean and covariance structure) toward more promising regions of parameter space — no gradient computation required anywhere in the process. This makes gradient-free methods like CMA-ES a genuine, sometimes necessary alternative specifically in settings where gradient-based methods cannot be applied at all, not merely a slower substitute for settings where gradients ARE available (where gradient descent typically remains far more sample-efficient).

ExplanationThe defining and most surprising property of in-context learning is that a model's demonstrated 'adaptation' to a new task — inferring a pattern from a handful of prompt examples and then applying it correctly to a new query — happens purely through the model's FORWARD PASS, with the model's weights remaining completely FROZEN and unchanged throughout. The few-shot examples are simply additional tokens the model reads and attends to (via its already-trained, fixed attention mechanism) as part of computing its response to the final query — there is no gradient computation, no backpropagation, and no weight update happening anywhere in this process, despite the model behaving AS IF it had 'learned' the demonstrated task pattern. This has a crucial practical consequence: this apparent learning is entirely EPHEMERAL, tied to that specific context window — the moment you start a new conversation or change the prompt's examples, any 'adaptation' vanishes completely, since nothing was ever actually stored in the model's parameters. Traditional few-shot FINE-TUNING is fundamentally different: it performs genuine gradient descent on the few examples, permanently updating the model's actual weights, so the adaptation persists across every future use of that fine-tuned model, indefinitely, regardless of what's in any future prompt. Exactly HOW a frozen forward pass manages to exhibit apparent learning behavior at all remains an active research question, with competing hypotheses including the formation of implicit 'task vectors' in the model's internal activations and specialized 'induction head' attention circuits.

ExplanationMAML's central goal is finding a set of initial parameters that are GENUINELY GOOD STARTING POINTS for rapid adaptation, not a set of parameters good at any single task directly. The INNER loop simulates this adaptation process explicitly: for each task sampled during meta-training, starting from the current shared initial parameters theta, the model takes one or a few gradient steps (using that task's own small amount of data) to produce task-ADAPTED parameters theta'. The OUTER loop then evaluates how well those ADAPTED parameters theta' perform (on held-out data from that same task), and uses THIS performance signal to update the ORIGINAL shared initial parameters theta — the objective being optimized in the outer loop is literally 'how good is theta AS A STARTING POINT for inner-loop adaptation', not 'how good is theta directly at any task'. The mathematical subtlety, and the source of MAML's computational cost, is that theta' (the adapted parameters used in the outer-loop loss) was itself COMPUTED via a gradient step that is a function of theta — so computing d(outer_loss)/d(theta) requires differentiating through that entire inner gradient-step computation, producing a genuine second-order derivative (a gradient with respect to theta of an expression that already contains a gradient with respect to theta). This is meaningfully more expensive than ordinary single-level gradient descent, which is precisely why 'first-order MAML' (a common approximation that simply ignores this second-order term, treating theta' as if it were independent of theta for gradient purposes) is widely used in practice as a cheaper, empirically-similar-performing alternative.

ExplanationCLIP's contrastive objective reframes the batch of (image, text) pairs as an implicit classification problem: for each image, treat its genuinely matching text as the correct 'label' among ALL texts in the batch (including the mismatched ones from other pairs), and apply a standard cross-entropy loss — this pushes the softmax probability mass on the diagonal entry (the true matching pair) toward 1, and correspondingly toward 0 for every off-diagonal (mismatched) entry in that row; the identical process runs in the other direction too (each text against all images, column-wise). A worked numeric example makes this concrete: given a 3x3 similarity matrix with strong diagonal entries (0.9, 0.8, 0.85) and weaker off-diagonal entries, scaling by a temperature of 0.1 before softmax produces sharply peaked row-wise probabilities — roughly 0.999, 0.991, and 0.995 on the three diagonal entries respectively — demonstrating how a WELL-SEPARATED similarity matrix, combined with a small temperature, drives the softmax toward the desired near-1 diagonal probabilities. Temperature matters specifically because raw cosine similarities are bounded in a narrow range (-1 to 1) and real embeddings for related-but-distinct pairs might only differ by a small margin (like 0.9 vs 0.8) — without dividing by a sufficiently small temperature before the exponential in softmax, these small raw differences would produce an insufficiently sharp, insufficiently informative probability distribution to drive strong learning signal; CLIP treats this temperature as a learnable parameter specifically so the model can find the right sharpness for its own embedding scale during training.

ExplanationThe fundamental computational paradigm shift in SNNs is from DENSE, SYNCHRONOUS computation to SPARSE, EVENT-DRIVEN computation. A conventional artificial neuron (say, with a ReLU activation) computes and outputs some real-valued number on every single forward pass, for every neuron in the network — this is dense computation, where 'work' is done everywhere, on every cycle, regardless of whether that neuron's output carries much useful signal at that moment. A LIF neuron behaves fundamentally differently: it accumulates ('integrates') incoming input over multiple timesteps, with its membrane potential naturally decaying ('leaking') if input is weak or absent, and it only actually COMMUNICATES anything (fires a discrete spike) at the specific moments its accumulated potential crosses a threshold — at every OTHER timestep, that neuron sends nothing at all, an event-driven silence rather than a dense zero-valued output. Specialized neuromorphic hardware (like Intel's Loihi chips) is architecturally designed to exploit exactly this property: rather than computing every neuron's output on every clock cycle (as conventional GPU/CPU hardware effectively does for dense networks), it can genuinely skip computation and communication entirely for neurons that aren't currently spiking, since there is quite literally nothing to compute or transmit for a silent neuron at that instant — for sufficiently sparse spiking activity, this can yield dramatic energy savings compared to the always-on, dense-computation model of conventional deep learning hardware, which is the primary motivation for continued SNN and neuromorphic-hardware research despite SNNs historically lagging behind conventional networks on standard accuracy benchmarks.

ExplanationDiffusion models split cleanly into two processes with very different roles. The FORWARD process — gradually corrupting a real image into pure noise over many steps — is deliberately designed to be a completely FIXED, hand-specified mathematical procedure: at each timestep, a small, precisely pre-determined amount of Gaussian noise (following a chosen 'noise schedule', like a linear or cosine schedule of variances) gets added to the current (partially noised) image. Because this entire procedure is defined by a simple, known formula with no learned components whatsoever, it requires NO neural network and NO training — you could compute the forward process's output at any timestep directly from the original image and the noise schedule's formula, in closed form, without ever training anything. All of the model's actual LEARNING effort is instead concentrated entirely on the REVERSE process: a neural network (typically a U-Net-style architecture) is trained to look at a noisy image at some timestep and predict either the noise that was added (or equivalently, a slightly-less-noisy version of the image) — learning to gradually reverse, one small step at a time, the exact corruption process the fixed forward process defines. This asymmetry — a simple, fixed, mathematically-defined forward process paired with a complex, learned reverse process — is the defining structural idea behind how diffusion models are trained and why they're able to generate novel, realistic images starting purely from random noise at generation time.

ExplanationThe message-passing framework underlying most GNNs works layer by layer: at each layer, every node aggregates information (via some combination function, like a sum, mean, or attention-weighted average) from its DIRECT (1-hop) neighbors' CURRENT representations, then updates its own representation using that aggregated signal. Critically, after just one layer, a node has only incorporated information from its immediate neighbors — but at the SECOND layer, when it aggregates from its neighbors again, those neighbors' representations (computed in layer 1) already reflect information from THEIR OWN neighbors, meaning the 2-hop neighborhood's information reaches the original node indirectly, through this two-step relay. Generalizing this pattern: after k stacked layers, a node's final representation has effectively been influenced by every node within k hops of it in the graph, since information propagates outward by exactly one additional hop per additional layer, analogous to how a k-layer CNN's receptive field grows with depth. However, a well-documented practical failure mode called OVER-SMOOTHING emerges when TOO MANY layers are stacked: as the effective receptive field grows to cover most or all of the graph, and as repeated averaging operations compound across many layers, individual nodes' representations tend to converge toward increasingly similar values — eventually becoming nearly indistinguishable from one another regardless of the nodes' actual distinct local structure or features, which destroys the very representational distinctiveness that made the embeddings useful for downstream tasks in the first place. This over-smoothing problem is a major reason why very deep GNNs (in the way very deep CNNs or Transformers are common) are comparatively rare, and why techniques like skip connections, normalization, or limiting depth are commonly used to mitigate it.

ExplanationNeRF's neural network is fundamentally a CONTINUOUS FUNCTION representation of a 3D scene — given any (x,y,z) coordinate and viewing direction, it returns a color and a density (roughly, 'how opaque is empty space at this point') at that single point, but it does NOT directly output a rendered 2D image on its own; converting this implicit representation into an actual viewable image requires a separate RENDERING step. This rendering works via classical VOLUMETRIC RENDERING, adapted to use the neural network as its color/density source: for each pixel in the desired output image, a ray is cast from the camera's position, through that pixel, out into the 3D scene; along this ray, MANY points are sampled at different depths, and the neural network is queried at EACH sampled point to get that point's color and density. These per-point values are then combined using the classical volume rendering integral — essentially, integrating color contributions along the ray, where each point's contribution is weighted both by its own density (denser points contribute more color) AND by the cumulative TRANSMITTANCE up to that point (how much light has already been absorbed/blocked by denser material earlier along the ray, so points 'hidden behind' opaque material contribute little to the final visible color) — producing a single final color for that pixel. This entire ray-marching-and-integration process is repeated INDEPENDENTLY for every pixel in the output image, which is precisely why naive NeRF rendering is computationally expensive (many network queries per ray, many rays per image) and why faster variants (like Instant-NGP's hash-grid encoding) were developed specifically to accelerate this repeated querying process.

ExplanationRing-allreduce's bandwidth efficiency comes from a specific communication PATTERN: it arranges all N GPUs in a logical ring, and the full gradient tensor is split into N roughly-equal-sized CHUNKS; over N-1 communication rounds, each GPU sends one chunk to its ring-neighbor while simultaneously receiving a different chunk from its OTHER ring-neighbor, progressively both accumulating partial sums (the 'reduce-scatter' phase) and then propagating the final fully-reduced chunks back around the ring (the 'all-gather' phase). The total data moved per GPU across this entire process works out to 2x(N-1)/N times the gradient's size — for N=4 and an 800MB gradient, that's 2x(3/4)x800=1200MB. This is meaningfully less than the naive parameter-server approach's roughly 2x800=1600MB per non-server worker, because the naive approach forces EVERY worker to transmit its COMPLETE gradient tensor to a single central server (and receive the complete averaged result back), creating both a higher per-worker cost AND a severe bottleneck at the server itself (which alone must handle traffic from and to all N-1 workers simultaneously, an (N-1)x higher load than any individual worker). Ring-allreduce instead distributes both the computational reduction work AND the communication load evenly across every participating GPU, with no GPU ever needing to hold or transmit more than its fair share of the total traffic at any single step — this decentralized, load-balanced design is precisely why ring-allreduce (and its variants) became the standard communication primitive for large-scale distributed deep learning training, particularly as GPU counts scale into the hundreds or thousands, where a naive centralized approach's server-bottleneck would become catastrophically limiting.

ExplanationClassical statistical learning theory predicts a single U-shaped curve: test error decreases as model capacity grows from very simple to a 'just right' amount, then increases as the model becomes complex enough to overfit the training data's noise. Double descent reveals a more nuanced empirical picture with an additional second decrease. The key insight involves what happens right AT the interpolation threshold — the specific model size where capacity is JUST barely enough to fit the training data perfectly (zero training error). At this exact point, there is typically only a narrow, specific way to achieve that perfect fit, and this particular solution tends to be forced into an awkward, high-variance shape (having to thread the needle exactly through every training point, including noisy ones, with no slack) that generalizes poorly — this is the classical overfitting peak. But once the model becomes even LARGER, moving well past this threshold, there are now MANY different parameter settings that all achieve zero training error (the training data no longer uniquely determines the fit) — and here, the specific OPTIMIZATION PROCEDURE used (like stochastic gradient descent) exhibits an implicit bias/regularization effect, tending to find comparatively simple, smooth solutions among this now-large space of valid options, rather than the erratic, high-variance solutions forced by having exactly-enough capacity. This implicit-regularization-driven preference for smoother solutions, once there's enough 'slack' in the parameter space to express that preference, is the leading explanation for why heavily overparameterized modern deep networks (with far more parameters than training examples) often generalize surprisingly well, defying the classical intuition that 'more parameters than data points' should be a recipe for catastrophic overfitting.

ExplanationRoPE's core mathematical trick is encoding position via ROTATION rather than addition: each query and key vector gets rotated by an angle that depends on its absolute position in the sequence (using a clever multi-frequency rotation scheme across different vector dimensions), applied BEFORE the dot-product attention score is computed. A specific, elegant mathematical consequence of using rotation specifically (rather than, say, simply adding a position-dependent vector) is that when you compute the dot product between a rotated query at position i and a rotated key at position j, the result depends ONLY on the RELATIVE distance (i-j) between them, not on their absolute positions individually — token 5 attending to token 10 (distance 5) produces a mathematically analogous relationship to token 1000 attending to token 1005 (also distance 5), even though the absolute positions are wildly different. A simple LEARNED positional embedding table, by contrast, assigns each absolute position index (0, 1, 2, ... up to some maximum trained length) its own independently-learned vector, with no built-in mathematical relationship connecting nearby versus distant positions at all — the model must learn any such relative-distance patterns purely from data, and critically, if the maximum sequence length seen during training was, say, 2048 tokens, the embedding table simply has NO learned vector at all for position 5000, making it fundamentally unable to handle longer sequences at inference time without additional tricks. RoPE's relative-distance property, baked in mathematically rather than learned from limited training data, is a major reason it has become the dominant choice for modern LLMs that need to handle long and variable-length contexts, including contexts longer than anything explicitly seen during training.