Question 201 · TF-IDF and BM25: Weighting Terms · hard
A search engine indexes N = 50 news articles, and the term "monsoon" appears in df = 10 of them. The average article length across the whole corpus is avgdl = 250 words. The engine is comparing two articles for the query "monsoon":
- Article P: 100 words long, with "monsoon" occurring 5 times.
- Article Q: 500 words long, with "monsoon" occurring 20 times.
Each article is scored with Okapi BM25 using k1 = 1.5 and b = 0.75:
score(D, "monsoon") = IDF("monsoon") * [f * (k1 + 1)] / [f + k1 * (1 - b + b * |D| / avgdl)]
Here f is how many times "monsoon" occurs in D, and |D| is D's length in words. IDF("monsoon") is a single fixed number for this comparison, since it depends only on N and df, not on which article is being scored. If you work out the length-normalisation factor and the resulting term-frequency component for each article, which one does BM25 rank higher for this query, and why?
BM25 gives "monsoon" a large IDF weight since it appears in only 10 of 50 articles, and applies that weight more heavily to longer documents - so Article Q's extra length amplifies the IDF penalty and Article P ends up scoring higher.
BM25 preserves the same ranking as a plain term-density comparison here: Article P's 5 occurrences in 100 words (5%) is denser than Article Q's 20 occurrences in 500 words (4%), so Article P scores higher and nothing in BM25 can invert that.
With k1 = 1.5, the BM25 term-frequency component saturates quickly, so Article Q's fourfold raw-count advantage (20 versus 5 occurrences) narrowly outweighs Article P's higher term density, even though Article Q absorbs a length penalty roughly three times steeper than Article P's - Article Q ends up scoring higher.
Once a term occurs in a document, BM25 stops applying any further length normalisation, so Article Q scores higher purely because it contains four times as many occurrences of the query term as Article P does.
Answer: C. With k1 = 1.5, the BM25 term-frequency component saturates quickly, so Article Q's fourfold raw-count advantage (20 versus 5 occurrences) narrowly outweighs Article P's higher term density, even though Article Q absorbs a length penalty roughly three times steeper than Article P's - Article Q ends up scoring higher.
ExplanationBM25's term-frequency component does not behave like a simple normalised-TF ratio: the k1 parameter makes it grow with diminishing returns as raw counts increase, while the b parameter applies an independent, continuously-scaling length penalty rather than switching off once a term has appeared. Working through the numbers shows this clearly. Article P's length-normalisation factor is 1 - 0.75 + 0.75 * (100/250) = 0.55, giving a term-frequency component of (5 * 2.5) / (5 + 1.5 * 0.55) = 12.5 / 5.825 ≈ 2.146. Article Q's length-normalisation factor is 1 - 0.75 + 0.75 * (500/250) = 1.75 - over three times as large as Article P's, confirming the length penalty is very much still active rather than absent - giving a term-frequency component of (20 * 2.5) / (20 + 1.5 * 1.75) = 50 / 22.625 ≈ 2.210. Since "monsoon" is the same term drawn from the same corpus, IDF("monsoon") depends only on N = 50 and df = 10, so it multiplies both articles' scores by an identical constant and cannot be the reason either article outranks the other - this rules out any explanation that ties the outcome to IDF being applied unevenly across documents of different lengths. What actually decides the ranking is that BM25's saturation curve compresses the gap between 5 and 20 occurrences far more than a plain density ratio (5% versus 4%) would suggest, so Article Q's much larger raw term count narrowly outweighs Article P's higher term density despite Q's much steeper length penalty - giving Article Q the higher BM25 score even though Article P would win under a plain normalised-TF-IDF density comparison.
Question 202 · Confounding & Causal Inference · hard
A hospital study finds that patients who take Drug X have a HIGHER recovery rate than those who don't. But patients were not randomly assigned Drug X — doctors preferentially gave it to patients who were already healthier and more likely to recover regardless. What specific statistical concept, when you analyze this study, explains why it CANNOT conclude 'Drug X causes better recovery', and what would fix it?
The sample size is too small — simply collecting more patients this same way would resolve the issue
This is CONFOUNDING: 'baseline health' is a confounder that influences BOTH who receives Drug X AND who recovers, creating a spurious association that has nothing to do with the drug's actual effect; a randomized controlled trial (RANDOMLY assigning Drug X vs not, so baseline health is balanced on average between groups) would isolate the drug's true causal effect
The study is unbiased and the conclusion 'Drug X causes recovery' is fully justified as stated
This is a labeling error — the data was probably just recorded incorrectly
Answer: B. This is CONFOUNDING: 'baseline health' is a confounder that influences BOTH who receives Drug X AND who recovers, creating a spurious association that has nothing to do with the drug's actual effect; a randomized controlled trial (RANDOMLY assigning Drug X vs not, so baseline health is balanced on average between groups) would isolate the drug's true causal effect
ExplanationA confounder is a variable that influences BOTH the treatment (who gets Drug X) and the outcome (who recovers), creating an association between treatment and outcome that isn't due to the treatment's actual causal effect at all. Here, baseline health does exactly that: healthier patients were more likely to be given the drug AND more likely to recover anyway, so the observed correlation could be entirely explained by baseline health alone, with the drug contributing nothing, because both branches of the comparison trace back to the same underlying cause. The gold-standard fix is a randomized controlled trial: by assigning Drug X purely randomly (e.g., a coin flip per patient), baseline health becomes statistically balanced between the treatment and control groups on average, breaking the confounder's connection to WHO gets the drug — any remaining difference in recovery rates can then be attributed to the drug itself, not to who happened to receive it.
Question 203 · Probability Distributions: Poisson vs Binomial · medium
A factory's quality-control system flags a defective circuit board on average once every 15 minutes, with defects occurring independently and at a roughly constant average rate over time. To model 'the number of defects flagged in the next hour', which probability distribution is the natural fit, and why NOT a Binomial distribution?
Binomial, because defects are either present or absent (a binary outcome)
Poisson — the Poisson distribution specifically models the COUNT of independent events occurring at a constant average rate over a fixed interval of time (or space), which is exactly this scenario; Binomial requires a FIXED, KNOWN number of discrete trials (like 'flip a coin 20 times'), but here there is no natural fixed trial count — defects could occur at literally any moment, an underlying continuum Binomial isn't built to model directly
Normal distribution, because all real-world data eventually becomes normally distributed given enough samples
Uniform distribution, because each minute is equally likely to have a defect
Answer: B. Poisson — the Poisson distribution specifically models the COUNT of independent events occurring at a constant average rate over a fixed interval of time (or space), which is exactly this scenario; Binomial requires a FIXED, KNOWN number of discrete trials (like 'flip a coin 20 times'), but here there is no natural fixed trial count — defects could occur at literally any moment, an underlying continuum Binomial isn't built to model directly
ExplanationThe Poisson distribution is the standard model for 'how many independent, rare-ish events happen in a fixed window of time or space, given a known average rate' — exactly this factory scenario, with an average rate of 1 defect per 15 minutes (so 4 per hour on average). Binomial, by contrast, models the count of successes across a FIXED, PRE-DEFINED number of discrete trials, each with the same success probability (like 'exactly 20 coin flips, what's the chance of exactly 12 heads') — there's no natural way to carve continuous time into a fixed number of discrete 'trials' here, since a defect could in principle occur at any instant, not just at pre-defined checkpoints. In fact, Poisson can be derived mathematically as the LIMIT of a Binomial distribution as the number of trials grows to infinity while the per-trial probability shrinks proportionally — which is exactly the intuition for why Poisson is the right tool when events are effectively continuous-time rather than discrete-trial.
Question 204 · Feature Engineering: Categorical Encoding · medium
Given data with features 'city' (values: Mumbai, Delhi, Bengaluru — no inherent order) and a decision to encode it for a linear regression model, what is WRONG with simply using label encoding (Mumbai=0, Delhi=1, Bengaluru=2) instead of one-hot encoding?
Label encoding is always the better choice for categorical features regardless of the model
Label encoding imposes a FALSE numeric ORDERING and MAGNITUDE relationship on a categorical variable that has no natural order — the model would treat 'Bengaluru (2) minus Mumbai (0) = 2' as meaningful, and 'Delhi is between Mumbai and Bengaluru' as meaningful, when neither is true; one-hot encoding (a separate binary 0/1 column per city) avoids inventing this false structure, letting the model learn each city's effect independently
One-hot encoding is mathematically impossible for more than 2 categories
Label encoding and one-hot encoding always produce mathematically identical model results
Answer: B. Label encoding imposes a FALSE numeric ORDERING and MAGNITUDE relationship on a categorical variable that has no natural order — the model would treat 'Bengaluru (2) minus Mumbai (0) = 2' as meaningful, and 'Delhi is between Mumbai and Bengaluru' as meaningful, when neither is true; one-hot encoding (a separate binary 0/1 column per city) avoids inventing this false structure, letting the model learn each city's effect independently
ExplanationLabel encoding assigns an arbitrary integer to each category, but a model like linear regression interprets numeric features as having real magnitude and ORDER — it would implicitly assume Bengaluru (2) is 'twice as much city' as Delhi (1), or that the cities exist on a meaningful numeric scale where distances between them matter, none of which reflects reality for a nominal (unordered) category like a city name. One-hot encoding sidesteps this entirely: it creates one binary column per category (is_Mumbai, is_Delhi, is_Bengaluru), each independently 0 or 1, so the model can learn a separate, unconstrained coefficient for each city's effect without any invented ordering or distance relationship between them. Label encoding IS appropriate for genuinely ORDINAL categories with a real inherent order (like 'small/medium/large'), where the implied ordering is actually meaningful — the choice depends entirely on whether the category has true order, not on convenience.
Question 205 · Data Augmentation: Label-Preserving Transforms · medium
You are augmenting a dataset of handwritten digit images (0-9) for training a classifier. Which of these augmentation choices is LIKELY TO HURT the model rather than help it, and why?
Randomly rotating each digit image by a small angle (e.g., -15 to +15 degrees) — this is generally safe and helpful
Randomly flipping each digit image HORIZONTALLY (mirror image) — this is likely to HURT, because a horizontally-flipped '2' or '3' looks like a fundamentally different (often nonsensical or wrong-class) shape, meaning the augmented image no longer represents its assigned label — the model would be trained on systematically MISLABELED data
Adding a small amount of random noise to each pixel — this is generally safe and helpful
Slightly shifting each image by a few pixels in a random direction — this is generally safe and helpful
Answer: B. Randomly flipping each digit image HORIZONTALLY (mirror image) — this is likely to HURT, because a horizontally-flipped '2' or '3' looks like a fundamentally different (often nonsensical or wrong-class) shape, meaning the augmented image no longer represents its assigned label — the model would be trained on systematically MISLABELED data
ExplanationThe single non-negotiable requirement for any data augmentation technique is that it must be LABEL-PRESERVING — the transformed image must still genuinely represent the same class it's labeled as. Small rotations, minor pixel noise, and small pixel shifts are all safe for digit recognition because a slightly rotated, slightly noisy, or slightly shifted '7' is still unambiguously a '7' to a human. But horizontal flipping is different in kind: a mirrored '2' does not look like a valid, correctly-oriented '2' at all (and a mirrored '3' looks similar to an 'E' or nothing meaningful) — the flip has effectively created a picture that no longer matches its label, actively teaching the model an incorrect association. This exact category of mistake — augmenting in a way that breaks the label — is a common real bug: horizontal flips are perfectly fine for many photo-classification tasks (a mirrored cat is still recognizably a cat) but actively harmful for tasks where orientation carries meaning, like digit or text recognition.
Question 206 · Node Embeddings: Geometric Similarity · medium
In a graph-based recommendation system, two products are represented as vectors (node embeddings) learned so that products frequently co-purchased or co-viewed end up CLOSE together in the embedding space. If Product A's embedding vector is very close (small distance) to Product B's embedding vector, what does that geometric closeness actually represent?
That Product A and Product B are literally the same physical product
That Product A and Product B tend to appear in similar CONTEXTS within the graph (e.g., frequently bought together, viewed by the same users, or connected through similar neighbors) — the embedding was specifically trained so that graph-structural similarity translates into geometric (vector-space) proximity, which is exactly what makes the embedding useful for tasks like 'find similar products'
That Product A costs the same amount as Product B
Nothing meaningful — embedding distances are randomly assigned and carry no interpretable information
Answer: B. That Product A and Product B tend to appear in similar CONTEXTS within the graph (e.g., frequently bought together, viewed by the same users, or connected through similar neighbors) — the embedding was specifically trained so that graph-structural similarity translates into geometric (vector-space) proximity, which is exactly what makes the embedding useful for tasks like 'find similar products'
ExplanationThe entire point of learning node embeddings (via methods like node2vec, DeepWalk, or a GNN) is to capture STRUCTURAL relationships in a graph as GEOMETRIC relationships in a continuous vector space — nodes that are 'close' in the graph's connectivity pattern (frequently co-occurring, sharing many neighbors, or connected through similar paths) are trained to end up with embedding vectors that are close together by some distance measure (like cosine similarity or Euclidean distance). This is what makes embeddings practically useful: instead of needing to traverse the raw graph structure every time you want to find similar items, you can just compute vector distances directly, which is fast and works well with standard machine-learning tools (like nearest-neighbor search or feeding embeddings into a downstream classifier). Two products being close in embedding space is a learned SIGNAL of behavioral or structural similarity — not a claim about their literal identity or price.
Question 207 · Naive Bayes: Bayes' Theorem Application · hard
A spam classifier uses a Naive Bayes approach. Prior: P(spam)=0.4, P(not spam)=0.6. Given an email contains the word 'free', the likelihoods are P('free' | spam)=0.7 and P('free' | not spam)=0.1. Using Bayes' theorem, what is P(spam | 'free'), the probability the email is spam GIVEN it contains 'free'?
0.70, simply equal to P('free' | spam) directly
Approximately 0.8235 — computed as [P(spam) x P('free'|spam)] / [P(spam) x P('free'|spam) + P(not spam) x P('free'|not spam)] = (0.4x0.7) / (0.4x0.7 + 0.6x0.1) = 0.28 / (0.28+0.06) = 0.28/0.34 ≈ 0.8235
0.40, simply equal to the prior P(spam)
1.00, since 'free' is commonly associated with spam so the classification is certain
Answer: B. Approximately 0.8235 — computed as [P(spam) x P('free'|spam)] / [P(spam) x P('free'|spam) + P(not spam) x P('free'|not spam)] = (0.4x0.7) / (0.4x0.7 + 0.6x0.1) = 0.28 / (0.28+0.06) = 0.28/0.34 ≈ 0.8235
ExplanationBayes' theorem combines a PRIOR belief with observed EVIDENCE to produce a POSTERIOR probability: P(spam|free) = P(free|spam)xP(spam) / P(free), where P(free) is the total probability of seeing 'free' across both classes (spam and not-spam), computed as P(free|spam)xP(spam) + P(free|not spam)xP(not spam). Plugging in the numbers: the numerator is 0.7x0.4=0.28 (probability an email is BOTH spam AND contains 'free'). The denominator is 0.28 + (0.1x0.6=0.06) = 0.34 (total probability of 'free' appearing, from either class). Dividing: 0.28/0.34 ≈ 0.8235. This means observing the word 'free' shifted the model's belief from a 40% prior probability of spam up to about 82.35% posterior probability — a substantial update, driven by the fact that 'free' is 7 times more likely to appear in a spam email (0.7) than a legitimate one (0.1).
Question 208 · SVM: Margin Maximization · medium
A Support Vector Machine (SVM) is trained to separate two classes with a linear decision boundary. Among all possible boundaries that correctly separate the training data, what specific criterion does SVM use to choose ONE particular boundary?
SVM picks the boundary closest to the majority of training points, to minimize total distance
SVM picks the boundary that MAXIMIZES THE MARGIN — the distance between the decision boundary and the nearest training points from EACH class (the 'support vectors') — the intuition being that a boundary with more breathing room on both sides is more likely to generalize well to new, unseen points near the boundary
SVM picks the boundary with the steepest possible slope
SVM picks a random valid boundary from the ones that separate the data correctly, since all separating boundaries perform equally well
Answer: B. SVM picks the boundary that MAXIMIZES THE MARGIN — the distance between the decision boundary and the nearest training points from EACH class (the 'support vectors') — the intuition being that a boundary with more breathing room on both sides is more likely to generalize well to new, unseen points near the boundary
ExplanationFor linearly separable data, there are generally INFINITELY many possible boundaries that would correctly separate the two classes on the training set — but not all of them generalize equally well to new data. SVM's specific criterion is to choose the boundary that maximizes the MARGIN: the gap between the boundary and the closest training points from each class (these closest points are called support vectors, since they are the ones that literally 'support', or define, the boundary's position). The intuition is that a boundary hugging one class too closely leaves little room for natural variation in new test points from that class before they'd be misclassified — maximizing the margin builds in the largest possible safety buffer on both sides. This margin-maximization objective is the defining, name-giving idea behind Support Vector Machines, and it's why SVMs tend to generalize well even with relatively few training examples, compared to boundaries chosen by less principled criteria.
Question 209 · Eigenvalues & Eigenvectors · hard
A matrix W = [[3, 0], [0, 3]] is applied to any 2D vector as a linear transformation. What are its eigenvalues, and geometrically, what does this specific matrix do to every vector in the plane?
Eigenvalues are 0 and 3; the matrix rotates every vector by 90 degrees
Both eigenvalues equal 3 (since W is exactly 3 times the identity matrix, W = 3I); geometrically, this matrix SCALES every vector in the plane by a factor of 3 in every direction uniformly, without any rotation or shearing — EVERY nonzero vector is an eigenvector here, all sharing the same eigenvalue 3, because scaling uniformly doesn't privilege any particular direction
Eigenvalues are 1 and -1; the matrix reflects vectors across the x-axis
This matrix has no real eigenvalues
Answer: B. Both eigenvalues equal 3 (since W is exactly 3 times the identity matrix, W = 3I); geometrically, this matrix SCALES every vector in the plane by a factor of 3 in every direction uniformly, without any rotation or shearing — EVERY nonzero vector is an eigenvector here, all sharing the same eigenvalue 3, because scaling uniformly doesn't privilege any particular direction
ExplanationAn eigenvector v of matrix W satisfies Wv = lambda*v for some scalar lambda (the eigenvalue) — meaning applying W to v just SCALES v by lambda, without changing its direction. For W = [[3,0],[0,3]] = 3 times the identity matrix: applying W to ANY vector (x,y) gives (3x, 3y), which is exactly 3 times the original vector, for literally every possible input vector. This means every nonzero vector in the plane is an eigenvector of W, all sharing the identical eigenvalue 3 (since the identity matrix itself has eigenvalue 1 for every vector, and multiplying by 3 scales every eigenvalue by 3 as well: computed formally, the characteristic equation det(W - lambda*I)=0 gives (3-lambda)^2=0, so lambda=3 is a repeated eigenvalue). Geometrically, this matrix performs a uniform scaling (dilation) by factor 3 in every direction — no rotation, no shear, no direction-dependent stretching, which is precisely why every direction qualifies as an eigenvector: uniform scaling doesn't treat any direction specially.
Question 210 · Evaluation Metrics: F1 Score Computation · medium
A binary classifier on a medical test dataset produces TP=42, FP=8, FN=10, TN=140. What is the F1 score (the harmonic mean of precision and recall) for this classifier?
F1 = 0.90, computed as a simple average of TP and TN rates
F1 ≈ 0.8235 — precision = TP/(TP+FP) = 42/50 = 0.84; recall = TP/(TP+FN) = 42/52 ≈ 0.8077; F1 = 2 x (precision x recall)/(precision+recall) = 2x(0.84x0.8077)/(0.84+0.8077) ≈ 0.8235
F1 = 1.00, since the classifier has more true positives than false positives or false negatives
F1 cannot be computed without knowing TN
Answer: B. F1 ≈ 0.8235 — precision = TP/(TP+FP) = 42/50 = 0.84; recall = TP/(TP+FN) = 42/52 ≈ 0.8077; F1 = 2 x (precision x recall)/(precision+recall) = 2x(0.84x0.8077)/(0.84+0.8077) ≈ 0.8235
ExplanationF1 score is the HARMONIC mean of precision and recall, specifically chosen (rather than a simple average) because it penalizes a large imbalance between the two more heavily — a classifier with, say, perfect precision but terrible recall gets a low F1, not a misleadingly high average. Step by step: precision = TP/(TP+FP) = 42/(42+8) = 42/50 = 0.84 (of everything the model FLAGGED as positive, 84% actually were). Recall = TP/(TP+FN) = 42/(42+10) = 42/52 ≈ 0.8077 (of everything that WAS actually positive, about 80.77% got correctly flagged). F1 = 2 x (0.84 x 0.8077) / (0.84 + 0.8077) ≈ 2 x 0.6785 / 1.6477 ≈ 0.8235. Notably, TN (true negatives) never enters the F1 formula at all — F1 deliberately focuses only on how the model performs on the POSITIVE class, which is exactly why it's the preferred metric over raw accuracy for imbalanced datasets (like rare-disease detection) where TN can be enormous and would otherwise dominate and hide poor positive-class performance.
A dataset has features on very different scales: 'age' (typically 18-90) and 'annual income in rupees' (typically 200,000-5,000,000). Before applying gradient descent to train a linear regression model on this data, why does feature SCALING (e.g., standardizing each feature to mean 0, std 1) matter?
Scaling is purely cosmetic and never affects model training in any way
Without scaling, gradient descent's loss surface becomes highly ELONGATED (steep in the income direction, shallow in the age direction, since a small income coefficient change causes a huge loss change while a small age coefficient change barely moves the loss) — this forces a much smaller learning rate to avoid overshooting on the steep axis, making convergence far slower and more erratic; scaling makes the loss surface more circular/well-conditioned, letting gradient descent converge faster and more reliably
Scaling is only needed for decision trees, never for linear models
Scaling changes the mathematically optimal solution itself, making the model less accurate
Answer: B. Without scaling, gradient descent's loss surface becomes highly ELONGATED (steep in the income direction, shallow in the age direction, since a small income coefficient change causes a huge loss change while a small age coefficient change barely moves the loss) — this forces a much smaller learning rate to avoid overshooting on the steep axis, making convergence far slower and more erratic; scaling makes the loss surface more circular/well-conditioned, letting gradient descent converge faster and more reliably
ExplanationGradient descent updates each parameter proportionally to the loss function's SLOPE (gradient) with respect to that parameter, using a single shared learning rate across all parameters. When features live on wildly different scales, the loss surface becomes elongated: because income values are roughly 100,000x larger than age values, the loss changes MUCH more steeply per unit change in the income coefficient than per unit change in the age coefficient. A learning rate large enough to make reasonable progress along the shallow (age) direction will wildly overshoot and oscillate along the steep (income) direction, while a learning rate small enough to be stable on the steep direction makes painfully slow progress on the shallow one — there is no single learning rate that's ideal for both simultaneously. Standardizing every feature to a comparable scale (like mean 0, standard deviation 1) makes the loss surface much more circular/well-conditioned, so a single learning rate works reasonably well in every direction, letting gradient descent converge both faster and more reliably. Note the underlying OPTIMAL solution itself is mathematically unaffected by scaling — only how EASILY gradient descent finds it changes.
Question 212 · Decision Trees: Information Gain · medium
A decision tree is choosing which feature to split on at a node. Feature A splits the data into two groups with entropy 0.2 and 0.3 respectively (weighted average 0.25). Feature B splits into two groups with entropy 0.6 and 0.7 (weighted average 0.65). If the parent node's entropy was 0.9, which feature gives HIGHER information gain, and why does that make it the better split?
Feature B, because higher child entropy means more information was extracted
Feature A — Information Gain = parent entropy MINUS weighted average child entropy; Feature A gives 0.9 - 0.25 = 0.65, while Feature B gives 0.9 - 0.65 = 0.25; Feature A's split produces MUCH purer child nodes (lower remaining entropy/disorder), representing a bigger REDUCTION in uncertainty about the class label, which is exactly what a good decision tree split should maximize
Both features give identical information gain since they both start from the same parent entropy
Information gain cannot be computed without knowing the exact class labels, only the entropy values
Answer: B. Feature A — Information Gain = parent entropy MINUS weighted average child entropy; Feature A gives 0.9 - 0.25 = 0.65, while Feature B gives 0.9 - 0.65 = 0.25; Feature A's split produces MUCH purer child nodes (lower remaining entropy/disorder), representing a bigger REDUCTION in uncertainty about the class label, which is exactly what a good decision tree split should maximize
ExplanationInformation Gain measures how much a split REDUCES uncertainty (entropy) about the class label, computed as: parent entropy - weighted average of the resulting children's entropy. Lower CHILD entropy means the split produced purer, more class-homogeneous groups — which is the actual goal of a decision tree split (separating classes cleanly). For Feature A: gain = 0.9 - 0.25 = 0.65. For Feature B: gain = 0.9 - 0.65 = 0.25. Feature A wins decisively, because its children (average entropy 0.25) are much purer/less mixed than Feature B's children (average entropy 0.65) — Feature A's split resolved much more of the original uncertainty. A common point of confusion: HIGHER child entropy is BAD, not good, since it means the split failed to separate the classes well; the feature to prefer is the one whose split leaves the LEAST remaining disorder, hence the biggest DROP (gain) from the parent's entropy.
Question 213 · PCA: Explained Variance Ratio · medium
After running PCA (Principal Component Analysis) on a dataset with 4 features, the eigenvalues of the covariance matrix (sorted descending) are: 8.2, 2.1, 0.9, 0.3. If you keep only the first 2 principal components, what percentage of the TOTAL VARIANCE in the original data is retained?
50%, since you're keeping 2 out of 4 components
Approximately 89.57% — total variance is the sum of all eigenvalues (8.2+2.1+0.9+0.3=11.5); the variance retained by the first 2 components is (8.2+2.1)/11.5 = 10.3/11.5 ≈ 0.8957, or about 89.57%, since each eigenvalue represents exactly how much of the total variance its corresponding principal component captures
100%, because PCA never loses any information by design
25%, corresponding to 1 out of 4 features being 'dropped' per component kept
Answer: B. Approximately 89.57% — total variance is the sum of all eigenvalues (8.2+2.1+0.9+0.3=11.5); the variance retained by the first 2 components is (8.2+2.1)/11.5 = 10.3/11.5 ≈ 0.8957, or about 89.57%, since each eigenvalue represents exactly how much of the total variance its corresponding principal component captures
ExplanationIn PCA, each eigenvalue of the covariance matrix directly represents the AMOUNT OF VARIANCE captured by its corresponding principal component (the eigenvalues are sorted so the first component captures the most variance, the second captures the next-most, and so on). The proportion of variance retained by keeping the top-k components is simply the SUM of those k eigenvalues divided by the sum of ALL eigenvalues (total variance). Here: total variance = 8.2+2.1+0.9+0.3 = 11.5. Keeping the top 2 components captures 8.2+2.1=10.3 of that. The ratio is 10.3/11.5 ≈ 0.8957, or about 89.57%. This is exactly why PCA is such an effective dimensionality-reduction tool: despite discarding HALF the components (2 out of 4), it retains nearly 90% of the original data's variance/information, because the discarded components (with small eigenvalues 0.9 and 0.3) were capturing comparatively little of the data's actual spread to begin with.
Question 214 · Pearson Correlation Coefficient · medium
Given the paired data points (x, y): (2,3), (4,5), (6,8), (8,9), (10,13), the Pearson correlation coefficient r comes out to approximately 0.986. What does this specific value indicate, and what would r = -0.986 mean instead?
r ≈ 0.986 means x causes y to increase; a negative r would mean y causes x to decrease — correlation directly establishes causation
r ≈ 0.986 indicates a VERY STRONG POSITIVE linear relationship (as x increases, y tends to increase almost proportionally, with data points lying very close to a straight upward-sloping line); r = -0.986 would indicate an equally strong NEGATIVE linear relationship (as x increases, y tends to decrease), with the SAME strength of linear association, just in the opposite direction — in neither case does correlation alone establish causation
r ≈ 0.986 means the two variables are completely unrelated
r must always be between 0 and 1; a value of -0.986 would be mathematically invalid
Answer: B. r ≈ 0.986 indicates a VERY STRONG POSITIVE linear relationship (as x increases, y tends to increase almost proportionally, with data points lying very close to a straight upward-sloping line); r = -0.986 would indicate an equally strong NEGATIVE linear relationship (as x increases, y tends to decrease), with the SAME strength of linear association, just in the opposite direction — in neither case does correlation alone establish causation
ExplanationPearson's r ranges from -1 to +1, measuring the STRENGTH and DIRECTION of a LINEAR relationship between two variables. A value close to +1 (like the computed ≈0.986 here) indicates a very strong POSITIVE linear association — points lie very close to an upward-sloping straight line, with x and y increasing together. A value close to -1 would indicate an equally strong relationship, but NEGATIVE (downward-sloping) — as one variable increases, the other tends to decrease proportionally; the MAGNITUDE (closeness to 1, ignoring sign) reflects the same strength of linear fit either way, only the sign flips. Critically, correlation — no matter how strong or how close to +-1 — never by itself establishes causation: two variables can be strongly correlated because one causes the other, because both are driven by a shared confounding factor, or purely by coincidence in a limited sample. 'Correlation is not causation' is one of the most important, frequently-violated principles in interpreting statistical results.
Question 215 · Train/Test Split: Shuffling & Data Leakage · hard
A machine learning pipeline splits data into training and test sets. If the ORIGINAL dataset was collected in chronological order (e.g., older transactions first, newer transactions last) and you split it WITHOUT shuffling — taking the first 80% as training and the last 20% as test — what specific problem can this create, separate from ordinary overfitting?
No problem at all — the split percentage (80/20) is all that matters, not the order
If there's a TIME-based pattern or drift in the data (e.g., prices generally rising over time, or a feature's distribution shifting), an unshuffled split means the model trains ONLY on older patterns and gets tested on systematically different, newer patterns it never saw examples of — this isn't a fair test of generalization to a RANDOM new example, it's closer to testing extrapolation into a shifted distribution, which can make test performance look artificially different (often worse) than the model's real capability on a representative random sample
Unshuffled splits always produce identical results to shuffled splits, since the model doesn't know about the file order
This only matters for image data, never for tabular/transaction data
Answer: B. If there's a TIME-based pattern or drift in the data (e.g., prices generally rising over time, or a feature's distribution shifting), an unshuffled split means the model trains ONLY on older patterns and gets tested on systematically different, newer patterns it never saw examples of — this isn't a fair test of generalization to a RANDOM new example, it's closer to testing extrapolation into a shifted distribution, which can make test performance look artificially different (often worse) than the model's real capability on a representative random sample
ExplanationWhen a dataset has any structure correlated with its stored ORDER — very commonly, chronological order — an unshuffled 80/20 split doesn't produce a representative random sample in each half; it instead cleanly separates 'older data' from 'newer data'. If the underlying relationships in the data drift over time (a common real phenomenon called concept drift or distribution shift — customer behavior changes, prices inflate, product mixes shift), the model trained purely on the older 80% may be tested on a genuinely different data distribution in the newer 20%, which is a fundamentally different question than 'does this model generalize to new random examples drawn from the same distribution it trained on'. This is why standard practice is to SHUFFLE data (breaking any incidental order-based structure) before splitting into train/test, UNLESS you specifically intend to test time-based generalization (as in real financial forecasting, where a deliberately chronological split — train on the past, test on the future — is actually the CORRECT choice, since that mirrors how the model will actually be used in production).
Question 216 · Softmax Function Computation · medium
A neural network's output layer produces raw scores (logits) [2.0, 1.0, 0.1] for a 3-class problem. Applying softmax converts these into probabilities. Roughly, what are the resulting probabilities, and what key property must they satisfy?
The probabilities are exactly [2.0, 1.0, 0.1] unchanged — softmax just relabels the logits
Approximately [0.659, 0.242, 0.099] — computed as exp(logit_i) / sum(exp(all logits)) for each class: exp(2.0)≈7.389, exp(1.0)≈2.718, exp(0.1)≈1.105, sum≈11.212, giving roughly 7.389/11.212≈0.659, 2.718/11.212≈0.242, and 1.105/11.212≈0.099; critically, these three probabilities MUST sum to exactly 1.0, since softmax is specifically constructed to produce a valid probability distribution over the classes
The probabilities are simply the logits divided by their sum without any exponentiation: [0.645, 0.323, 0.032]
Softmax cannot be applied when logits contain a value as small as 0.1
Answer: B. Approximately [0.659, 0.242, 0.099] — computed as exp(logit_i) / sum(exp(all logits)) for each class: exp(2.0)≈7.389, exp(1.0)≈2.718, exp(0.1)≈1.105, sum≈11.212, giving roughly 7.389/11.212≈0.659, 2.718/11.212≈0.242, and 1.105/11.212≈0.099; critically, these three probabilities MUST sum to exactly 1.0, since softmax is specifically constructed to produce a valid probability distribution over the classes
ExplanationSoftmax converts a vector of arbitrary real-valued logits into a valid probability distribution via softmax(z_i) = exp(z_i) / sum(exp(z_j) for all j) — exponentiating first ensures every output is positive (a probability can't be negative), and dividing by the sum of all exponentials ensures the results sum to exactly 1 (a valid distribution). Computing this for [2.0, 1.0, 0.1]: exp(2.0)≈7.389, exp(1.0)≈2.718, exp(0.1)≈1.105, summing to ≈11.212. Dividing each exponential by that sum gives approximately [0.659, 0.242, 0.099] (which does sum to 1.0, up to rounding). Notice the RELATIVE gaps get exaggerated by the exponential — a logit gap of just 1.0 (between 2.0 and 1.0) becomes a probability gap of roughly 0.659 vs 0.242, a much larger relative difference — this is a deliberate property of softmax, making the highest-scoring class's probability dominate more sharply than the raw logit gap alone would suggest, and why temperature scaling (dividing logits by a constant before softmax) is used to control this sharpness.
Question 217 · Cosine Similarity Computation · medium
Two vectors represent word-frequency profiles for documents: a=(3,4,0) and b=(4,3,0). What is their COSINE similarity, and why is cosine similarity often preferred over Euclidean distance for comparing text documents of different lengths?
Cosine similarity = 1.0, since the two vectors contain the same numbers just swapped
Cosine similarity = 0.96 — computed as (a·b)/(|a||b|): dot product = 3x4+4x3+0x0 = 24; |a| = sqrt(9+16+0) = 5; |b| = sqrt(16+9+0) = 5; so cosine similarity = 24/(5x5) = 24/25 = 0.96; cosine similarity is preferred for text because it measures the ANGLE between vectors (their relative proportions/direction), ignoring MAGNITUDE — so a short document and a long document discussing the exact same topics in the same proportions would still be judged highly similar, unlike Euclidean distance, which is sensitive to overall vector length/magnitude
Cosine similarity is undefined whenever any vector component is 0
Cosine similarity always equals the dot product directly, with no need for normalization
Answer: B. Cosine similarity = 0.96 — computed as (a·b)/(|a||b|): dot product = 3x4+4x3+0x0 = 24; |a| = sqrt(9+16+0) = 5; |b| = sqrt(16+9+0) = 5; so cosine similarity = 24/(5x5) = 24/25 = 0.96; cosine similarity is preferred for text because it measures the ANGLE between vectors (their relative proportions/direction), ignoring MAGNITUDE — so a short document and a long document discussing the exact same topics in the same proportions would still be judged highly similar, unlike Euclidean distance, which is sensitive to overall vector length/magnitude
ExplanationCosine similarity is computed as the dot product of two vectors divided by the product of their magnitudes (lengths): (a·b)/(|a||b|). Working it out: the dot product a·b = (3)(4)+(4)(3)+(0)(0) = 12+12+0 = 24. The magnitude of a is sqrt(3^2+4^2+0^2) = sqrt(25) = 5. The magnitude of b is sqrt(4^2+3^2+0^2) = sqrt(25) = 5. So cosine similarity = 24/(5x5) = 24/25 = 0.96, indicating the vectors point in nearly (though not exactly) the same direction. The reason this metric suits text comparison specifically: it captures the ANGLE between vectors — essentially the relative PROPORTIONS of each word/dimension — while being completely insensitive to overall magnitude/length. A 100-word document and a 10,000-word document discussing the same topic in the same relative word-frequency proportions would have a HIGH cosine similarity (nearly parallel vectors) despite having wildly different Euclidean distances (since one vector is simply much 'longer' than the other in raw magnitude) — exactly the behavior you want when document length shouldn't affect a topical-similarity judgment.
Question 218 · Ridge vs Lasso Regularization · hard
A linear regression model is regularized with L1 penalty (Lasso) versus L2 penalty (Ridge). Both add a penalty term to the loss based on the magnitude of the model's coefficients. What is the key PRACTICAL difference in the resulting coefficients, and why does it happen?
There is no practical difference; L1 and L2 always produce identical coefficient values
Lasso (L1) tends to push some coefficients EXACTLY to zero, effectively performing automatic FEATURE SELECTION, while Ridge (L2) tends to shrink all coefficients toward zero but rarely to EXACTLY zero — this happens because L1's penalty (sum of absolute values) has a geometric 'corner' at zero that the optimization solution often lands on exactly, while L2's penalty (sum of squares) is smooth everywhere and has no such corner to encourage exact zeros
Ridge always eliminates more features than Lasso
L1 and L2 regularization both increase model coefficients rather than shrinking them
Answer: B. Lasso (L1) tends to push some coefficients EXACTLY to zero, effectively performing automatic FEATURE SELECTION, while Ridge (L2) tends to shrink all coefficients toward zero but rarely to EXACTLY zero — this happens because L1's penalty (sum of absolute values) has a geometric 'corner' at zero that the optimization solution often lands on exactly, while L2's penalty (sum of squares) is smooth everywhere and has no such corner to encourage exact zeros
ExplanationBoth regularization methods discourage overly large coefficients (which helps prevent overfitting), but they penalize magnitude differently, with a real geometric consequence. L1 (Lasso) penalizes the SUM OF ABSOLUTE VALUES of the coefficients; its constraint region (in coefficient space) has sharp CORNERS located exactly on the axes (where some coefficient equals precisely zero) — the optimal solution frequently lands exactly on one of these corners, driving some coefficients to EXACTLY zero and effectively removing those features from the model entirely, which is why Lasso is popular for automatic feature selection on high-dimensional data. L2 (Ridge) penalizes the SUM OF SQUARED coefficients; its constraint region is a smooth circle/sphere with no corners, so the optimal solution can land anywhere on that smooth boundary — coefficients get shrunk toward zero (reducing their influence) but essentially never hit EXACTLY zero, so Ridge keeps every feature in the model, just with reduced weight. Choosing between them in practice often comes down to whether you want a sparser, more interpretable model (favoring Lasso) or want to retain and moderately shrink every available feature (favoring Ridge).
Question 219 · Standard Deviation & Bessel's Correction · hard
A dataset of exam scores: [4, 8, 6, 5, 3, 10]. What is the POPULATION standard deviation, and why does the SAMPLE standard deviation formula divide by (n-1) instead of n?
Population std ≈ 2.38, and sample std uses (n-1) purely as an arbitrary historical convention with no real justification
Population std ≈ 2.38 (mean=6; sum of squared deviations from mean = 4+4+0+1+9+16=34; divide by n=6 gives variance≈5.667; sqrt≈2.38); the SAMPLE version divides by (n-1)≈2.61 instead, because when you only have a SAMPLE (not the whole population), the sample MEAN itself is computed FROM that same sample, which makes the data points appear slightly closer to their own sample mean than they'd be to the true (unknown) population mean — dividing by the smaller (n-1) corrects this downward bias, giving a more accurate estimate of the population's true variability
Sample standard deviation is always exactly double the population standard deviation
Standard deviation cannot be computed for a dataset with fewer than 10 values
Answer: B. Population std ≈ 2.38 (mean=6; sum of squared deviations from mean = 4+4+0+1+9+16=34; divide by n=6 gives variance≈5.667; sqrt≈2.38); the SAMPLE version divides by (n-1)≈2.61 instead, because when you only have a SAMPLE (not the whole population), the sample MEAN itself is computed FROM that same sample, which makes the data points appear slightly closer to their own sample mean than they'd be to the true (unknown) population mean — dividing by the smaller (n-1) corrects this downward bias, giving a more accurate estimate of the population's true variability
ExplanationFor the population standard deviation: the mean of [4,8,6,5,3,10] is (4+8+6+5+3+10)/6 = 36/6 = 6. The squared deviations from this mean are (4-6)^2=4, (8-6)^2=4, (6-6)^2=0, (5-6)^2=1, (3-6)^2=9, (10-6)^2=16, summing to 34. Dividing by n=6 gives variance≈5.667, and the square root gives population std≈2.38. The SAMPLE standard deviation instead divides that same sum of squared deviations by (n-1)=5, giving a slightly LARGER value (≈2.61). The reason for this adjustment, known as Bessel's correction: when you compute a sample's mean FROM the sample itself (rather than knowing the true population mean independently), the sample's data points are, on average, slightly CLOSER to that self-computed sample mean than they would be to the true, unknown population mean — the sample mean is, in a specific mathematical sense, the value that MINIMIZES the sum of squared deviations for that particular sample. Dividing by the smaller n-1 instead of n compensates for this built-in underestimation, producing an unbiased estimate of the true population variance when working from sample data — which is why nearly every real statistics library defaults to n-1 for sample data.
Question 220 · Central Limit Theorem & Standard Error · hard
A fair six-sided die (population mean 3.5, population standard deviation ≈1.708) is rolled repeatedly, and you compute the AVERAGE of each batch of rolls. According to the Central Limit Theorem, how does the standard error of that average change as the batch size n grows from 1 to 30 to 100, and why does this matter for machine learning?
The standard error stays constant regardless of n, since the die itself never changes
The standard error SHRINKS as n grows, following standard_error = population_std / sqrt(n): at n=1 it's ≈1.708 (a single roll, maximally variable), at n=30 it drops to ≈0.312, and at n=100 it drops further to ≈0.171 — larger samples give more STABLE, reliable estimates of the true underlying mean, which is exactly why machine learning cares about having enough validation/test data: a metric computed on too few examples is itself a noisy, unreliable estimate of true model performance
The standard error grows linearly with n, making larger samples strictly worse for estimation
Standard error only applies to normally-distributed populations, not to die rolls
Answer: B. The standard error SHRINKS as n grows, following standard_error = population_std / sqrt(n): at n=1 it's ≈1.708 (a single roll, maximally variable), at n=30 it drops to ≈0.312, and at n=100 it drops further to ≈0.171 — larger samples give more STABLE, reliable estimates of the true underlying mean, which is exactly why machine learning cares about having enough validation/test data: a metric computed on too few examples is itself a noisy, unreliable estimate of true model performance
ExplanationThe Central Limit Theorem describes how the distribution of a SAMPLE MEAN behaves as sample size grows: regardless of the original population's shape (a die roll is uniformly distributed, not normal, yet the theorem still applies), the sampling distribution of the mean becomes increasingly concentrated around the true population mean, with standard error = population_std / sqrt(n). Computing this for the die (population std ≈1.708): at n=1, standard error ≈1.708/sqrt(1)=1.708 (identical to the raw population spread, since a 'batch' of 1 roll IS just that one roll). At n=30, standard error ≈1.708/sqrt(30)≈0.312 — over 5 times tighter. At n=100, standard error ≈1.708/sqrt(100)≈0.171 — 10 times tighter than a single roll. This sqrt(n) shrinkage is precisely why machine learning practitioners insist on adequately sized validation and test sets: a model's measured accuracy on just 20 test examples is itself a highly UNSTABLE estimate of its true real-world performance (large standard error), while the same measured accuracy on 10,000 test examples is a far more trustworthy, stable estimate (small standard error) of the same underlying quantity.
Question 221 · L1 vs L2 Distance (Norms) · medium
Two points in feature space: a=(1,7) and b=(4,3). What are the L1 (Manhattan) distance and the L2 (Euclidean) distance between them, and what is the geometric difference between what each measures?
L1 = 5.0 and L2 = 7, the reverse of the correct values
L1 (Manhattan) distance = |1-4| + |7-3| = 3+4 = 7 — the sum of absolute per-dimension differences, like navigating a city grid where you can only move along axis-aligned streets; L2 (Euclidean) distance = sqrt((1-4)^2 + (7-3)^2) = sqrt(9+16) = sqrt(25) = 5.0 — the straight-line 'as the crow flies' distance; L2 is always LESS THAN OR EQUAL TO L1 for the same two points, since the diagonal straight line is never longer than the sum of the axis-aligned detour
Both distances are always identical for any two points, regardless of their coordinates
L1 and L2 distance can only be computed for points with exactly 2 dimensions
Answer: B. L1 (Manhattan) distance = |1-4| + |7-3| = 3+4 = 7 — the sum of absolute per-dimension differences, like navigating a city grid where you can only move along axis-aligned streets; L2 (Euclidean) distance = sqrt((1-4)^2 + (7-3)^2) = sqrt(9+16) = sqrt(25) = 5.0 — the straight-line 'as the crow flies' distance; L2 is always LESS THAN OR EQUAL TO L1 for the same two points, since the diagonal straight line is never longer than the sum of the axis-aligned detour
ExplanationL1 (Manhattan/taxicab) distance sums the ABSOLUTE differences along each dimension independently: |1-4|+|7-3| = 3+4 = 7 — geometrically, this is the distance you'd travel moving only along grid-aligned streets (like navigating city blocks), never diagonally. L2 (Euclidean) distance instead takes the straight-line distance: sqrt((1-4)^2+(7-3)^2) = sqrt((-3)^2+4^2) = sqrt(9+16) = sqrt(25) = 5.0 — the direct 'as the crow flies' path. Notice L2 (5.0) is smaller than L1 (7) here, which is not a coincidence: by the triangle inequality, the direct diagonal path can never be LONGER than the sum of the axis-aligned detour, so L2 <= L1 always holds for any pair of points. This distinction matters practically in ML: L2 distance is more sensitive to one large per-dimension difference (since squaring amplifies large gaps disproportionately), while L1 treats every unit of difference in any dimension equally — which is part of why L1 regularization (Lasso) tends to produce sparser solutions than L2 regularization (Ridge), echoing the same underlying geometric distinction.