Question 1 · Bias-Variance Tradeoff and Overfitting · hard
In a binary classification problem, you train a logistic regression model and observe: training accuracy = 96%, validation accuracy = 62%, with training loss = 0.08 and validation loss = 0.85. What is the primary diagnostic issue here, and how would you address it using cross-validation and regularization parameters?
Validation accuracy is low because the validation set is too small; increasing its size to match the training set size will automatically close the gap, since sample size alone determines the training-validation difference regardless of model complexity.
The model is underfitting because the training accuracy is too high, so you should collect more training data to make it harder for the model to reach 96% accuracy, which would pull training accuracy down toward the validation accuracy and eliminate the gap.
This indicates severe overfitting because the 34-point gap between training and validation accuracy suggests the model memorized training data rather than learning generalizable patterns. Apply k-fold cross-validation (k=5 or 10) to detect this across splits, then increase L2 regularization penalty λ from 0.001 to 0.1 to constrain coefficient magnitudes, reducing model complexity and the training-validation gap as demonstrated by validation accuracy recovering toward training accuracy across CV folds
The problem is data leakage, so you must shuffle the dataset randomly and retrain the model from scratch on the shuffled data, since leakage is caused by the ordering of examples rather than by overlapping information between the training and validation sets.
Answer: C. This indicates severe overfitting because the 34-point gap between training and validation accuracy suggests the model memorized training data rather than learning generalizable patterns. Apply k-fold cross-validation (k=5 or 10) to detect this across splits, then increase L2 regularization penalty λ from 0.001 to 0.1 to constrain coefficient magnitudes, reducing model complexity and the training-validation gap as demonstrated by validation accuracy recovering toward training accuracy across CV folds
ExplanationFirst, a 34% gap between training (96%) and validation (62%) accuracy is the defining signature of overfitting, not underfitting. The model achieves low training loss (0.08) but high validation loss (0.85), because regularization λ is too small. Then, since logistic regression loss is L = -(1/n)Σ[y log(ŷ) + (1-y) log(1-ŷ)] + λ/2 Σw², increasing λ penalizes large weights. K-fold cross-validation measures this gap across k splits; if mean validation accuracy ≈ 62% across folds but training ≈ 96%, overfitting is confirmed. Finally, therefore, regularization λ should increase 100× (from 0.001 to 0.1) to constrain the decision boundary.
Question 2 · Learning Rate and Convergence · hard
You implement gradient descent optimization for linear regression on a dataset with 10,000 samples and 50 features. Starting with learning rate α=0.1, after 100 iterations the cost function J(θ) stops decreasing and plateaus at 0.42. What is the most likely cause, and how would you diagnose and fix it using the step-size analysis and loss curves?
The dataset has too many features (50) relative to samples, causing overfitting; the fix is to randomly remove features until J(θ) decreases further, which is the standard way to escape a plateau in gradient descent.
The model has already reached the global minimum, since the cost function for linear regression is convex; a plateau after 100 iterations always indicates true convergence, and no adjustment to the learning rate can reduce J(θ) below 0.42.
The learning rate α=0.1 is too large, causing gradient descent to overshoot the minimum and oscillate around θ*. Diagnose by plotting J(θ) vs iteration number — oscillation or divergence confirms this — then reduce α to 0.01 or 0.001 and retrain; convergence should become smooth and J should reach a lower minimum.
Gradient descent is fundamentally unsuitable for this problem, so the correct fix is to abandon it in favor of the normal equation θ = (XᵀX)⁻¹Xᵀy, which always converges to the optimal solution in a single step regardless of learning rate issues.
Answer: C. The learning rate α=0.1 is too large, causing gradient descent to overshoot the minimum and oscillate around θ*. Diagnose by plotting J(θ) vs iteration number — oscillation or divergence confirms this — then reduce α to 0.01 or 0.001 and retrain; convergence should become smooth and J should reach a lower minimum.
ExplanationWhen J(θ) plateaus before reaching a low value, an oversized learning rate α is the most common cause. For linear regression with squared-error loss J(θ) = (1/2n)||Xθ - y||², the update rule is θ := θ - α∇J(θ), where ∇J(θ) = (1/n)Xᵀ(Xθ-y). If α is too large, each update overshoots the minimum, so θ oscillates instead of converging. Diagnosis: plot J(θ) against iteration number — an oscillating or diverging curve confirms α is too large — then reduce α to 0.01 or 0.001 and retrain; a well-tuned run should show J decreasing smoothly to a lower plateau. This is why practitioners typically grid-search α over values like {0.001, 0.01, 0.1} and monitor the loss curve for oscillation before trusting a plateau as true convergence.
Question 3 · Multi-class Metrics and Threshold Optimization · hard
A 3-class email classifier (Spam, Promo, Ham) is evaluated on a test set of 200 emails, producing the confusion matrix below (rows = true label, columns = predicted label):
```
Pred:Spam Pred:Promo Pred:Ham
True Spam: 57 6 32
True Promo: 3 54 3
True Ham: 0 0 45
```
What are the macro-averaged F1-score and the support-weighted F1-score for this classifier, and what does the gap between the two values reveal about where the model's errors are concentrated?
Since Spam, Promo, and Ham have supports of 95, 60, and 45 out of 200 emails, weighted F1 is calculated as the simple unweighted mean of the three per-class F1 values (0.735, 0.900, 0.720) — giving weighted F1 ≈ 0.785 — while macro F1 is calculated as the support-weighted mean, giving macro F1 ≈ 0.781.
Spam has precision 57/60 = 0.95 and recall 57/95 = 0.60 (F1 ≈ 0.735), Promo has precision = recall = 54/60 = 0.90 (F1 = 0.900), and Ham has precision 45/80 = 0.5625 and recall 45/45 = 1.0 (F1 = 0.720); averaging these three F1 values equally gives macro F1 ≈ 0.785, while weighting them by support (95, 60, 45) gives weighted F1 ≈ 0.781 — slightly lower than macro because Spam, the largest class by support, is also the weakest performer (its recall of 0.60 means 38 of 95 spam emails are missed), so its poor F1 pulls the support-weighted average down more than it pulls the unweighted macro average.
Using precision = TP / column-sum and recall = TP / row-sum for each class still gives macro F1 ≈ 0.785 and weighted F1 ≈ 0.781, but the gap arises because Ham's small support (45 of 200) causes it to be dropped from the weighted calculation entirely, so weighted F1 reduces to an average over only Spam and Promo's F1-scores while macro F1 still averages over all three classes.
Dividing each class's true-positive count by the total dataset size of 200 instead of by the relevant row or column sum gives Spam a precision and recall of 57/200 ≈ 0.285, Promo 54/200 ≈ 0.270, and Ham 45/200 ≈ 0.225, which pulls both macro and weighted F1 down to roughly 0.26–0.27 and would incorrectly suggest the classifier performs poorly across the board despite correctly classifying 156 of 200 emails.
Answer: B. Spam has precision 57/60 = 0.95 and recall 57/95 = 0.60 (F1 ≈ 0.735), Promo has precision = recall = 54/60 = 0.90 (F1 = 0.900), and Ham has precision 45/80 = 0.5625 and recall 45/45 = 1.0 (F1 = 0.720); averaging these three F1 values equally gives macro F1 ≈ 0.785, while weighting them by support (95, 60, 45) gives weighted F1 ≈ 0.781 — slightly lower than macro because Spam, the largest class by support, is also the weakest performer (its recall of 0.60 means 38 of 95 spam emails are missed), so its poor F1 pulls the support-weighted average down more than it pulls the unweighted macro average.
ExplanationFor each class, precision = TP divided by that class's column total (everything predicted as that class) and recall = TP divided by that class's row total (everything actually belonging to that class). Spam: TP = 57, column total = 57+3+0 = 60, row total = 57+6+32 = 95, so precision = 57/60 = 0.95 and recall = 57/95 = 0.60, giving F1 = 2(0.95)(0.60)/(0.95+0.60) ≈ 0.735. Promo: TP = 54, column total = 6+54+0 = 60, row total = 3+54+3 = 60, so precision = recall = 0.90 and F1 = 0.900. Ham: TP = 45, column total = 32+3+45 = 80, row total = 0+0+45 = 45, so precision = 45/80 = 0.5625 and recall = 45/45 = 1.0, giving F1 = 2(0.5625)(1.0)/(0.5625+1.0) = 0.720. Macro F1 treats every class equally regardless of size: (0.735 + 0.900 + 0.720)/3 ≈ 0.785. Weighted F1 instead scales each class's F1 by how many true examples it has out of 200: (0.735×95 + 0.900×60 + 0.720×45)/200 ≈ 0.781. The weighted score comes out lower than the macro score because Spam — the class with by far the largest support (95 of 200 emails) — is also the weakest performer, dragging the support-weighted average toward its low recall of 0.60 (38 of 95 spam emails are being misclassified, mostly as Ham) more than it drags down the simple three-way average. Practically, this means the classifier's biggest real-world problem is spam slipping into the inbox mislabeled as Ham; lowering the confidence threshold for predicting Spam, or reweighting the Spam class during training, would directly target that weakness.
Question 4 · Probability & Statistics · hard
A factory has two machines producing the same component. Machine A produces 60% of all components with a defect rate of 2%, while Machine B produces the remaining 40% with a defect rate of 5%. If a randomly selected component is found to be defective, what is the probability that it was produced by Machine B?
40%, since Machine B accounts for 40% of total production, and equal prior share should carry over directly as the posterior probability of defect origin
71.4%, obtained from 0.05/(0.05+0.02) by comparing the two machines' defect rates directly without weighting by their production volumes
5%, because Machine B's own defect rate is 5%, which some mistakenly treat as already answering how likely a defective item is to have come from Machine B
Answer: A. 62.5% — apply Bayes' theorem: P(D) = (0.6)(0.02) + (0.4)(0.05) = 0.032, so P(B|D) = (0.4)(0.05)/0.032 = 0.625
ExplanationFirst, compute the total probability of a defective component using the law of total probability: P(D) = P(A)·P(D|A) + P(B)·P(D|B) = (0.6)(0.02) + (0.4)(0.05) = 0.012 + 0.02 = 0.032, meaning 3.2% of all components are defective. Then apply Bayes' theorem to find P(B|D) = P(B)·P(D|B)/P(D) = (0.4)(0.05)/0.032 = 0.02/0.032 = 0.625, so 62.5% of defective components came from Machine B. This is higher than Machine B's 40% share of overall production because its defect rate (5%) is more than double Machine A's (2%), so defective items are disproportionately likely to have originated from Machine B even though Machine A makes more components overall. Answers that use the production share alone, compare the raw defect rates without weighting by production volume, or equate a machine's own defect rate with the reverse conditional probability all confuse prior probability, likelihood, and posterior probability — the three quantities Bayes' theorem combines.
Question 5 · Gradient Descent · hard
Consider a loss function L(w) = (w - 5)² starting from w₀ = 0 with learning rate α = 0.1. Compute the gradient ∇L(w) = 2(w - 5) and perform 3 gradient descent steps. Show how w evolves: w_{n+1} = w_n?
Gradient descent converges immediately: after 3 steps, w = 5.0 exactly, since the loss is quadratic and convex.
The steps overshoot repeatedly: after 3 steps, w = -2.3, because α = 0.1 is too large for this loss curvature.
The loss surface is non-convex here: after 3 steps, w = 0.5, so gradient descent moves away from the true minimum at w = 5.
Answer: C. Step 1: w₁ = 0 − 0.1·2(0−5) = 1. Step 2: w₂ = 1 − 0.1·2(1−5) = 1.8. Step 3: w₃ = 1.8 − 0.1·2(1.8−5) = 2.44. So after 3 gradient descent steps, w = 2.44.
ExplanationStep 0: w₀ = 0, ∇L(0) = 2(0−5) = −10, so w₁ = 0 − 0.1(−10) = 1. Step 1: ∇L(1) = 2(1−5) = −8, so w₂ = 1 − 0.1(−8) = 1.8. Step 2: ∇L(1.8) = 2(1.8−5) = −6.4, so w₃ = 1.8 − 0.1(−6.4) = 2.44. The gradient magnitude shrinks each step (10 → 8 → 6.4) as w approaches the minimum at w = 5, and with α = 0.1 the updates move steadily toward that minimum without overshooting.
Question 6 · Gradient Descent - Learning Rate · hard
A gradient descent algorithm minimizes the loss function L(w) = w², whose gradient is ∇L(w) = 2w. Starting from w₀ = 8, a single update w₁ = w₀ − α·∇L(w₀) is computed for three learning rates: α₁ = 0.05, α₂ = 0.4, and α₃ = 1.5. Which option correctly computes all three values of w₁ and correctly classifies each learning rate's long-run behavior on this loss surface?
With gradient ∇L(w)=2w, the updates are w₁=8−0.05(16)=7.2 for α₁ (slow, steady convergence), w₁=8−0.4(16)=1.6 for α₂ (fast convergence), and w₁=8−1.5(16)=−16 for α₃, which diverges as the weight's magnitude grows every step.
Computing each update gives w₁=7.2 for α₁, w₁=1.6 for α₂, and w₁=−16 for α₃, but all three learning rates still converge eventually since L(w)=w² is convex and convexity guarantees gradient descent converges regardless of step size.
Using the same update rule, α₁ produces w₁=7.2 and converges slowly, while both α₂ (w₁=1.6) and α₃ (w₁=−16) diverge, because any learning rate above 0.1 causes gradient descent to overshoot the minimum on this loss surface.
Applying w₁=w₀−α·w₀ since the gradient equals w for this loss function gives w₁=7.6 for α₁, w₁=4.8 for α₂, and w₁=−4 for α₃, with only the largest learning rate causing divergence.
Answer: A. With gradient ∇L(w)=2w, the updates are w₁=8−0.05(16)=7.2 for α₁ (slow, steady convergence), w₁=8−0.4(16)=1.6 for α₂ (fast convergence), and w₁=8−1.5(16)=−16 for α₃, which diverges as the weight's magnitude grows every step.
ExplanationFor L(w) = w², the gradient is ∇L(w) = 2w, so at w₀ = 8 the gradient equals 16. Each update is w₁ = w₀ − α(16), which can be rewritten as w₁ = w₀(1 − 2α). For α₁ = 0.05: w₁ = 8 − 0.05(16) = 8 − 0.8 = 7.2, with multiplier (1 − 2·0.05) = 0.9 — close to 1, so the weight shrinks toward the minimum slowly. For α₂ = 0.4: w₁ = 8 − 0.4(16) = 8 − 6.4 = 1.6, with multiplier (1 − 2·0.4) = 0.2 — small and positive, so the weight collapses toward zero quickly. For α₃ = 1.5: w₁ = 8 − 1.5(16) = 8 − 24 = −16, with multiplier (1 − 2·1.5) = −2; since |−2| > 1, every subsequent update flips sign and doubles in magnitude, so the sequence diverges instead of settling near the minimum. The general rule on this loss surface is that gradient descent converges only when 0 < α < 1, the point where |1 − 2α| crosses 1; rates in that range shrink w toward zero (slowly near 0, quickly near 1), while rates above it cause overshoot that grows without bound. This is why convexity alone does not guarantee convergence for an arbitrarily large step size, and why the divergence threshold is not simply "any rate above 0.1" — it depends on the curvature of the loss (here, twice the coefficient of w², i.e., 2), so using w instead of 2w for the gradient understates every update's true step size.
Question 7 · Linear Regression - Residuals · hard
A linear regression model fitted to a dataset produces the equation ŷ = 3x - 2. For a new data point where x = 5 and the actual observed value is y = 17, compute the residual e = y - ŷ, and determine why the least-squares method minimizes the sum of squared residuals ∑e² rather than the sum of raw residuals ∑e — which of the following is correct?
The residual is e = 17 - 13 = 4, showing the actual value exceeds the prediction by 4 units; least squares minimizes ∑e² instead of ∑e because squaring prevents positive and negative residuals from canceling out, keeps the objective differentiable everywhere so a closed-form minimum can be solved for, and penalizes larger errors more heavily than smaller ones.
Since residual is defined as e = ŷ - y, the value here equals 13 - 17 = -4, and least squares uses ∑e² purely to keep the arithmetic simpler than working with ∑e, since both objectives always produce the identical best-fit line.
The residual equals e = 17 - 13 = 4, but least squares instead minimizes ∑e² because summing the raw residuals ∑e for any regression line, good or bad, always equals zero, making ∑e useless for comparing how well different lines fit the data.
Dividing the actual value by the predicted value gives e = 17/13 ≈ 1.31, and least squares minimizes ∑e² because ratios greater than 1 indicate underprediction that must be squared to remove the effect of division.
Answer: A. The residual is e = 17 - 13 = 4, showing the actual value exceeds the prediction by 4 units; least squares minimizes ∑e² instead of ∑e because squaring prevents positive and negative residuals from canceling out, keeps the objective differentiable everywhere so a closed-form minimum can be solved for, and penalizes larger errors more heavily than smaller ones.
ExplanationSubstituting x = 5 into the fitted line gives ŷ = 3(5) - 2 = 13, and the residual is e = y - ŷ = 17 - 13 = 4, meaning the actual value lies 4 units above the model's prediction. Least squares targets ∑e² instead of ∑e for three linked reasons: squaring removes the sign, so a residual of +4 and one of -4 both contribute positively instead of canceling in the total; the squared function is smooth and differentiable everywhere, which lets the minimum be found by setting derivatives to zero and solving directly for the slope and intercept; and squaring grows faster than the residual itself, so one point that is far off the line is penalized far more heavily than several points that are only slightly off, discouraging large individual errors. The claim that raw residuals always sum to zero for any regression line is false — that property (∑e = 0) holds specifically for the line obtained by ordinary least squares with an intercept term, not for an arbitrary line chosen by eye or by some other rule.
Question 8 · Random Forests · hard
A Random Forest is trained with 50 decision trees, and each tree is built on a bootstrap sample of size n = 2000 drawn with replacement from a training set of exactly 2000 examples. For any single tree, the probability that one specific training example is never chosen across all 2000 draws is (1 − 1/n)^n, which for large n converges to 1/e ≈ 0.368. Based on this large-n approximation, which statement correctly describes how many training examples are out-of-bag (OOB) for a given tree and how the OOB mechanism is used to estimate generalization error without a separate validation set?
Roughly 36.8% of the training examples are excluded from any single tree's bootstrap sample, so a given example is OOB for about 18 of the 50 trees on average, and averaging that example's predictions across only those trees yields an internal estimate of test error.
Exactly 50% of the training examples are excluded from each bootstrap sample, since sampling with replacement gives every example a 1-in-2 chance of selection, leaving half the trees available for OOB validation of every example.
None of the training examples are excluded from any bootstrap sample, because drawing 2000 times with replacement from 2000 examples always selects every distinct example at least once, so Random Forests cannot compute an OOB error.
About 63.2% of the training examples are excluded from each bootstrap sample, which means most trees never see most of the data, making the OOB estimate overly pessimistic relative to k-fold cross-validation.
Answer: A. Roughly 36.8% of the training examples are excluded from any single tree's bootstrap sample, so a given example is OOB for about 18 of the 50 trees on average, and averaging that example's predictions across only those trees yields an internal estimate of test error.
ExplanationFor sampling with replacement, the probability a specific example is missed in one draw is (1 − 1/n); over n draws this is (1 − 1/n)^n, which converges to e^(−1) ≈ 0.368 as n grows large — so about 36.8% of examples are left out of any given tree's bootstrap sample, not 0%, not 50%, and not 63.2% (that figure is the fraction that IS included, i.e. 1 − 0.368). Across 50 trees, a given example is expected to be OOB for 0.368 × 50 ≈ 18.4, i.e. about 18, trees. The Random Forest OOB error is computed by predicting each training example only from the trees for which it was OOB, then comparing those predictions to the true label — this gives an internal, unbiased estimate of generalization performance without needing to hold out a separate validation set, one of bagging's practical advantages over methods that require explicit k-fold cross-validation.
Question 9 · K-Means - Choosing K · hard
In K-Means clustering, the Elbow method plots within-cluster sum of squares (WCSS) = ΣΣ||x − μc||² against the number of clusters k, and WCSS always decreases (or stays the same) as k increases. A dataset produces the following WCSS values: k=2 → 300, k=3 → 210, k=4 → 155, k=5 → 143, k=6 → 134. Based on where the rate of decrease in WCSS changes most sharply, at which value of k is the elbow located?
k = 6 is the elbow because it has the lowest WCSS (134) of all the values tested, and the elbow method is defined as selecting whichever k minimizes WCSS overall.
Choosing k = 3 is correct because the drop from k=2 to k=3 (90 units) is the single largest decrease anywhere in the table, making it the most significant improvement in cluster fit.
k = 4 marks the elbow because the marginal drop falls sharply from 55 units (k=3→k=4) to just 12 units (k=4→k=5), and the following drop (9 units, k=5→k=6) stays similarly small — showing that gains beyond k=4 are only marginal compared to the sharp improvements recorded before it.
The smallest cluster count, k = 2, is the elbow because using fewer clusters keeps the model as simple as possible and avoids the overfitting risk that comes with adding any additional clusters.
Answer: C. k = 4 marks the elbow because the marginal drop falls sharply from 55 units (k=3→k=4) to just 12 units (k=4→k=5), and the following drop (9 units, k=5→k=6) stays similarly small — showing that gains beyond k=4 are only marginal compared to the sharp improvements recorded before it.
ExplanationWCSS decreases monotonically as k increases, but the elbow method looks for the point where that decrease stops being substantial and starts flattening out, not for the single biggest drop or the lowest raw value. The successive drops here are 90 units (k=2→k=3), 55 units (k=3→k=4), 12 units (k=4→k=5), and 9 units (k=5→k=6). Going into k=4 the drop is still a healthy 55 units, but immediately after k=4 it collapses to 12 units, and the next drop (9 units, k=5→k=6) stays in that same small range — each cluster added beyond k=4 buys only a marginal reduction in WCSS. That flattening pattern is what defines the elbow, so k=4 is the point that balances a good fit against unnecessary model complexity, even though k=6 technically has the lowest WCSS and k=2→k=3 technically has the largest single drop.
Question 10 · Neural Network Fundamentals · hard
A feedforward neural network has the following architecture: an input layer with 4 features, a fully connected hidden layer with 5 neurons using ReLU activation, and an output layer with 3 neurons for a 3-class classification task. Every layer includes a bias vector — one bias per neuron — added after the weight multiplication. What is the total number of trainable parameters (weights and biases combined) in this network?
Layer 1 contributes (4×5 weights) + 5 biases = 25 parameters, and Layer 2 contributes (5×3 weights) + 3 biases = 18 parameters, for a total of 43 parameters.
Since only the weight matrices define the transformation, the total is (4×5) + (5×3) = 35 parameters, with biases excluded from the count.
Adding up the neurons in each layer gives 4 + 5 + 3 = 12 total parameters for the network.
Multiplying the combined neuron counts of adjacent layers, (4+5)×(5+3) = 72 total parameters.
Answer: A. Layer 1 contributes (4×5 weights) + 5 biases = 25 parameters, and Layer 2 contributes (5×3 weights) + 3 biases = 18 parameters, for a total of 43 parameters.
ExplanationEach fully connected layer's weight matrix has shape (inputs_to_layer, neurons_in_layer), and it contributes one bias per neuron. For the input-to-hidden layer, W₁ has shape (4,5), giving 4×5 = 20 weights, plus 5 biases (one per hidden neuron), for 25 parameters. For the hidden-to-output layer, W₂ has shape (5,3), giving 5×3 = 15 weights, plus 3 biases (one per output neuron), for 18 parameters. Summing both layers: 25 + 18 = 43 trainable parameters. Counting only the weight matrices and dropping the bias terms undercounts to 35; treating the layer widths themselves as the parameters ignores the connections entirely and gives just 12; and multiplying the sums of adjacent layer widths, (4+5)×(5+3) = 72, conflates neuron counts with the actual weight-matrix dimensions and overcounts substantially. The reliable method is to compute (inputs×outputs + outputs) for every layer and add the results across the network.
Question 11 · Loss Functions · hard
For classification, cross-entropy loss is L = -∑_i y_i log(ŷ_i) where y_i ∈ {0,1} are true labels (one-hot) and ŷ_i ∈ (0,1) are predicted probabilities. For a 3-class problem with true label y=[1,0,0] and predicted probabilities ŷ=[0.7,0.2,0.1], what is the loss L?
L = -(1·log(0.7) + 0·log(0.2) + 0·log(0.1)) = -log(0.7) ≈ 0.357. Cross-entropy is ideal for classification because it penalizes confident wrong predictions severely.
L = 0.7 + 0.2 + 0.1 = 1.0 (sum of predictions, unrelated to true labels).
Cross-entropy cannot be used for multi-class problems; only binary classification.
Answer: A. L = -(1·log(0.7) + 0·log(0.2) + 0·log(0.1)) = -log(0.7) ≈ 0.357. Cross-entropy is ideal for classification because it penalizes confident wrong predictions severely.
ExplanationFirst, cross-entropy calculation: L = -∑y_i log(ŷ_i) = -(1·log(0.7) + 0·log(0.2) + 0·log(0.1)) = -log(0.7) ≈ 0.357. Key property: only the true class (y₀=1) contributes to loss. Then, cross-entropy penalizes confident wrong predictions severely: if ŷ_true→0, loss→∞. It's preferred over MSE for classification because it provides stronger gradients when predictions are wrong, accelerating convergence through the softmax output layer.
Question 12 · Backpropagation · hard
In backpropagation, the chain rule computes gradients: ∂L/∂w = (∂L/∂z) · (∂z/∂w), where z is the pre-activation and w is the weight. For a simple network: z = w·x + b, then a = ReLU(z), compute ∂L/∂w?
The gradient works out to ∂L/∂w = (∂L/∂a) · x, since the ReLU activation is treated as contributing a derivative of exactly 1 for every value of z, so the piecewise gradient rule is skipped entirely.
Using the sigmoid derivative formula a·(1−a) here gives ∂L/∂w = (∂L/∂a) · a·(1−a) · x, applying that formula to the ReLU activation instead of its actual step-function derivative.
∂z/∂w = x (from z = w·x + b). ∂a/∂z = 1 if z>0, 0 if z<0 (ReLU gradient). Therefore ∂L/∂w = (∂L/∂a) · (∂a/∂z) · (∂z/∂w) = (∂L/∂a) · x when z>0, and 0 when z<0.
∂L/∂w = (∂L/∂a) / x. Division is used instead of multiplication in backpropagation.
Answer: C. ∂z/∂w = x (from z = w·x + b). ∂a/∂z = 1 if z>0, 0 if z<0 (ReLU gradient). Therefore ∂L/∂w = (∂L/∂a) · (∂a/∂z) · (∂z/∂w) = (∂L/∂a) · x when z>0, and 0 when z<0.
ExplanationBackpropagation applies the chain rule from calculus layer by layer. Forward pass computes: z = w·x + b (pre-activation), then a = ReLU(z) = max(0, z). Backward pass: ∂z/∂w = x (the input to this layer), ∂a/∂z = 1 if z > 0 else 0 (ReLU derivative is a step function — this is the "dying ReLU" problem since gradients vanish for negative z). Full gradient: ∂L/∂w = (∂L/∂a) · (∂a/∂z) · (∂z/∂w) = upstream_gradient × ReLU_derivative × input. The vanishing gradient for z < 0 means neurons that output 0 receive no gradient updates, permanently "dying" — motivating alternatives like Leaky ReLU and GELU.
Question 13 · Evaluation Metrics - Precision/Recall · hard
A spam filter is tested on 200 emails, of which 50 are actually spam and 150 are legitimate. The filter flags 40 emails as spam, and of those flagged, 30 are genuinely spam. Using precision = TP/(TP+FP), recall = TP/(TP+FN), and F1 = 2·(precision·recall)/(precision+recall), what are the filter's precision, recall, and F1 score?
Swapping the ratios gives precision = 30/50 = 0.60 and recall = 30/40 = 0.75, meaning precision measures how much real spam was caught and recall measures how many flagged emails were correct.
Precision = 30/40 = 0.75, recall = 30/50 = 0.60, and F1 = 2·(0.75·0.60)/(0.75+0.60) = 0.90/1.35 ≈ 0.67, the harmonic mean pulled toward the lower of the two values.
Averaging directly, F1 equals the simple mean of the two rates, (0.75 + 0.60)/2 = 0.675, since precision is 0.75 and recall is 0.60 for this filter.
Dividing each correct catch by the full email count yields precision = recall = 30/200 = 0.15, and thus F1 = 0.15 for this spam filter.
Answer: B. Precision = 30/40 = 0.75, recall = 30/50 = 0.60, and F1 = 2·(0.75·0.60)/(0.75+0.60) = 0.90/1.35 ≈ 0.67, the harmonic mean pulled toward the lower of the two values.
ExplanationOf the 40 emails flagged as spam, 30 are true positives and 10 are false positives, so precision = TP/(TP+FP) = 30/40 = 0.75 — three-quarters of what the filter flagged actually was spam. Of the 50 actual spam emails, 30 were caught and 20 were missed (false negatives), so recall = TP/(TP+FN) = 30/50 = 0.60 — the filter caught 60% of real spam. F1 is the harmonic mean, not a simple average: F1 = 2·(0.75·0.60)/(0.75+0.60) = 0.90/1.35 ≈ 0.67. This sits closer to the lower value (0.60) than the arithmetic mean (0.675) would, because the harmonic mean penalizes imbalance between precision and recall more heavily than a plain average does. A claim that swaps which ratio is precision and which is recall gets the individual values backwards even though the flagged/caught counts stay the same. Treating F1 as a plain average conflates it with the arithmetic mean and understates how sensitive F1 actually is to the smaller of the two metrics. Dividing by the total sample count of 200 instead of by (TP+FP) or (TP+FN) confuses these ratios with an accuracy-style measure and produces values far too low to be meaningful.
Question 14 · Evaluation Metrics - Specificity · hard
In a classification model with confusion matrix: TP=45, FP=5, TN=90, FN=10, calculate specificity = TN/(TN+FP) and sensitivity = TP/(TP+FN). Explain the difference between these metrics and when each?
ExplanationFirst, confusion matrix: TN=90 (correctly predicted negative), FP=5 (false positives), TP=45 (true positives), FN=10 (false negatives). Specificity = TN/(TN+FP) = 90/95 ≈ 0.947 = 94.7% — the model correctly identifies 94.7% of actual negatives. Then, sensitivity (recall) = TP/(TP+FN) = 45/55 ≈ 0.818 = 81.8% — the model correctly identifies 81.8% of actual positives. The gap (specificity > sensitivity) means the model is better at ruling out negatives than detecting positives. Sensitivity matters most when missing a true positive is costly, such as disease screening, while specificity matters most when false alarms are costly, such as flagging a legitimate transaction as fraud.
Question 15 · Gradient Descent and Optimization · hard
Given a convex loss function L(θ) = θ² - 6θ + 10 with gradient ∂L/∂θ = 2θ - 6, consider:
def gradient_descent_momentum(theta_init=5, learning_rate=0.00100, momentum_coeff=0.500, iterations=100):
theta = theta_init
velocity = 0
for _ in range(iterations):
grad = 2 * theta - 6
velocity = momentum_coeff * velocity + grad
theta = theta - learning_rate * velocity
return theta
If you optimize with learning_rate=0.00100 and momentum=0.500, what would be the convergence behavior?
What happens to parameter θ as iterations increase toward 100?
θ remains at initial value 5 because gradient descent cannot modify parameters in this loss function
θ diverges to infinity because learning_rate=0.00100 and momentum=0.500 exceed stability thresholds
θ oscillates indefinitely between initial value 5 and 0 without converging to any fixed point
θ converges toward optimal θ*=3.0, but the error decays at an effective per-step rate near 0.996 (the dominant eigenvalue of the momentum recurrence), only marginally faster than plain gradient descent's 0.998 decay rate at this learning rate
Answer: D. θ converges toward optimal θ*=3.0, but the error decays at an effective per-step rate near 0.996 (the dominant eigenvalue of the momentum recurrence), only marginally faster than plain gradient descent's 0.998 decay rate at this learning rate
ExplanationSubstituting e = θ - 3 (the distance from the optimum θ*=3) turns the momentum update into a 2x2 linear recurrence for (e, v) with matrix [[1-2·lr, -lr·momentum_coeff], [2, momentum_coeff]]. Plugging in lr=0.00100 and momentum_coeff=0.500 gives [[0.998, -0.0005], [2, 0.5]], whose characteristic equation λ² - 1.498λ + 0.500 = 0 has roots λ≈0.996 and λ≈0.502. It is the larger root, λ≈0.996, that governs long-run convergence speed, not momentum_coeff itself. Starting from θ=5 (initial error 2), simulating the exact loop shows the error θ-3 is still about 1.343 after 100 iterations — nowhere near zero. Running the same 100 iterations without momentum (plain gradient descent, whose single decay rate is 1-2·lr=0.998) leaves an error of about 1.637. So momentum does converge and does help slightly, but the two rates (0.996 vs 0.998) are close, giving only a modest edge rather than the dramatic exponential speed-up that a 0.500 per-step decay would produce.
Question 16 · Linear Algebra · hard
Given matrix A = [[2, 1], [1, 2]] with determinant det(A) = 3, consider:
def matrix_inverse(A):
det = A[0, 0] * A[1, 1] - A[0, 1] * A[1, 0]
return (1/det) * np.array([[A[1, 1], -A[0, 1]], [-A[1, 0], A[0, 0]]])
If you compute A⁻¹, what would be the value at position [0,0]?
2 because A[1,1]=2, mistakenly skipping the division by det(A) that the adjugate formula requires before returning the [0,0] entry
1/3 because det(A)=3, mistakenly using only the reciprocal of the determinant without multiplying it by the adjugate entry A[1,1]
2/3 ≈ 0.667 because det(A)=3 and A[1,1]=2, so inverse [0,0] = 2/3
2/5 because det(A) is miscalculated as A[0,0]*A[1,1] + A[0,1]*A[1,0] = 4+1 = 5 instead of subtracting, giving inverse [0,0] = 2/5
Answer: C. 2/3 ≈ 0.667 because det(A)=3 and A[1,1]=2, so inverse [0,0] = 2/3
Given matrix A = [[3, 1], [1, 3]] with characteristic polynomial det(A - λI) = 0, consider:
def compute_eigenvalues(A):
eigenvalues = np.linalg.eigvals(A)
return np.sort(eigenvalues)[::-1]
If (3-λ)² - 1 = 0, what would be the eigenvalues?
λ₁ = 2, λ₂ = 2 because taking the square root of (3-λ)² = 1 gives only 3-λ = 1, so λ = 2 (repeated), ignoring the negative root
λ₁ = 3, λ₂ = 3 because diagonal elements equal eigenvalues
λ₁ = 4, λ₂ = 2 because (3-λ)² = 1 gives 3-λ = ±1, yielding λ = 2 or 4
λ is undefined because characteristic polynomial has no real roots
Answer: C. λ₁ = 4, λ₂ = 2 because (3-λ)² = 1 gives 3-λ = ±1, yielding λ = 2 or 4
ExplanationFirst, (3-λ)² - 1 = 0 expands to (3-λ)² = 1. Taking square roots on both sides: 3-λ = ±1, so λ = 3∓1, giving λ = 2 or λ = 4. Verification: trace(A) = λ₁ + λ₂ = 4 + 2 = 6, which matches 3 + 3 = 6 from the diagonal of A. Determinant check: λ₁ × λ₂ = 4 × 2 = 8, which matches det(A) = (3×3) - (1×1) = 9 - 1 = 8. These eigenvalues represent the stretching factors along the eigenvector directions of A, a concept central to diagonalization and techniques such as PCA.
Question 18 · Linear Algebra · hard
Consider the following NumPy function used to compute vector norms:
```python
import numpy as np
def vector_norms(x):
l1 = np.sum(np.abs(x))
l2 = np.sqrt(np.sum(x**2))
linf = np.max(np.abs(x))
return l1, l2, linf
x = np.array([3, -4, 12])
l1, l2, linf = vector_norms(x)
```
Working out the exact values returned by `vector_norms(x)`, what is the correct relationship among ||x||₁, ||x||₂, and ||x||∞?
Computing directly: ||x||₁ = 3+4+12 = 19, ||x||₂ = √(9+16+144) = √169 = 13, and ||x||∞ = max(3,4,12) = 12, so the strict order is ||x||₁ > ||x||₂ > ||x||∞.
Summing the signed entries directly gives ||x||₁ = 3 + (-4) + 12 = 11, while ||x||₂ = 13 and ||x||∞ = 12 stay unchanged, producing the order ||x||₂ > ||x||∞ > ||x||₁.
Skipping the square root step yields ||x||₂ = 9+16+144 = 169, with ||x||₁ = 19 and ||x||∞ = 12 unaffected, making ||x||₂ the overwhelmingly largest of the three norms.
Taking the smallest-magnitude entry instead of the largest gives ||x||∞ = 3, while ||x||₁ = 19 and ||x||₂ = 13 stay correct, so the order becomes ||x||₁ > ||x||₂ > ||x||∞ with an unusually small infinity norm.
Answer: A. Computing directly: ||x||₁ = 3+4+12 = 19, ||x||₂ = √(9+16+144) = √169 = 13, and ||x||∞ = max(3,4,12) = 12, so the strict order is ||x||₁ > ||x||₂ > ||x||∞.
ExplanationTaking absolute values first, |3| = 3, |-4| = 4, |12| = 12, so ||x||₁ = 3+4+12 = 19. Squaring each entry gives 9, 16, and 144, which sum to 169 — a perfect square because (3, 4, 12, 13) is a Pythagorean quadruple — so ||x||₂ = √169 = 13. The largest magnitude among the entries is 12, so ||x||∞ = 12 (the max, not the min, of the absolute values). Comparing the three results, 19 > 13 > 12, giving the strict order ||x||₁ > ||x||₂ > ||x||∞. This ordering is no coincidence of this particular vector: for any x with more than one nonzero entry, ||x||₁ ≥ ||x||₂ ≥ ||x||∞ always holds, with equality only in the degenerate case of a single nonzero entry. Summing absolute values (L1) can only be at least as large as the Euclidean length (L2), since squaring and taking a square root never inflates a sum of nonnegative terms beyond their plain sum, and the Euclidean length can never be smaller than the single largest magnitude present in the vector (L∞).
Question 19 · Probability and Statistics · hard
Given events A (rain) and B (cloudy) with P(A) = 0.3, P(B) = 0.7, and P(A|B) = 0.4, what is P(A∩B), the joint probability that it is both rainy and cloudy?
P(A∩B) = 0 because A and B are mutually exclusive by definition
P(A∩B) = 0.3 exactly, matching marginal P(A) because intersection equals single event
P(A∩B) = 1.0 because both probabilities sum beyond 0.5
P(A∩B) = 0.4 × 0.7 = 0.28 (28%), the conditional probability multiplied by the marginal probability of B
Answer: D. P(A∩B) = 0.4 × 0.7 = 0.28 (28%), the conditional probability multiplied by the marginal probability of B
ExplanationJoint probability from a conditional probability is computed as P(A∩B) = P(A|B) × P(B) = 0.4 × 0.7 = 0.28, so 28% of days are both rainy and cloudy. This is consistent with the definition of conditional probability: rearranging gives P(A|B) = P(A∩B)/P(B) = 0.28/0.7 = 0.4, which matches the value given in the problem. From the same joint probability we can also find P(B|A) = P(A∩B)/P(A) = 0.28/0.3 ≈ 0.933, meaning about 93.3% of rainy days are also cloudy, and Bayes' theorem confirms consistency: P(A|B) = P(B|A) × P(A)/P(B) = 0.933 × 0.3/0.7 ≈ 0.4. A common mistake is to simply multiply P(A) by P(B) (0.3 × 0.7 = 0.21) as if rain and cloudiness were independent events, ignoring the conditional relationship stated in the problem.
Question 20 · Probability and Statistics · hard
Given X ~ N(μ=100, σ²=100) with σ=10, consider:
def cdf_standard_normal(z):
from scipy.stats import norm
return norm.cdf(z)
If P(X < 120) is computed as Φ((120-100)/10) = Φ(2), what would be the probability?
P(X < 120) = 0.5 exactly, because 120 exceeds mean by 20 which is unrelated to probability
P(X < 120) ≈ 0.5793 (Φ(0.2)), from mistakenly dividing 20 by σ²=100 instead of by σ=10 to standardize
P(X < 120) = Φ(2) ≈ 0.9772 (97.72%), where Φ is standard normal CDF at z=2
P(X < 120) is undefined without explicit distribution table lookup
Answer: C. P(X < 120) = Φ(2) ≈ 0.9772 (97.72%), where Φ is standard normal CDF at z=2
ExplanationStandardizing converts X to a standard normal variable: Z = (X - μ)/σ = (120 - 100)/10 = 2. The standard normal CDF at z=2 is Φ(2) ≈ 0.9772, so P(X < 120) = P(Z < 2) ≈ 0.9772, or about 97.72%. The complement P(X ≥ 120) = 1 - 0.9772 = 0.0228, confirming the two probabilities sum to 1.