Suppose you are helping build the scouting dashboard for an IPL franchise ahead of the next auction. The analytics team hands you a spreadsheet with one row per shortlisted batter and twelve columns: batting average, strike rate, boundary percentage, dot-ball percentage, runs scored against pace, runs scored against spin, powerplay strike rate, death-overs strike rate, and a few more. The head coach does not want twelve numbers per player. He wants one chart, two axes, one dot per batter, that the whole coaching staff can glance at during a bidding war and instantly see who is a steady anchor, who is a boundary-hitting finisher, and who sits somewhere in between.
You have a problem. A chart only has two axes, and you have twelve columns. Pick any two columns, say average and strike rate, and you have thrown away ten columns' worth of information about boundary-hitting, spin-versus-pace form, and death-overs temperament. Players who looked completely different on those discarded columns will now look identical on your chart, even if they play entirely different roles on the field.
Dimensionality reduction is the general name for techniques that solve this kind of problem, and Principal Component Analysis, or PCA, is the technique reached for most often. PCA does not simply pick two of your twelve original columns and discard the rest. It builds two brand-new columns, each one a carefully weighted blend of all twelve original stats, engineered so that the first new column captures as much of the player-to-player variation as any single number possibly could, and the second captures as much of whatever is left over. If your original twelve stats were even loosely correlated with each other, two or three of these engineered columns can often preserve the large majority of the real differences between players. Real cricket statistics almost always are correlated this way, because "aggressive scoring" shows up simultaneously in strike rate, boundary percentage, and death-overs numbers. You compress twelve dimensions down to two, and the picture you get to draw is still, in a precise mathematical sense, honest.
To see how that is possible, we need to answer one question with complete precision: when you are only allowed to keep a handful of new columns, what exactly should go into them so that you throw away as little truth as possible? The answer starts with a statistical idea you already know, variance, and ends with an object from linear algebra you have already met: the eigenvector.
What Does "Dimension" Really Mean Here?
Every column in a dataset defines one axis. Track only a batter's strike rate, and each player becomes a single point on a number line, a one-dimensional space. Add batting average, and each player becomes a point on a flat plane, positioned at coordinates (average, strike rate), a two-dimensional space you can sketch on graph paper. Add boundary percentage as a third column, and every player becomes a point floating inside a cube: three dimensions, still drawable with a bit of shading and effort. Add a fourth statistic and something breaks. There is no picture for a four-dimensional space. You cannot sketch it, and your brain cannot visualize it directly.
The mathematics, however, does not care. A batter described by twelve statistics is simply a point with twelve coordinates, a vector in a twelve-dimensional space, exactly as legitimate a mathematical object as a point in two or three dimensions, just impossible to draw directly. Matrices and vectors do not require a picture in order to exist or to be manipulated correctly.
High dimensionality is not merely inconvenient for chart-making. It is expensive: more columns mean more storage and more computation for every downstream algorithm, and it brings on a well-known statistical headache that computer scientists call the curse of dimensionality. As the number of dimensions grows, data points that would look close together in two or three dimensions spread out and become sparse relative to the sheer size of the high-dimensional space they now occupy. Algorithms that depend on "closeness" (nearest-neighbour search, clustering, many recommendation engines) start to struggle, because in very high dimensions almost every pair of points ends up looking roughly equally far apart. Twelve columns is not an extreme case, but the same mathematics scales to datasets with hundreds or thousands of columns: pixel values in an image, word-embedding vectors, gene-expression profiles.
What makes compression possible at all is that real-world columns are rarely independent of one another. A batter with a high strike rate typically also has a high boundary percentage; both numbers are really reporting on the same underlying trait: how aggressively this player scores runs. Whenever two columns move together like this, they are telling an overlapping story, and PCA is built specifically to find and exploit that overlap.
Variance Is How We Measure "Information"
To compress data without losing what matters, you first need a precise definition of what matters. PCA's answer is variance, the average squared distance of each data point from the mean. A column with high variance is a column where players differ a lot from one another; a column with variance near zero is a column where almost every player has nearly the same value, which means that column is doing very little work to distinguish one player from another.
Picture a thirteenth column added to the scouting sheet: "matches played at home venue X", which happens to equal 4 for every single shortlisted player, purely because of how the shortlist was built. That column has zero variance. It takes up space in the spreadsheet, but it carries no information that helps tell players apart, because it never changes. Contrast that with strike rate, which might swing from under 110 for a patient anchor to over 170 for an explosive finisher. That spread, that variance, is exactly what a coach is trying to see on the chart. A dimensionality-reduction technique worth using should hunt for the directions in the data that carry the most variance and preserve those first, discarding only the near-constant, low-variance directions that were barely distinguishing anyone in the first place.
This is the entire strategy behind PCA in one sentence: find new axes, built as combinations of the original columns, ranked by how much variance they capture, and keep only the highest-ranked ones. Everything that follows in this chapter is about making that idea mathematically exact.
Centering the Data and Building the Covariance Matrix
The first mechanical step is always the same: subtract the mean of each column from every value in that column. This is called mean-centering, and it shifts the entire cloud of data points so that it is centred on the origin, with an average value of zero in every column. Centering matters because PCA is about to search for directions of maximum spread, and spread should be measured relative to the centre of the data, not relative to an arbitrary point like zero on the original, uncentred scale. Skip this step, and PCA's first "principal component" ends up pointing toward wherever the bulk of the data happens to sit relative to the origin, rather than describing how the data actually varies.
Once the data is centred, PCA needs a way to describe not just how much each column varies on its own, but how columns vary together. That is what covariance measures. For two centred columns x and y across n data points, covariance is defined as:
Cov(x, y) = (1/n) × sum of (x_i × y_i) over all points i
Because the data is already centred, this is simply the average of each point's x-value multiplied by its y-value. If x and y tend to be positive together and negative together (batters who score fast also hit lots of boundaries), most of the products x_i·y_i are positive, and the covariance is positive. If one column tends to be high exactly when the other is low, covariance is negative. If the two columns have no consistent relationship, the positive and negative products cancel out and the covariance sits near zero. Applying this same formula with x and y set to the same column, Cov(x, x), gives back the ordinary variance of that column.
For a dataset with d columns, you compute the covariance between every pair of columns, including each column with itself, and arrange the results in a d × d grid called the covariance matrix. The diagonal entries are the plain variances of each column; the off-diagonal entries are the pairwise covariances. This single matrix is a complete summary of how every column relates to every other column, and it is the only ingredient PCA needs from your data.
From Covariance to Principal Components
Recall from earlier in this course that a nonzero vector v is an eigenvector of a square matrix A if multiplying it by A only stretches or shrinks v without turning it, that is, Av = λv for some scalar λ, called the corresponding eigenvalue. Covariance matrices have two properties that make them especially well behaved: they are always symmetric (Cov(x, y) equals Cov(y, x) by definition), and a d × d symmetric matrix is guaranteed, by a result called the spectral theorem, to have exactly d real eigenvalues and d eigenvectors that are mutually perpendicular, or orthogonal, to one another.
Here is the connection that makes PCA work: the eigenvectors of the covariance matrix are exactly the new axes we were looking for, and their eigenvalues are exactly the amount of variance captured along each axis. This is not a coincidence, and it can be shown directly. Pick any candidate direction, a unit vector w, and project every centred data point x_i onto it by computing the dot product x_i · w. This gives one score per data point, and the variance of these scores, averaged over all n points, is (1/n) × sum of (x_i · w)², which expands to wTCw, where C is the covariance matrix. Because C is symmetric, its d orthonormal eigenvectors v₁, v₂, …, v_d, with eigenvalues λ₁ ≥ λ₂ ≥ … ≥ λ_d, form a complete basis, so any unit vector w can be written as a mix w = c₁v₁ + c₂v₂ + … + c_dv_d, where the weights satisfy c₁² + c₂² + … + c_d² = 1. Substituting this into wTCw and using the fact that the eigenvectors are orthonormal collapses the expression to c₁²λ₁ + c₂²λ₂ + … + c_d²λ_d, a weighted average of the eigenvalues. To make a weighted average of numbers as large as possible, with weights that must sum to 1, you put all the weight on the largest number. So the maximum possible variance, λ₁, is achieved exactly when w = v₁, the eigenvector belonging to the largest eigenvalue.
That eigenvector is called the first principal component, conventionally written PC1. It is the single direction along which the data spreads out the most. The eigenvector with the second-largest eigenvalue, PC2, is the direction of maximum remaining variance among all directions perpendicular to PC1, and so on down the list. Because the eigenvalues add up to the total variance in the data (the sum of the diagonal entries of C, a quantity called the trace of the matrix), each eigenvalue's share of that total tells you exactly what fraction of the original information survives if you keep that component. This fraction, λ_i divided by the sum of all the eigenvalues, is called the explained variance ratio of component i, and it is the number you will check, every time you run PCA, to know how much you are really keeping and how much you are throwing away.
A Complete Worked Example: Four Players, Two Skills
Numbers make this concrete. Suppose an analytics team has scored four hypothetical players, P, Q, R, and S, on two standardized 0-5 skill scales: an "aggression score" (capturing strike rate and boundary frequency) and a "finishing score" (capturing performance in the closing overs). These are illustrative figures built for this walkthrough, not real statistics from actual players. The raw data is:
- Player P: aggression = 1, finishing = 1
- Player Q: aggression = 2, finishing = 3
- Player R: aggression = 3, finishing = 2
- Player S: aggression = 4, finishing = 4
Step 1, centre the data. The mean aggression score is (1+2+3+4)/4 = 2.5, and the mean finishing score is (1+3+2+4)/4 = 2.5. Subtracting these means from every point gives the centred coordinates: P′ = (−1.5, −1.5), Q′ = (−0.5, 0.5), R′ = (0.5, −0.5), S′ = (1.5, 1.5).
Step 2, build the covariance matrix. Using the centred values, Var(aggression) = [(−1.5)² + (−0.5)² + 0.5² + 1.5²] / 4 = 5/4 = 1.25, and by identical arithmetic Var(finishing) = 1.25 too. The covariance is Cov(aggression, finishing) = [(−1.5)(−1.5) + (−0.5)(0.5) + (0.5)(−0.5) + (1.5)(1.5)] / 4 = [2.25 − 0.25 − 0.25 + 2.25] / 4 = 4/4 = 1.0. The covariance matrix is:
C = [[1.25, 1.00], [1.00, 1.25]]
The positive off-diagonal value confirms what the raw numbers already suggested: aggression and finishing tend to rise and fall together across these four players.
Step 3, find the eigenvalues and eigenvectors. For a 2×2 matrix of the form [[a, b], [b, a]], the eigenvalues have a convenient shortcut: they are a + b and a − b. Here that gives λ₁ = 1.25 + 1.00 = 2.25 and λ₂ = 1.25 − 1.00 = 0.25. As a sanity check, λ₁ + λ₂ = 2.5, which matches the trace of C (1.25 + 1.25 = 2.5) exactly, as it must, since the eigenvalues of any square matrix always sum to its trace. Solving (C − λI)v = 0 for each eigenvalue gives the eigenvectors: for λ₁ = 2.25, the eigenvector is proportional to (1, 1), which normalizes to (0.7071, 0.7071); for λ₂ = 0.25, the eigenvector is proportional to (−1, 1), which normalizes to (−0.7071, 0.7071).
Step 4, read the explained variance ratio. PC1 explains 2.25 / 2.5 = 0.9, or 90 percent of the total variance in the data. PC2 explains the remaining 0.25 / 2.5 = 0.1, or 10 percent. A single new axis, PC1, is already capturing nine-tenths of everything that distinguishes these four players.
Step 5, project the data and interpret the result. To compress each player down to one number, take the dot product of their centred coordinates with the PC1 eigenvector (0.7071, 0.7071):
- P: (−1.5)(0.7071) + (−1.5)(0.7071) = −2.1213
- Q: (−0.5)(0.7071) + (0.5)(0.7071) = 0
- R: (0.5)(0.7071) + (−0.5)(0.7071) = 0
- S: (1.5)(0.7071) + (1.5)(0.7071) = 2.1213
This single PC1 score is a genuinely interpretable "overall attacking output" index, because its two weights are equal and positive: it rewards a player for being high on both aggression and finishing at once. Notice what happened to Q and R: both collapse to a PC1 score of exactly 0. That makes sense, because both have coordinates that average out to the grand mean of 2.5. Q's (2, 3) and R's (3, 2) both sum to 5. PC1 alone cannot tell Q and R apart at all.
That is precisely what PC2 is for. Projecting onto the second eigenvector (−0.7071, 0.7071) gives P: 0, Q: (−0.5)(−0.7071) + (0.5)(0.7071) = 0.7071, R: (0.5)(−0.7071) + (−0.5)(0.7071) = −0.7071, S: 0. PC2 behaves like a "style balance" axis, because its weights point in the (−1, 1) direction, so it is large and positive exactly when finishing exceeds aggression, and large and negative when aggression exceeds finishing. Q, whose finishing score of 3 is higher than its aggression score of 2, gets a positive PC2 score; R, the mirror image, gets a negative one. P and S land at exactly zero on PC2 because each has identical aggression and finishing scores: they are perfectly balanced, just at different overall levels.
Keeping PC1 alone and discarding PC2 amounts to reconstructing every player using only their overall-level score. P and S, whose true PC2 score was already zero, are reconstructed perfectly: back to (1, 1) and (4, 4) exactly. Q and R are the ones who lose information; both get reconstructed at the same point, (2.5, 2.5), the grand mean, because that is the best estimate of their location using PC1 alone. The average squared reconstruction error across all four players works out to exactly 0.25, which is precisely λ₂, the eigenvalue that was discarded. This is not a coincidence: the variance you throw away by dropping a component is exactly the reconstruction error that dropping it costs you, measured in squared distance. It is the mathematical meaning of compressing without losing information, made completely literal.
The Same Computation, in Code
Every one of those hand calculations is exactly what a PCA library does internally, just faster, and for datasets with far more than two columns. Here is the walkthrough reproduced in NumPy, matching every number computed by hand above:
import numpy as np
# Two standardized skill scores for players P, Q, R, S
data = np.array([
[1.0, 1.0], # Player P
[2.0, 3.0], # Player Q
[3.0, 2.0], # Player R
[4.0, 4.0], # Player S
])
# Step 1: mean-center each column
mean = data.mean(axis=0)
centered = data - mean
print(mean) # [2.5 2.5]
# Step 2: covariance matrix (ddof=0 matches the by-hand /n calculation)
cov_matrix = np.cov(centered, rowvar=False, ddof=0)
print(cov_matrix) # [[1.25 1. ]
# [1. 1.25]]
# Step 3: eigenvalues/eigenvectors of the symmetric covariance matrix
eigenvalues, eigenvectors = np.linalg.eigh(cov_matrix)
order = np.argsort(eigenvalues)[::-1] # eigh sorts ascending; flip it
eigenvalues = eigenvalues[order]
eigenvectors = eigenvectors[:, order]
print(eigenvalues) # [2.25 0.25]
# Step 4: explained variance ratio
print(eigenvalues / eigenvalues.sum()) # [0.9 0.1]
# Step 5: project onto PC1 to compress each player to one number
pc1 = eigenvectors[:, 0]
scores_1d = centered @ pc1
print(scores_1d) # [-2.1213 0. 0. 2.1213]
np.linalg.eigh, rather than the more general np.linalg.eig, is the right tool whenever the matrix is symmetric, which every covariance matrix is: it is faster, and it guarantees real eigenvalues returned in a predictable order. Eigenvectors carry an inherent sign ambiguity, since (0.7071, 0.7071) and (−0.7071, −0.7071) point along the exact same line and are equally valid answers, so do not be surprised if a library flips a sign compared with a hand calculation. The direction, and every distance and variance computed from it, stays unchanged either way.
In practice, nobody assembles the covariance matrix and calls eigh by hand every time; they use a library that has already been tested and optimized. scikit-learn's implementation reaches the identical answer:
from sklearn.decomposition import PCA
pca = PCA(n_components=1)
scores_1d = pca.fit_transform(data)
print(scores_1d.ravel()) # [-2.1213 0. 0. 2.1213]
print(pca.explained_variance_ratio_) # [0.9]
print(pca.components_) # [[0.7071 0.7071]]
You may see Q and R print as tiny values like -8e-17 instead of a clean 0; that is ordinary floating-point rounding noise, not a real signal, far smaller than any measurement your data could actually resolve. Internally, scikit-learn does not even build the covariance matrix. It runs a related decomposition called the Singular Value Decomposition directly on the centred data matrix, which is more numerically stable on large datasets, but it is mathematically guaranteed to land on the same eigenvectors and the same explained variance ratios.
How Many Components Should You Keep?
With only two original columns the choice is trivial, but real scouting data has a dozen or more. Two checks make the decision in practice. The first is the explained variance ratio itself: add up the ratios for your top k components and stop once the cumulative total crosses a threshold you are comfortable with; 90 or 95 percent is a common target for exploratory analysis. The second is a scree plot: sort the eigenvalues from largest to smallest and plot them. Eigenvalues from genuine, informative directions tend to be much larger than eigenvalues from directions that are essentially noise, so the plot typically drops steeply at first and then flattens into a long, low tail: an "elbow." Components before the elbow are usually worth keeping; the flat tail after it is usually safe to discard.
One caveat is easy to miss and important to get right: PCA is sensitive to the scale of your columns, because variance is measured in whatever units the column happens to be in. Batting average might range from about 15 to 55, while dot-ball percentage ranges from 0 to 1. Fed directly into PCA, the average column's variance will be numerically enormous purely because of its units, and PC1 will end up pointing almost entirely along the average axis regardless of what is actually most informative. The standard fix is standardization: rescale every column to have mean 0 and standard deviation 1 before computing the covariance matrix, which is equivalent to running PCA on the correlation matrix rather than the raw covariance matrix. Whenever your columns are not already measured in comparable units, standardize first; skipping this step is one of the most common real-world PCA mistakes.
Where This Shows Up Beyond Cricket
PCA is over a century old: the statistician Karl Pearson described the method in 1901, and the statistician Harold Hotelling gave it its modern computational form and name in 1933. It has stayed central to applied mathematics ever since, because the same underlying problem, many correlated measurements but few genuinely informative directions, keeps showing up everywhere. In computer vision, the 1991 "eigenfaces" method developed by Matthew Turk and Alex Pentland showed that a large collection of face images, each one really a vector of thousands of pixel-brightness values, could be reconstructed almost recognizably using only a few dozen principal components, because most faces share the same broad structure, and only a handful of directions of variation, such as face width or lighting angle, carry most of the difference between individuals. In machine learning pipelines generally, PCA is a standard preprocessing step: reducing hundreds of correlated input features to a few dozen uncorrelated ones before training a model, which trains faster, resists overfitting better, and is easier to visualize for debugging. In finance, analysts run PCA on the covariance matrix of stock returns to identify a small number of dominant "risk factors" driving an entire market of hundreds of individual stocks. The mathematics never changes; only the meaning of the columns does.
Back to the Auction Table
Return to the coaching staff waiting on their two-axis chart. With PCA run properly on twelve standardized, mean-centred statistics and decomposed into eigenvectors and eigenvalues, the analytics team hands over a plot where every shortlisted batter is a single dot, positioned using PC1 and PC2 instead of any two hand-picked original stats. If those two components together explain, say, 88 percent of the total variance across the shortlist, the coach is looking at a chart that is 88 percent as informative as the full twelve-column spreadsheet, compressed into something a human eye can actually read in one glance. Batters bunched together on the chart really do have similar all-round profiles across every one of the original twelve measurements, not just the two an analyst happened to like best; batters far apart really do differ substantially. Clusters that were invisible when spread across twelve separate columns (a group of aggressive finishers, a group of patient anchors, an outlier who does not fit either mould) become visible as soon as the data is projected onto the two directions that were mathematically guaranteed to preserve the most truth.
That is the real promise of dimensionality reduction, and of PCA specifically: not that information gets thrown away for convenience, but that the technique is unusually precise about which information matters least, so that is the only part it ever leaves behind.
Think About It
Think about this: How would you explain dimensionality reduction with pca: compressing data without losing information to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.