Causal Inference: Cause vs Correlation

Two treatments, one paradox

In 1986 the British Medical Journal published a study by Charig, Webb, Payne and Wickham comparing two ways of removing kidney stones: open surgery, the older invasive procedure, and percutaneous nephrolithotomy (PCNL), a newer keyhole technique. Pooled across all 700 patients in the two arms, PCNL looked like the clear winner: 289 of 350 patients treated with PCNL recovered successfully (82.6%), against 273 of 350 for open surgery (78.0%). A hospital administrator reading only that headline number would switch every patient to PCNL.

Julious and Mullee revisited the same data in 1994 and split it by stone size, because surgeons don't choose a technique at random — they choose based on how hard the case looks. For small stones, open surgery succeeded in 81 of 87 cases (93.1%) versus PCNL's 234 of 270 (86.7%). For large stones, open surgery succeeded in 192 of 263 cases (73.0%) versus PCNL's 55 of 80 (68.8%). Open surgery wins in both subgroups, individually, by a comfortable margin — yet it loses in the pooled total. Nothing in the arithmetic is wrong. This is Simpson's paradox, and it is the cleanest possible demonstration of why "the data shows X causes Y" is a different claim from "X and Y are correlated in the data."

The explanation is case mix. Surgeons reserved PCNL, the gentler technique, disproportionately for small, easy stones — cases that were likely to succeed under almost any treatment. Open surgery absorbed a disproportionate share of the large, difficult stones — cases with a lower success ceiling no matter what a surgeon does. Stone size decided which treatment a patient was likely to receive, and it independently decided how likely that patient was to recover. A variable that does both of those things at once is a confounder, and a confounder can flip the sign of a pooled comparison without any single number in the table being wrong.

What a confounder is, formally

Call the treatment variable T, the outcome Y, and a third variable X a confounder of the effect of T on Y if X causally influences both T and Y, and X is not itself a downstream consequence of either. Drawn as a causal diagram, that is two arrows: X → T and X → Y, sitting alongside the arrow you actually care about, T → Y. X points into both T and Y, forming what statisticians call a backdoor path from T to Y that runs through X rather than along the causal arrow you want to measure.

Raw correlation between T and Y cannot distinguish "signal traveling along T → Y" from "signal traveling along the backdoor path T ← X → Y." Both show up mixed together in Cor(T, Y). In the kidney-stone data, stone size (X) pushed easy cases toward PCNL (X → T) and pushed success probability up for easy cases regardless of technique (X → Y). The backdoor path alone was strong enough to overwhelm and reverse the true within-stratum advantage of open surgery once the two treatment groups were pooled with different case mixes.

"Controlling for X" — stratifying by stone size, as Julious and Mullee did, or the more general techniques below — means closing that backdoor path so that whatever association remains between T and Y can only be traveling along the arrow you actually want to read.

The arithmetic that produces the reversal

The stratified numbers above were computed exactly like this:

kidney = {
    "open_surgery": {"small": (81, 87), "large": (192, 263)},
    "pcnl":         {"small": (234, 270), "large": (55, 80)},
}

for treatment, groups in kidney.items():
    total_success = sum(s for s, n in groups.values())
    total_n = sum(n for s, n in groups.values())
    overall = 100 * total_success / total_n
    small_rate = 100 * groups["small"][0] / groups["small"][1]
    large_rate = 100 * groups["large"][0] / groups["large"][1]

Evaluating those expressions gives, for open surgery: overall 78.0%, small-stone rate 93.1%, large-stone rate 73.0%. For PCNL: overall 82.6%, small-stone rate 86.7%, large-stone rate 68.8%. A pooled rate is nothing more than a weighted average of the stratum rates, weighted by how many patients of each type ended up in that arm. Open surgery's overall 78.0% is a mix that is heavily weighted toward the harder large-stone cases (263 of 350 patients, 75% of its caseload); PCNL's overall 82.6% is heavily weighted toward the easier small-stone cases (270 of 350, 77% of its caseload). The pooled comparison is really comparing "open surgery, mostly on hard cases" against "PCNL, mostly on easy cases" — a comparison of case mixes as much as of techniques. Change the weights (the case mix) and the pooled ranking can flip even though every stratum rate stays fixed. That is the entire mechanism of Simpson's paradox: it is a weighted-average artifact, not a mysterious phenomenon.

Randomized controlled trials: solving confounding by design

A randomized controlled trial (RCT) sidesteps the whole problem at the assignment stage rather than fixing it afterward with arithmetic. If treatment is assigned by a coin flip — or, more precisely, by a random-number generator, independent of every patient characteristic — then in expectation the treatment and control arms have the same distribution of stone size, the same distribution of age, kidney function, and every other variable, including variables nobody measured or thought to measure. Randomization doesn't just balance the confounders you know about; it structurally severs every possible X → T arrow at once, for every X, known or unknown. That is the one guarantee no amount of clever statistics on observational data can fully replicate.

This is exactly why agricultural field trials — the randomized block designs used across ICAR and university farms to compare crop varieties or fertilizer regimes — randomly assign varieties to plots rather than letting farmers pick which variety goes on their best soil. A farmer who puts the new variety on the most fertile plot would reproduce the kidney-stone problem in a field: soil fertility becomes a confounder of variety choice and yield, and a plain yield comparison would be unreadable.

RCTs are also expensive, slow, and sometimes unethical or impossible: you cannot randomly assign a patient's kidney stone size, randomly assign someone to smoke, or randomly assign a company's marketing budget across years. Most of the data available to a data scientist — hospital records, sales logs, sensor feeds, historical experiments nobody designed as experiments — is observational: treatment was chosen by someone or something, for reasons, and those reasons are exactly the confounders that corrupted the kidney-stone comparison. Simple stratification like Julious and Mullee's works when there is one confounder with a handful of categories. Real observational data usually has several confounders, some continuous, and stratifying on all of them at once runs out of patients in each cell almost immediately. That is the gap propensity score matching is built to fill.

When you can't randomize: propensity score matching

The idea, due to Rosenbaum and Rubin (1983), is to compress every confounder into a single number: the propensity score e(X), the estimated probability that a unit with covariates X received the treatment. Two patients with the same propensity score looked, to whatever decided treatment, equally likely to be treated — so pairing a treated patient with a control patient who has a nearly identical propensity score approximates comparing two patients who were, in effect, coin-flipped between arms. Match every treated unit to its closest-propensity control, and the difference in outcomes averaged across matched pairs approximates the causal effect that a mini-randomized trial would have shown for that population.

Work through a small, fully synthetic example end to end. Six patients are given a new drug (T) or the standard drug (control); X is a baseline severity score from 0 (mild) to 10 (severe), measured before treatment. Suppose a logistic model, already fit on this population, gives propensity scores from logit(e) = −2 + 0.4X. And suppose — this is the ground truth, hidden from the analyst, constructed here purely so we can check whether our estimate recovers it — that recovery time in days truly follows Y = 4 + 0.5X − 2T: sicker patients take longer to recover regardless of drug, and the true causal effect of the new drug is a 2-day reduction.

import math

X = {"T1": 3, "T2": 6, "T3": 8, "C1": 2, "C2": 4, "C3": 7}
T = {"T1": 1, "T2": 1, "T3": 1, "C1": 0, "C2": 0, "C3": 0}

# propensity score from the (given) fitted model logit(e) = -2 + 0.4X
p = {k: 1 / (1 + math.exp(-(-2 + 0.4 * v))) for k, v in X.items()}

# true, unknown-to-the-analyst outcome, shown here only to check the estimate
Y = {k: 4 + 0.5 * X[k] - 2 * T[k] for k in X}

treated, control = ["T1", "T2", "T3"], ["C1", "C2", "C3"]
naive_effect = sum(Y[t] for t in treated) / 3 - sum(Y[c] for c in control) / 3

Evaluating p for each patient gives T1 = 0.3100, T2 = 0.5987, T3 = 0.7685, C1 = 0.2315, C2 = 0.4013, C3 = 0.6900 — patients with higher baseline severity were, by construction of the model, more likely to have received the new drug (plausible: doctors often reach for a newer treatment on harder cases). Evaluating Y gives T1 = 3.5, T2 = 5.0, T3 = 6.0 days for the treated group and C1 = 5.0, C2 = 6.0, C3 = 7.5 days for the control group. The naive difference of means is (3.5 + 5.0 + 6.0)/3 − (5.0 + 6.0 + 7.5)/3 = 4.833 − 6.167 = −1.333 days — it looks like the drug saves 1.33 days, understating the true 2-day effect, because the treated group happened to be sicker on average (mean X = 5.67) than the control group (mean X = 4.33), and sicker patients recover more slowly regardless of drug.

Now match on propensity score, nearest neighbour, without replacement:

rem = control[:]
matches = []
for t in sorted(treated, key=lambda k: p[k]):
    best = min(rem, key=lambda c: abs(p[c] - p[t]))
    matches.append((t, best, abs(p[best] - p[t])))
    rem.remove(best)

Walking through that loop by hand, in order of ascending propensity score: T1 (0.3100) is closest to C1 (0.2315), a gap of 0.0785. T2 (0.5987) is closest to the remaining C3 (0.6900), a gap of 0.0913. T3 (0.7685) is left with only C2 (0.4013), a gap of 0.3672 — a forced, poor match, because no control patient was nearly as severe as T3. This is the common-support problem: matching can only work where the treated and control propensity distributions actually overlap, and T3 sits past the edge of that overlap. Apply a caliper — a maximum acceptable gap, here 0.10 — and drop matches that exceed it: the (T3, C2) pair is discarded, leaving (T1, C1) and (T2, C3).

The matched estimate is now the average within-pair difference for the surviving pairs: (3.5 − 5.0) and (5.0 − 7.5), averaged: (−1.5 + −2.5)/2 = −2.0 days — an exact match to the true causal effect built into the data. Trimming the one badly-matched pair, which was contaminating the comparison with an apples-to-oranges gap in severity, removed exactly the distortion that biased the naive estimate. The cost is that this matched estimate no longer describes the full treated population — it describes only the subpopulation with adequate overlap, formally the average treatment effect on the treated within common support, not the full-sample ATE.

How the two mechanisms fit together

The diagram below shows both pieces in one picture: at top, the confounder X sitting on backdoor paths into both T and Y, which is exactly the structure that produced the kidney-stone reversal; below, the actual matching mechanism from the worked example — where each treated patient's propensity score sits relative to the nearest control, and which pairs survive the caliper.

Common misconception: "matching removes confounding the way randomizing does"

Students who have just seen propensity score matching recover the exact true effect in a worked example often conclude that PSM is a substitute for randomization — a way to get RCT-quality answers from whatever observational data happens to be lying around. It is not, and the difference matters. A logistic regression for the propensity score can only include variables that were actually measured and entered into the model. In the worked example, severity X was the sole confounder by construction, so matching on it closed the only backdoor path there was, and the estimate landed exactly on the truth. Real observational data almost never has that guarantee: there is routinely some unmeasured factor — which hospital a patient could reach, family history nobody recorded, a doctor's private judgement of how the patient "looked" — that influenced both treatment choice and outcome but never made it into any column of the dataset. PSM balances only what you measured; an unmeasured confounder sails straight through, unbalanced, and biases the matched estimate exactly as it would have biased the naive one. Randomization is the only tool here that balances every confounder, known or unknown, because it doesn't rely on the analyst having thought to record it — it balances by the physical mechanics of the coin flip itself. Propensity score matching is best understood as the strongest available tool once randomization is off the table, not as randomization's equivalent.

Active recall

Attempt these before reading the answers.

1. In the kidney-stone data, identify the confounder and explain, in terms of the X → T and X → Y arrows, why the pooled comparison favoured PCNL while both stratified comparisons favoured open surgery.
2. State, in one sentence, the two conditions a variable must satisfy to count as a confounder of T's effect on Y.
3. In the synthetic drug example, why does propensity score matching fail to correct for a confounder — say, hospital resourcing — that was never recorded in the dataset at all?
4. Patient T3 (X = 8) was matched to C2 (X = 4) at a propensity gap of 0.3672, well outside the 0.10 caliper. What population does the caliper-matched estimate of −2.0 days actually describe once that pair is dropped, and why is that narrower than "the effect on all treated patients"?
5. Suppose the true data-generating process in the synthetic example had instead been Y = 4 + 0.5X − 3T (drug saves 3 days, not 2). Recompute the naive difference-in-means and the caliper-matched estimate. Which quantities change, which stay exactly the same, and why?
6. A colleague running a 500-patient RCT worries: "by chance, more severe patients might have landed in the drug arm." Is that a real risk, and how does randomization address it differently than propensity score matching would?

Worked answers

1. Stone size (X) is the confounder. Surgeons routed easy, small stones toward PCNL and difficult, large stones toward open surgery, so X → T. Small stones also recover better regardless of technique, so X → Y. Because PCNL's caseload was 77% small stones (270 of 350) while open surgery's was 75% large stones (263 of 350), PCNL's pooled rate (82.6%) was pulled up by an easy case mix and open surgery's pooled rate (78.0%) was pulled down by a hard case mix — even though open surgery beat PCNL in both the small-stone stratum (93.1% vs 86.7%) and the large-stone stratum (73.0% vs 68.8%).

2. X must causally influence T (it affects which treatment a unit is likely to receive) and causally influence Y (it affects the outcome independently of treatment), without itself being caused by T or Y.

3. The propensity score is estimated purely from the covariates fed into the model — here, only severity X. A variable that was never recorded, like hospital resourcing, can never enter that model, so matching on the resulting propensity score cannot balance it across treated and control groups. Any bias travelling through that unmeasured backdoor path remains in the matched estimate untouched.

4. It describes the average effect only for patients whose propensity scores fall within the region where both treated and control patients exist in the data (common support) — concretely, patients resembling T1 and T2, not patients resembling T3. T3's severity level had no comparably severe control patient to compare against, so no matching-based method can say anything reliable about the causal effect for patients that extreme; claiming the −2.0 days applies to "all treated patients including the most severe" would overstate what the matched comparison actually covers.

5. Recompute with T's coefficient changed from −2 to −3: treated outcomes become T1 = 2.5, T2 = 4.0, T3 = 5.0 (control outcomes are untouched, since they don't depend on the treatment coefficient: C1 = 5.0, C2 = 6.0, C3 = 7.5). The naive difference becomes (2.5+4.0+5.0)/3 − (5.0+6.0+7.5)/3 = 3.833 − 6.167 = −2.333. The propensity scores and therefore the matched pairs do not change at all — they depend only on X, never on Y — so the surviving matches are still (T1, C1) and (T2, C3). The matched estimate becomes (2.5−5.0 + 4.0−7.5)/2 = (−2.5 + −3.5)/2 = −3.0, again landing exactly on the new true effect, while the naive estimate (−2.333) is off by more than before. The lesson: which units get matched to which is a property of the confounder model alone; only the outcome numbers, and hence the resulting effect estimate, ripple when the outcome-generating process changes.

6. Yes, it's a real risk — with any finite sample, random assignment can produce an imbalance in some covariate purely by chance, and the risk is larger in smaller trials. Randomization's guarantee is about expectation, not about any single realized trial: averaged over many hypothetical re-randomizations, the treatment and control arms are balanced on every covariate, measured and unmeasured, so there is no systematic bias, only sampling noise that shrinks as the sample grows and that standard statistical tests already account for. Propensity score matching cannot offer that same guarantee for unmeasured covariates under any sample size, however large — it can only ever balance what was recorded. Researchers running an RCT can still check post-hoc balance on measured covariates and adjust the analysis if an imbalance shows up, but the structural protection against unmeasured confounders is unique to randomization, not something PSM replicates no matter how carefully it's applied.

Think About It

Think about this: How would you explain causal inference: cause vs correlation to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where causal inference: cause vs correlation is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting causal inference: cause vs correlation to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind causal inference: cause vs correlation, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.