The Problem With "Not Out"
An analyst at an IPL franchise is comparing two young openers before the auction. Both have batted in eight innings this season. She pulls up how many balls each one faced before his innings ended and averages the numbers. Batsman A averages 28 balls. Batsman B averages 19. The recommendation looks obvious — sign Batsman A, since he clearly lasts longer at the crease.
There is a problem hiding inside that average. Three of Batsman B's eight innings ended in "not out." The team's innings finished around him — all the overs were bowled, or all ten wickets fell at the other end, or in a chase the target was already reached — while he was still batting. In each of those three innings, the only fact on record is that he survived at least that many balls. How many more he might have survived if the innings had carried on is simply unknown. Averaging those three innings as though he got out at that exact ball is not a small rounding error. It treats a lower bound as if it were an exact measurement, and it drags his average down every time.
The instinctive fix is to throw out the not-out innings and average only the ones where he actually got out. That is not a fix at all. It removes precisely the innings in which he lasted long enough to still be at the crease when the game ended — which, if anything, are his best performances. Whichever way the analyst handles those three innings, the number she reports is biased. This is not a quirk of cricket scorecards. It is a data problem with a name, and a whole branch of statistics exists to handle it correctly.
Time, Events, and Censoring
Survival analysis, also called time-to-event analysis, studies how long it takes for a specific, well-defined event to occur, measured from some starting point. The event does not need to be dramatic — it is simply the outcome being tracked: a batsman getting out, a customer cancelling a subscription, a machine part failing, a patient relapsing. What matters is that it is unambiguous: at any given moment, either it has happened or it has not.
Every subject under study has a duration — the elapsed time from a starting point (ball one, day one of a trial, the day a customer signs up) to whichever comes first: the event itself, or the moment observation stops. When the event is observed directly, the exact duration is known. When it is not — because the study ends, the subject drops out of contact, or observation simply stops before the event happens — the record is said to be censored, more precisely right-censored, which is the most common type and the only one this chapter needs: all that is known is that the true duration is at least as long as the time observed. A right-censored record is not missing data and it is not bad data. It carries real information, a guaranteed lower bound, and it simply cannot be treated as an exact value.
Every observation in a survival dataset is stored as a pair: a time and a status flag, 1 if the event was observed at that time and 0 if the record is censored there. Batsman B's season might look something like this as eight (time, status) pairs: (8, 1), (12, 0), (14, 1), (19, 1), (20, 0), (24, 0), (25, 1), (30, 1). Five clean dismissals, three not-outs — and if every value is taken at face value and averaged, 152 balls across 8 innings comes to exactly 19, matching the biased number the analyst started with.
The Survival Function and the Hazard Function
Survival analysis is built on two related functions of time. The survival function, written S(t), is the probability that the event has not yet happened by time t. At t = 0, before anything has had a chance to occur, S(0) = 1: everyone is still in. As t increases, S(t) can only stay the same or fall, because the event can only ever shrink the group still going — a batsman dismissed at ball 20 cannot un-happen his dismissal and rejoin the "still batting" group at ball 25.
The hazard function, h(t), asks a narrower, more local question: given that a subject has survived up to just before time t, what is the chance the event happens exactly at t? Where S(t) looks back over the whole journey from the start, h(t) only looks at the immediate next instant, conditional on having made it that far. For the discrete data this chapter works with — balls faced, days since diagnosis, months since sign-up — the hazard at any observed time t is simply the fraction of the still-active group that experiences the event right at t: the number of events at t divided by the number of subjects still under observation just before t.
The two functions describe the same process from different angles: surviving to time t means surviving every hazard along the way, one step at a time. This is also why an ordinary regression or classification model cannot be applied to this problem without running straight into the same bias from the opening example. A regression trained to predict "balls faced" has no valid target value for a not-out innings, since the true number is unknown, and using the observed ball count as if it were final reintroduces the exact bias described above. A classifier trained to predict "out within 20 balls: yes or no" only dodges that labelling problem by fixing one horizon and discarding every innings, dismissed or not, that does not clearly resolve by then — which quietly throws away data again. Survival analysis is built to use every observation, censored or not, without committing either mistake.
The Kaplan-Meier Estimator
The standard way to estimate S(t) directly from data containing censored observations is the Kaplan-Meier estimator, published by Edward Kaplan and Paul Meier in 1958 and still one of the most cited methods in applied statistics. Its formula is:
Ŝ(t) = product of (1 - d_i / n_i), taken over every distinct event time t_i ≤ t
Each symbol does real work. t_i is a time at which at least one event was actually observed — censoring-only times are skipped, since nothing about S(t) changes at a moment when nobody's status flips. d_i is the number of events that happened exactly at t_i, usually one but sometimes more if several subjects share a recorded time. n_i is the size of the risk set at that moment: everyone who has, up to just before t_i, neither experienced the event nor been censored. The fraction d_i / n_i is exactly the discrete hazard from the previous section, and (1 - d_i / n_i) is the probability of surviving that particular moment, given a subject had made it that far.
The product-limit idea is what makes censored data usable instead of troublesome. A censored subject stays inside the risk set n_i for every event time up to and including their own censoring time — their continued presence there is genuine evidence they survived at least that long — and then they quietly leave the risk set afterward, without ever being counted in any d_i. No guess is made about what happens to them after that point. The estimator simply stops using them, because the data no longer holds any information about them one way or the other.
Worked Example: The Survival Curve of Eight Batsmen
Suppose a coach pools data across eight separate innings, drawn from different matches, and records how many balls each batsman faced before his innings ended — either through dismissal, or because the team's innings finished around him while he was still in. Sorted by balls faced, with a "+" marking a not-out (censored) innings, the eight records are:
6, 11, 11+, 18, 24, 24, 40+, 55
Two of the eight are censored, 11+ and 40+; the rest are clean dismissals. There are five distinct times at which a dismissal actually happened — 6, 11, 18, 24, and 55 — so the survival curve only needs to be updated at those five points. Working through the product-limit formula one step at a time:
- t = 6: all eight batsmen are still in, so n = 8. One is out, d = 1. Survival factor = 1 - 1/8 = 0.875. Ŝ(6) = 1 × 0.875 = 0.875.
- t = 11: the batsman out at ball 6 has left the risk set, so n = 7 — this still includes both players recorded at 11, since both were "in" up to that instant. One of them is out, d = 1; the other is the not-out, and he leaves the risk set here without ever being counted as a d. Factor = 1 - 1/7 ≈ 0.857. Ŝ(11) = 0.875 × 0.857 = 0.75.
- t = 18: n = 5, d = 1. Factor = 1 - 1/5 = 0.8. Ŝ(18) = 0.75 × 0.8 = 0.6.
- t = 24: n = 4, and this time two batsmen are out on the same ball count, d = 2. Factor = 1 - 2/4 = 0.5. Ŝ(24) = 0.6 × 0.5 = 0.3.
- t = 40: no dismissal happens here, only the second not-out leaving the risk set. Since d = 0 the factor is 1, so Ŝ stays at 0.3 — but n for every later time drops accordingly.
- t = 55: only one batsman remains in the risk set, n = 1, and he is out, d = 1. Factor = 1 - 1/1 = 0. Ŝ(55) = 0.3 × 0 = 0.
Plotted on paper, Ŝ(t) looks like a staircase heading down and to the right: flat everywhere between dismissals, with a vertical drop at each ball count where one or more batsmen actually got out, and no drop at all at the censoring points. It steps down from 1 to 0.875, 0.75, 0.6, 0.3, and finally all the way to 0 — and it only reaches exactly 0 because the very last observation in the dataset, 55, happens to be a dismissal. Had that last observation instead been a not-out, the curve would trail off at whatever value it last reached and simply stay beyond that point, because the data would carry no information past it.
One useful number falls straight out of this curve: the median survival time, defined as the earliest time at which the estimated survival probability falls to 0.5 or below. Here the curve drops straight from 0.6 to 0.3 at t = 24, jumping past 0.5 without ever landing on it exactly — a normal consequence of working with a step function built from only eight observations. By the standard convention, 24 balls is still reported as the median survival time: the first point at which the data shows survival has fallen to half or less. It is a single, defensible summary that a plain average of the eight raw ball counts, whether the not-outs are counted at face value or dropped altogether, could not have produced correctly either way.
Tracing It in Code
The hand calculation above is mechanical enough to hand to a function. The code below sorts the records, finds every distinct time an event happened, and multiplies the survival factors together exactly as done above:
def kaplan_meier(times, events):
"""
times : observed balls faced (or any time-to-event value)
events : 1 if the event happened (out), 0 if censored (not out)
Returns a list of (time, at_risk, deaths, survival_estimate)
"""
records = sorted(zip(times, events))
distinct_event_times = sorted(set(t for t, e in records if e == 1))
survival = 1.0
table = []
for t in distinct_event_times:
at_risk = sum(1 for time, e in records if time >= t)
deaths = sum(1 for time, e in records if time == t and e == 1)
survival *= (1 - deaths / at_risk)
table.append((t, at_risk, deaths, round(survival, 4)))
return table
balls_faced = [6, 11, 11, 18, 24, 24, 40, 55]
out = [1, 1, 0, 1, 1, 1, 0, 1] # 1 = out, 0 = not out (censored)
for t, n, d, s in kaplan_meier(balls_faced, out):
print(f"balls={t:>3} at_risk={n} events={d} S(t)={s}")
Running this prints:
balls= 6 at_risk=8 events=1 S(t)=0.875
balls= 11 at_risk=7 events=1 S(t)=0.75
balls= 18 at_risk=5 events=1 S(t)=0.6
balls= 24 at_risk=4 events=2 S(t)=0.3
balls= 55 at_risk=1 events=1 S(t)=0.0
Ball 40 never appears as its own row, because the loop only visits distinct_event_times, and 40 was a censoring time with zero events there. Its effect is still felt one row later, hidden inside the at_risk count: at t = 55, at_risk is 1, not 2, because the condition time >= t correctly stops counting the batsman who was censored at 40. That single condition is what keeps a censored observation contributing to the risk set for exactly as long as it was actually observed, and drops it cleanly the moment it was not — the entire logic of correct censoring handling, in one line of Python.
Where This Shows Up Beyond the Boundary Rope
The same product-limit formula, and the same idea of a risk set losing members to events and to censoring, appears across fields that have nothing to do with cricket:
- Telecom subscriber churn. Indian telecom operators track, for every prepaid and postpaid customer, how long they stay before cancelling or porting to another network. The event is churn; every customer still active on the day a report is generated is censored, known to have survived at least that long, with the remaining time unknown. Comparing survival curves between two pricing plans is the standard way to see which plan actually keeps customers for longer, rather than comparing raw average tenure and running into the same bias as the batting average.
- Clinical trials. This is the field the method was built for. Kaplan and Meier's 1958 paper addressed exactly this setup: some patients in a trial relapse or pass away, which is the event, while others are still healthy when the trial concludes or lose contact with researchers before then, which is censoring. Comparing the survival curve of a treatment group against a control group remains a standard statistical backbone of how new treatments are evaluated.
- Reliability engineering. A manufacturer testing how long a battery, a bearing, or a hard drive lasts cannot wait for every unit in a test batch to fail before shipping a warranty policy. Units still working when the test window closes are censored, and the resulting survival curve is what an actual warranty period gets calculated from.
- App and product analytics. "Time until a free-trial user converts to paid" and "time until a user uninstalls an app" are both time-to-event questions, and every user still active today is, once again, a censored observation.
When the goal shifts from estimating one overall curve to asking how a specific factor — treatment type, price plan, device model — changes the hazard, the standard next step is the Cox proportional hazards model, introduced by David Cox in 1972, which extends this same hazard-based reasoning to handle several predictor variables at once. The Kaplan-Meier curve remains the starting point for all of this. It is usually the first thing plotted before any more complex model is fit, because it is the most direct, assumption-light answer to the original question: how long, honestly, do things last.
Back to the Crease
The IPL analyst from the opening now has a better tool than a spreadsheet average. Instead of one biased number per batsman, she can plot each one's Kaplan-Meier survival curve — the probability he is still batting past 10 balls, past 20, past 40 — and read off a defensible median survival time for each. A not-out innings is no longer an awkward row to argue about deleting or mislabelling. It is exactly as informative as a dismissal, just informative about a different thing, a guaranteed lower bound instead of an exact value, and the estimator uses both kinds correctly, at the same time, inside the same formula.
The pattern worth watching for, in cricket or anywhere else, is the word "still." A customer who is still subscribed, a patient who is still alive at the end of a trial, a component that is still running when the test rig is switched off, a batsman who is still not out — every one of these is a censored observation carrying real, partial information. Deleting it throws away the longest survivors in the dataset. Treating it as if the event had already happened understates how long things actually last. Survival analysis, built on nothing more exotic than a risk set shrinking one honest step at a time, is the tool that puts that whole "still" to work instead of discarding it.
Think About It
Think about this: How would you explain survival analysis: time until event to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Practice Exercises
Now it is time to practice! Complete these challenges to solidify your understanding:
- Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
- Exercise 2: Find a real-world example where survival analysis: time until event is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
- Exercise 3: Create a mind-map connecting survival analysis: time until event to at least 3 other topics you have studied.