Benchmarks and Leaderboards: Measuring AI Progress

In 2020, researchers at AI4Bharat (IIT Madras) hit a wall that had nothing to do with model architecture. India has 22 scheduled languages and hundreds of millions of speakers of Hindi, Tamil, Bengali, Telugu, and more, yet every major NLP benchmark of the day, GLUE and SuperGLUE included, was built entirely from English text. A model could top the GLUE leaderboard while being unable to correctly classify the sentiment of a Tamil movie review or resolve a pronoun in a Hindi news sentence. The benchmark was not lying: it measured exactly what it was built to measure, English natural-language understanding. The mistake would have been to read a GLUE score as a claim about the model's general language ability. So the team built IndicNLPSuite and the IndicGLUE benchmark (Kakwani et al., 2020, EMNLP Findings), a task suite spanning eleven Indian languages covering headline classification, article genre prediction, named entity recognition, and cross-lingual sentence retrieval, specifically so that "how good is this model at Indian languages" could be asked and answered with the same rigor English got for free. That single design decision, choosing what to measure and how, is the entire subject of this chapter. A benchmark is not a neutral yardstick lying around in nature; it is an engineered artifact, and every number a leaderboard reports inherits the assumptions baked into that artifact.

What a benchmark actually is

Formally, a benchmark bundles four things: a dataset (typically split into public training data and a held-out test set the model has never seen), a task format (multiple-choice, exact-match generation, code that must pass unit tests, free-form text judged by humans or another model), a metric (accuracy, F1, exact match, pass@k, an Elo rating), and an evaluation protocol (how models are allowed to query the test set, whether the test labels are ever released, how ties and partial credit are handled). Change any one of the four and you get a different benchmark, even if the underlying data looks similar. SuperGLUE (Wang et al., 2019, NeurIPS) exists as a distinct benchmark from GLUE (Wang et al., 2018) precisely because by 2019 models had pushed GLUE's average score above the estimated human baseline, so the task set itself, not just the models, had to be redesigned to keep measuring something meaningful. A leaderboard is simply the aggregation layer on top: it runs many models through the same protocol, applies the same metric, and ranks the resulting scores. The ranking is only as trustworthy as the protocol underneath it.

From a raw test item to a leaderboard rank

The mechanism that turns a dataset into a published rank is worth tracing explicitly, because every failure mode discussed later in this chapter is a failure at one specific stage of this pipeline.

Every stage in that diagram is a place where the number can drift away from the capability it claims to measure. The red dashed arrow marks the most consequential one: if the pretraining corpus overlaps with the held-out test set, the model is not being tested, it is being quizzed on material it memorized. That is why GLUE and SuperGLUE never released their test labels publicly, submissions had to go through a server that ran the hidden evaluation and returned only the score. BIG-bench (Srivastava et al., 2022) took a different defense: every test item is prefixed with a fixed canary GUID string, so if that string later turns up inside a model's training corpus (detectable because web crawls are searchable), the curators know the benchmark has leaked.

Worked example 1: the pass@k metric

HumanEval, introduced alongside OpenAI's Codex model (Chen et al., 2021), evaluates code generation by having the model draw n independent samples for a programming problem and checking how many, c, pass a hidden unit-test suite. The naive metric "generate one sample, check if it passes" is noisy: a model might solve a problem 40% of the time by chance across resamples, and a single draw does not reveal that. The paper's fix is the unbiased pass@k estimator: the probability that at least one of k samples an engineer is allowed to try would pass, estimated from a larger pool of n total samples without the high variance of literally drawing only k.

The derivation: if c of the n samples are correct, then n - c are wrong. The probability that a random draw of k samples from the pool of n contains zero correct ones equals the number of ways to choose k items entirely from the n - c wrong ones, divided by the number of ways to choose any k items from all n:

from math import comb

def pass_at_k(n: int, c: int, k: int) -> float:
    """Unbiased pass@k estimator (Chen et al., 2021, eq. 1).
    n: total samples generated per problem
    c: number of those samples that passed all hidden unit tests
    k: number of attempts the metric is scored at
    """
    if n - c < k:
        return 1.0
    return 1.0 - comb(n - c, k) / comb(n, k)

# One HumanEval problem: 5 samples drawn, 2 pass the hidden tests
print(pass_at_k(n=5, c=2, k=1))   # 0.4
print(pass_at_k(n=5, c=2, k=5))   # 1.0

Tracing the first call: n=5, c=2, k=1, so n - c = 3. Since 3 >= 1, the function computes 1 - C(3,1)/C(5,1) = 1 - 3/5 = 1 - 0.6 = 0.4. That matches intuition directly: 2 of the 5 draws were correct, so a single random draw succeeds with probability 2/5. The formula becomes useful precisely when it stops matching naive intuition, for example pass_at_k(n=5, c=1, k=1) gives 1 - C(4,1)/C(5,1) = 1 - 4/5 = 0.2, again equal to c/n, but for k > 1 the combinatorics genuinely differ from any simple ratio. The second call, k=5, hits the guard clause: n - c = 3 < 5, so it returns 1.0 directly, correctly reflecting that if you are permitted to try all five generated samples and at least one already passed, you are certain to succeed. A benchmark's reported HumanEval score is then the mean of pass_at_k across every problem in the test set, which is exactly the "aggregate = mean(score_i)" step in the scoring box of the diagram above.

Worked example 2: Elo rating on a human-preference leaderboard

Not every benchmark has a ground-truth answer key. Judging whether one chatbot's reply is more helpful than another's is a preference judgment, not an exact match, so LMSYS Chatbot Arena (Chiang et al., 2024) collects hundreds of thousands of pairwise human votes and aggregates them with an Elo-style rating system borrowed from competitive chess. Each model starts at a rating, and every match updates both models' ratings based on whether the outcome matched what their rating gap predicted.

Given ratings R_A and R_B, the expected probability A wins is E_A = 1 / (1 + 10^((R_B - R_A)/400)). After the match, with outcome S_A (1 for a win, 0 for a loss, 0.5 for a tie) and a K-factor controlling update size, the new rating is R_A' = R_A + K(S_A - E_A), and symmetrically for B.

def elo_update(r_a, r_b, score_a, k=32):
    expected_a = 1 / (1 + 10 ** ((r_b - r_a) / 400))
    new_a = r_a + k * (score_a - expected_a)
    new_b = r_b + k * ((1 - score_a) - (1 - expected_a))
    return round(new_a, 2), round(new_b, 2), round(expected_a, 4)

# Model A (1200) beats Model B (1250) in one Arena vote
print(elo_update(1200, 1250, score_a=1, k=32))
# (1218.29, 1231.71, 0.4285)

Tracing it by hand: R_B - R_A = 50, so 10^(50/400) = 10^0.125 ≈ 1.3335. Then E_A = 1 / (1 + 1.3335) = 1 / 2.3335 ≈ 0.4285, meaning the model was only expected to win about 43% of the time given it started lower-rated. Since it won anyway, it earns a positive update: R_A' = 1200 + 32(1 - 0.4285) = 1200 + 32(0.5715) = 1200 + 18.29 = 1218.29. Model B loses points: E_B = 1 - 0.4285 = 0.5715, and R_B' = 1250 + 32(0 - 0.5715) = 1250 - 18.29 = 1231.71. A useful sanity check on any Elo computation is that rating is zero-sum per match: 1218.29 + 1231.71 = 2450.00 = 1200 + 1250, confirming no rating was created or destroyed, only transferred based on the surprise of the outcome.

The misconception: "the top score means the best model"

The single most common error students make reading a leaderboard is treating the ranking as a total, context-free ordering of model quality. It is not, for three independent reasons visible directly in the pipeline above. First, construct validity: MMLU's 78% on Model C tells you about performance on 57 subjects of exam-style multiple-choice recall under a specific zero-shot prompt format, nothing more, exactly as IndicGLUE was built because GLUE's English score said nothing about Tamil. Second, contamination: Sainz et al. (2023, EMNLP Findings, "NLP Evaluation in Trouble") document widespread cases where public benchmark test sets had been scraped into pretraining corpora, inflating scores without any corresponding gain in real capability, which is why the red dashed arrow in the diagram is drawn as a defect path, not a normal part of the pipeline. Third, saturation: once most models cluster near a benchmark's ceiling, as happened to GLUE by 2019, ranking differences of a fraction of a percentage point are statistical noise, not meaningful capability gaps, which is precisely why SuperGLUE had to be built as a harder successor rather than continuing to rank models on GLUE. A single leaderboard number is a compressed, protocol-specific summary; treating it as an unqualified verdict on "which model is smarter" throws away exactly the information (what was measured, how, and whether the test set stayed truly held out) needed to interpret it correctly.

Active recall

Attempt each question before reading its answer.

Q1. A model draws n=10 samples for a HumanEval problem and 3 pass the hidden tests. Compute pass@1.
Q2. Using the same problem, compute pass@5.
Q3. Two Arena models both start at rating 1400. Model X beats Model Y in a vote, K=32. Compute the new rating for X.
Q4. Rework the original worked Elo example (Model A at 1200, Model B at 1250, A wins) using K=16 instead of K=32. Trace the full effect: the new ratings for both models, and how the gap between them compares to the K=32 case.
Q5. A team's 7B model jumps from 55% to 89% on MMLU overnight with zero architecture or training change, and shows no corresponding gain on private internal evaluations. Name the phenomenon and describe one concrete way a leaderboard maintainer could detect it.
Q6. Why does Chatbot Arena use pairwise human votes aggregated by Elo rather than a fixed accuracy metric like MMLU uses?

A1. n=10, c=3, k=1. n - c = 7 ≥ 1, so pass@1 = 1 - C(7,1)/C(10,1) = 1 - 7/10 = 0.3. This equals c/n exactly, as it always does at k=1.

A2. n=10, c=3, k=5. n - c = 7 ≥ 5, so the guard clause does not trigger: pass@5 = 1 - C(7,5)/C(10,5) = 1 - 21/252 = 1 - 0.0833 = 0.9167. Being allowed five attempts out of ten samples with three correct makes success very likely, but not certain, since the five failing-only combinations chosen from seven wrong samples still have a small nonzero chance (8.3%) of being drawn.

A3. Equal starting ratings mean E_X = 1/(1+10^0) = 1/2 = 0.5. R_X' = 1400 + 32(1 - 0.5) = 1400 + 16 = 1416. A gain of 16 is not the maximum a single win can earn; it is the midpoint. The update is K(1 - E_X), which grows as the winner's pre-match win probability shrinks, approaching the full K=32 only when the winner was a heavy underdog (E_X near 0), and shrinking toward 0 when the winner was already the expected victor (E_X near 1). At equal ratings E_X = 0.5, so the win carried no "upset" surprise, and the gain lands exactly halfway between those two extremes: K/2 = 16.

A4. Halving K to 16 does not change E_A, since the expected-score formula depends only on the rating gap, not K: E_A is still 0.4285 as computed earlier. What changes is the size of every subsequent update. R_A' = 1200 + 16(1 - 0.4285) = 1200 + 16(0.5715) = 1200 + 9.14 = 1209.14. R_B' = 1250 - 16(0.5715) = 1250 - 9.14 = 1240.86. The gap between the two models shrinks from 50 to 1240.86 - 1209.14 = 31.72 instead of the 50 - 2(18.29) = 13.42 gap the K=32 case produced. The ripple effect: a smaller K makes the leaderboard more stable match to match (less noise from any single vote) but also slower to converge to a model's true relative strength, a real tradeoff Arena-style systems must tune, not just an arithmetic detail.

A5. This is benchmark contamination: the model's training data almost certainly came to include text overlapping with the MMLU test set (for instance, a web crawl that ingested a page listing the exam answer key), so the model is recalling memorized answers rather than reasoning. A maintainer can detect it by running an n-gram overlap search between the model's training corpus and the benchmark's test items, checking whether a canary string embedded in the test set (as BIG-bench does) appears anywhere in the training data, or maintaining a private held-out split that mirrors the public one and checking for a large gap between public and private scores, a genuine capability gain should move both together.

A6. Open-ended chat responses do not have one correct string to exact-match against; two different replies can both be excellent. Absolute scoring (assigning each response a 1 to 10 rating) is also known to be noisy and drifts across annotators and sessions. Relative pairwise comparison, "which of these two replies is better", is a judgment humans make far more consistently, and Elo (or the closely related Bradley-Terry model Arena actually fits its ratings with) is specifically built to convert many noisy pairwise outcomes into one globally consistent ranking, the same way chess ratings turn thousands of individual games into a single ordered ladder of players who never all played each other directly.

Think About It

Think about this: How would you explain benchmarks and leaderboards: measuring ai progress to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where benchmarks and leaderboards: measuring ai progress is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting benchmarks and leaderboards: measuring ai progress to at least 3 other topics you have studied.