A tutoring system that discovered the grade level was fiction
In 2015, economists Karthik Muralidharan, Abhijeet Singh, and Alejandro Ganimian ran a randomized controlled trial on an Indian adaptive-learning product called Mindspark, built by the Educational Initiatives team and deployed in after-school centers in Delhi. The study, published in the American Economic Review in 2019 as "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," found something more interesting than "the software worked." Mindspark begins every student with a short diagnostic that walks up and down across grade levels — testing a Class 8 student on Class 6 content, or Class 10 content, until it locates where the student's actual competence sits. The diagnostic routinely placed students two to three grade levels away from the syllabus they were formally enrolled in, in both directions. A student sitting in a Class 8 classroom, being taught Class 8 mathematics by a teacher working through the Class 8 syllabus, might genuinely be ready for Class 10 problems, or might be missing Class 6 fundamentals the classroom pace had already left behind. The randomized trial found large learning gains from the after-school Mindspark sessions — on the order of several tenths of a standard deviation in math over a few months, roughly comparable to intensive one-on-one tutoring — and the gains were not concentrated in one ability band; they showed up for students who were behind grade level and for students who were ahead of it.
The mechanism worth isolating from that result is not "computers are patient" or "software never gets bored." It's that a classroom serves one pace to thirty-plus students of genuinely different competence, and an adaptive system doesn't have to. But "adaptive" is a word that gets used loosely enough to mean almost nothing. This chapter makes it precise: what exactly is being estimated, how the estimate updates from one problem attempt to the next, and where that architecture starts to break down.
What "adaptive" is actually estimating
A student's knowledge of a skill — solving a quadratic by factoring, say — is not directly observable. You cannot open the student's head and read off a mastery percentage. All you ever see is a stream of behavior: attempt, correct or incorrect, time taken, which wrong answer, whether a hint was requested. An adaptive tutoring system's entire job is to maintain a running estimate of an unobservable (latent) quantity — call it P(L), the probability the student currently knows the skill — from a noisy stream of observable evidence. This is a state-estimation problem in the same family as tracking a rocket's position from noisy radar returns, and the estimator conventionally used is a form of Bayesian filtering. Every serious intelligent tutoring system (ITS), from the 1980s Carnegie Mellon Cognitive Tutors through to modern platforms, is built from four components that separate cleanly, and keeping them separate is the actual engineering discipline: a domain model (what the skills are, and their prerequisite structure — you can't attempt "solve the quadratic formula" meaningfully before "evaluate a square root"), a student model (the running latent-mastery estimate per skill, per learner — this is where Bayesian Knowledge Tracing lives), a pedagogical model (the policy that decides what to do with that estimate — present another problem on this skill, escalate to a hint, or advance to the next skill), and an interface (where the learner actually attempts problems and the raw correct/incorrect signal is captured). The loop these four run is shown below, followed by the arithmetic that drives the student-model box.
Bayesian Knowledge Tracing: the arithmetic of "does the student know this?"
Bayesian Knowledge Tracing (BKT) was introduced by Albert Corbett and John Anderson in their 1994 paper "Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge" (User Modeling and User-Adapted Interaction, vol. 4), built into the Cognitive Tutor algebra systems at Carnegie Mellon. It models each skill independently with a two-state hidden Markov chain: at any moment, the student either knows the skill (state L) or doesn't (state ¬L). You never observe the state directly — you observe a correct or incorrect response, which is a noisy function of the state, governed by four parameters:
P(L0) — the prior probability the student already knows the skill before any evidence from this system. P(T) — the transition probability: the chance that an unknowing student moves into the "knows" state as a result of this one practice opportunity (this is the model's stand-in for actual learning). P(G) — the guess probability: the chance of a correct answer even in the ¬L state (multiple-choice items inflate this). P(S) — the slip probability: the chance of an incorrect answer even in the L state (a careless arithmetic error, a misread question, a rushed click).
Given these four numbers, updating the estimate after an observed response is a direct application of Bayes' rule, done in two steps. First, revise the belief given the response just observed, holding the transition aside — this asks "what does this one data point tell me about whether the student already knew it before answering?" If the response was correct:
P(L | correct) = [P(L)·(1−P(S))] ⁄ [P(L)·(1−P(S)) + (1−P(L))·P(G)]
The numerator is the probability of the joint event "student knew it AND didn't slip." The denominator is the total probability of a correct answer, marginalizing over both hidden states — "knew it and didn't slip" plus "didn't know it but guessed right." This is exactly P(A|B) = P(B|A)P(A) / P(B) with A = "knows the skill" and B = "answered correctly," expanded by the law of total probability in the denominator. If the response was incorrect, the mirror-image expression applies:
P(L | incorrect) = [P(L)·P(S)] ⁄ [P(L)·P(S) + (1−P(L))·(1−P(G))]
Second, having updated the belief given this observation, account for the fact that the practice attempt itself is an opportunity to learn — even a student who didn't know the skill going in may have learned it from the act of attempting the problem and seeing the correct-answer feedback. This is the transition step, applied after the observation update:
P(L)next = P(L | observation) + [1 − P(L | observation)] · P(T)
Note the asymmetry that makes this a genuine hidden Markov model rather than a simple running average: the transition only moves probability mass in one direction, from ¬L toward L. BKT assumes forgetting doesn't happen within a tutoring session — a simplification real deployments sometimes relax with an added "forgetting" parameter, but the four-parameter version above is the one you'll see named in the literature and the one CBSE-adjacent platforms typically implement.
Worked example: tracing one skill across three attempts
Take a skill — factoring a quadratic — with parameters typical of the values Corbett and Anderson fit from real classroom data: P(L0) = 0.40, P(T) = 0.30, P(G) = 0.20, P(S) = 0.10. A student attempts three problems on this skill: correct, correct, incorrect. Trace it by hand for the first step, then confirm with code.
Attempt 1, correct. Observation update: numerator = 0.40 × (1 − 0.10) = 0.36. Denominator = 0.36 + (1 − 0.40) × 0.20 = 0.36 + 0.12 = 0.48. So P(L | correct) = 0.36 / 0.48 = 0.75. Transition: P(L)next = 0.75 + (1 − 0.75) × 0.30 = 0.75 + 0.075 = 0.825. One correct answer moves the estimate from a 40% prior to 82.5% — a large jump, because with only a 20% guess rate, a correct answer is fairly diagnostic of actually knowing the skill.
The full three-step trace is mechanical from here, so run it rather than hand-carry the decimals:
def update(L, correct, T, G, S):
if correct:
num = L*(1-S)
den = L*(1-S) + (1-L)*G
else:
num = L*S
den = L*S + (1-L)*(1-G)
L_post = num/den
L_next = L_post + (1-L_post)*T
return L_post, L_next
L = 0.40 # P(L0): prior probability the skill is known
T, G, S = 0.30, 0.20, 0.10 # learn, guess, slip
for i, correct in enumerate([True, True, False], start=1):
L_post, L = update(L, correct, T, G, S)
print(f"attempt {i}: correct={correct} P(L|obs)={L_post:.4f} P(L)={L:.4f}")Output:
attempt 1: correct=True P(L|obs)=0.7500 P(L)=0.8250
attempt 2: correct=True P(L|obs)=0.9550 P(L)=0.9685
attempt 3: correct=False P(L|obs)=0.7935 P(L)=0.8554This is exactly the trajectory plotted in the diagram above. Two things are worth reading off it precisely. First, after attempt 2 the estimate (0.9685) has already crossed a typical mastery threshold of 0.95 — under the Cognitive Tutor's original policy, the pedagogical model would mark this skill mastered and advance the student to the next one before attempt 3 ever happens. Second, and this is the part most students get wrong on first encounter: attempt 3 is a wrong answer, yet the posterior estimate only falls to 0.7935 and the post-transition estimate to 0.8554 — nowhere close to 0.40, let alone zero. The model doesn't treat one wrong answer as proof of not knowing the material, because it was built, from the start, to expect that 10% of the time a student who knows the skill will still slip.
The misconception: "a wrong answer means the system thinks I don't know it"
This is the single most common wrong intuition students and even some teachers bring to adaptive systems, and it comes from over-generalizing item-response-theory-based computerized adaptive testing (CAT) — the format used in many standardized entrance exams, which does select a harder or easier next item based on each response — onto tutoring systems, which are solving a different problem. A CAT is trying to pinpoint a fixed ability level as efficiently as possible in a one-shot exam; a tutoring system is trying to track a slowly evolving mastery state across many practice opportunities, where a single response is a noisy sample, not a verdict. The guess and slip parameters exist precisely to prevent the estimate from swinging wildly on noise. A P(S) = 0.10 means the model has, from the outset, budgeted for the fact that competent students misclick, misread, or make an arithmetic slip about one time in ten — punishing every such event with a full reset to "doesn't know this" would make the estimate useless, since it would then be dominated by the observation with the worst luck rather than the balance of evidence. The corrected mental model: BKT's output at any point is a belief integrated over the full observed history, and how far any single response is allowed to move that belief is set by G and S, not by the correct/incorrect label alone. Concretely, in the worked trajectory above the estimate needed only one correct answer to jump 35 points (0.40 → 0.75), and one slip after two strong correct answers only cost 11 points (0.9685 → 0.8554) — the same "one wrong answer," wildly different consequences, because Bayesian updating always weighs new evidence against however much evidence already accumulated, never in isolation.
Beyond BKT: what changes when you don't hand-label skills
BKT has a real limitation: it requires every problem in the item bank to be hand-tagged with which skill (or skills) it exercises, and it models each skill's mastery completely independently — practicing factoring tells the model nothing about the student's quadratic-formula skill, even though a human tutor would expect transfer between them. Piech, Bassen, Huang, Ganguli, Sahami, Guibas, and Sohl-Dickstein's 2015 NeurIPS paper "Deep Knowledge Tracing" replaced the per-skill hidden Markov chain with a single recurrent neural network (an LSTM) that ingests the entire sequence of (skill attempted, correct/incorrect) pairs across all skills and predicts the probability of a correct response on every skill at the next timestep, for every timestep. Instead of one scalar P(L) per skill maintained by hand-derived Bayes updates, DKT compresses the whole interaction history into a distributed hidden state, and lets backpropagation discover which skills predict which — including cross-skill transfer BKT structurally cannot represent.
The architecture, standard sequence classification: a one-hot encoding of (skill, correct/incorrect) at each timestep feeds an LSTM, and a final linear layer plus sigmoid maps the hidden state down to a per-skill mastery probability vector.
import torch
import torch.nn as nn
class DKT(nn.Module):
def __init__(self, num_skills, hidden_dim=20):
super().__init__()
self.lstm = nn.LSTM(input_size=2*num_skills, hidden_size=hidden_dim, batch_first=True)
self.out = nn.Linear(hidden_dim, num_skills)
def forward(self, x):
h, _ = self.lstm(x)
return torch.sigmoid(self.out(h))
num_skills = 5
model = DKT(num_skills)
x = torch.zeros(1, 3, 2 * num_skills) # batch=1, 3 opportunities, 5 skills
y = model(x)
print("input shape :", tuple(x.shape))
print("output shape:", tuple(y.shape))
print("all outputs in [0,1]:", bool((y >= 0).all() and (y <= 1).all()))input shape : (1, 3, 10)
output shape: (1, 3, 5)
all outputs in [0,1]: TrueThe shapes are worth reading carefully: the input at each timestep is 2 × num_skills wide (10, for 5 skills) because it one-hot encodes both which skill was attempted and whether the response was correct. The output at each timestep is num_skills wide (5) — one mastery probability per skill, at every point in the sequence, not just the skill just attempted, which is exactly the cross-skill transfer BKT can't do. This is also where DKT trades away something BKT has: interpretability. In BKT, P(L) is a single named scalar you can point to and explain to a teacher. In DKT, the hidden_dim=20 LSTM units and the 5 output probabilities are connected by a dense weight matrix — no individual hidden unit corresponds to "mastery of factoring," and the 20-dimensional internal state is a genuinely entangled, learned representation. A 2011 meta-analysis by Kurt VanLehn ("The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems," Educational Psychologist, 46(4)) is the standard reference point for how much any of this matters in outcomes: it found that step-based intelligent tutoring systems produced learning gains close to, though slightly below, one-on-one human tutoring — both landing well short of the "two sigma" (2 standard deviation) improvement Benjamin Bloom's original 1984 studies had reported for human tutors, with VanLehn's pooled estimates closer to roughly three-quarters of a standard deviation for both ITS and human tutors relative to classroom instruction. The practical reading: the specific algorithm inside the student model (BKT versus DKT versus an item-response-theory model doing computerized adaptive testing rather than tutoring at all) matters less to outcomes than simply having some mechanism that estimates individual mastery and adapts pacing to it — which is the same lesson the Mindspark trial's grade-level finding pointed at from a completely different angle.
Active recall
Attempt each question before reading its answer.
Q1. Using the chapter's base parameters (P(L0)=0.40, P(T)=0.30, P(G)=0.20, P(S)=0.10), a student's very first attempt on the skill is incorrect. Compute P(L | incorrect) and the post-transition P(L1).
Q2. Suppose the item bank for this skill is redesigned as two-option multiple choice, raising P(G) from 0.20 to 0.50, with everything else unchanged. If the student's first attempt is correct, compute the new P(L | correct) and explain, in terms of the Bayes' rule expression, why it moved the direction it did.
Q3. In the chapter's worked example, P(L2) = 0.9685 already exceeds a 0.95 mastery threshold after attempt 2 — before the incorrect attempt 3 occurs. If the pedagogical model advances the student to the next skill the moment P(L) first crosses 0.95, does attempt 3's incorrect response ever get recorded against this skill's BKT chain? What does this imply about which of the four ITS components — domain, student, pedagogical, or interface — owns the decision to stop practicing a skill, and why must that decision live outside the student model itself?
Q4. In the DKT code shown, hidden_dim=20 and num_skills=5, so the output at each timestep has 5 values but the LSTM's internal state has 20. Why can't you read off "P(mastery of skill 3)" from a single one of the 20 hidden units the way you can read P(L) directly as BKT's entire state?
Q5. A school deploys an adaptive tutor whose item bank has ambiguously worded questions in an unfamiliar second language, so students who genuinely know the material end up answering incorrectly far more often than the modeled P(S)=0.10 assumes — the model's slip parameter is calibrated too low for what's actually happening. Trace what happens to a competent student's P(L) trajectory over several such answers, and explain why this is specifically an equity risk when P(L) is used to place students into remedial tracks.
Q6. A curriculum redesign is found to teach faster, raising P(T) from 0.30 to 0.50, with P(L0)=0.40, P(G)=0.20, P(S)=0.10 unchanged. Recompute the full three-step trajectory (correct, correct, incorrect) from the chapter's worked example under this new P(T), and compare the final P(L3) to the original 0.8554.
A1. Numerator = 0.40 × 0.10 = 0.04. Denominator = 0.04 + (1 − 0.40) × (1 − 0.20) = 0.04 + 0.60 × 0.80 = 0.04 + 0.48 = 0.52. P(L | incorrect) = 0.04 / 0.52 ≈ 0.0769. Transition: P(L1) = 0.0769 + (1 − 0.0769) × 0.30 ≈ 0.0769 + 0.2769 = 0.3538. A single wrong answer on the very first attempt, with no prior evidence to protect it, drops the estimate sharply — from 0.40 down to roughly 0.35 after accounting for the chance of same-attempt learning. Contrast this with attempt 3 in the main worked example, where the same "incorrect" observation barely dented an already-high estimate (0.9685 → 0.8554): identical evidence, very different impact, because Bayesian updating weighs new evidence against however much prior evidence already exists.
A2. Numerator is unchanged: 0.40 × 0.90 = 0.36. Denominator = 0.36 + (1 − 0.40) × 0.50 = 0.36 + 0.30 = 0.66. P(L | correct) = 0.36 / 0.66 ≈ 0.5455, versus 0.75 with the original P(G)=0.20. The posterior moved down because the denominator — the total probability of seeing a correct answer regardless of the hidden state — grew: with a 50% guess rate, a correct answer is much less informative about whether the student actually knew the material, since a coin-flip-guessing non-knower would get it right half the time anyway. This is the general principle behind why well-designed adaptive systems avoid two-option items: raising P(G) doesn't just make guessing easier, it structurally weakens every correct answer's diagnostic value for the rest of that student's history on the skill.
A3. No — if the pedagogical model's policy is "advance the instant P(L) first exceeds 0.95," the student is moved to the next skill immediately after attempt 2 (P(L2)=0.9685), and attempt 3 would be a problem on the new skill, updating that skill's own BKT chain, not this one's. The original skill's estimate would be frozen at 0.9685 forever in this student's record. This shows that "when to stop practicing a skill" is not a property the raw mastery estimate can answer by itself — the student model only ever outputs a number; deciding what threshold to act on, and whether to demand a margin of extra practice past the threshold as insurance against exactly this kind of one-slip false completion, is a policy choice, which is why it belongs to the pedagogical model. Folding the threshold logic into the student model would conflate "what do I believe" with "what should I do about what I believe" — the same separation of concerns that keeps the domain model's skill graph independent of both.
A4. Because the linear output layer maps all 20 hidden units to each of the 5 outputs through a dense weight matrix — every output skill's probability is a weighted combination of all 20 hidden units, and every hidden unit's activation is shaped during training by its contribution to predicting all 5 skills' next-step correctness, not just one. There is no constraint forcing the learning process to dedicate one hidden unit exclusively to one skill; the representation is distributed and entangled by construction. That's exactly the tradeoff against BKT, where P(L) is deliberately a single, named, human-legible number precisely because the model was built with one such number per skill and nothing else.
A5. Because the model believes P(S)=0.10 while the true slip-from-ambiguous-wording rate is much higher, every incorrect answer this student produces gets attributed mostly to "doesn't know the skill" rather than to the language-driven slip that's actually happening — the update equation has no way to distinguish a genuine conceptual gap from a wording-driven misread, since it only sees "correct" or "incorrect." Repeated incorrect responses will drag this student's P(L) down toward "doesn't know," even though the underlying competence never moved, and if that P(L) feeds a decision to place the student in a remedial track, a student is streamed downward based on a language or item-design artifact rather than actual knowledge. This risk concentrates specifically on students least matched to the item bank's assumed language or phrasing — which in a multilingual system is exactly the population most likely to be underserved already, making a single miscalibrated parameter an equity issue, not just a measurement-noise issue.
A6. Attempt 1 correct: posterior is unaffected by P(T) (=0.75, same as before), but the transition step changes: P(L1) = 0.75 + (1 − 0.75) × 0.50 = 0.75 + 0.125 = 0.875. Attempt 2 correct: numerator = 0.875 × 0.90 = 0.7875; denominator = 0.7875 + (1 − 0.875) × 0.20 = 0.7875 + 0.025 = 0.8125; posterior = 0.7875/0.8125 ≈ 0.9692; transition: P(L2) = 0.9692 + (1 − 0.9692) × 0.50 ≈ 0.9846. Attempt 3 incorrect: numerator = 0.9846 × 0.10 ≈ 0.09846; denominator ≈ 0.09846 + (1 − 0.9846) × (1 − 0.20) = 0.09846 + 0.01538 × 0.80 ≈ 0.09846 + 0.01231 = 0.11077; posterior ≈ 0.09846/0.11077 ≈ 0.8889; transition: P(L3) ≈ 0.8889 + (1 − 0.8889) × 0.50 ≈ 0.8889 + 0.0556 = 0.9444. Faster learning (higher P(T)) pushes every post-transition estimate higher across the whole trajectory, not just the step where it's applied — because each step's transition compounds into the starting point of the next step's posterior calculation. Both versions of the trajectory still cross the 0.95 mastery threshold at exactly the same point, after attempt 2 (0.9846 here versus 0.9685 originally) — raising P(T) doesn't move the crossing earlier once two straight correct answers have already done that. What it changes is the floor the slip lands on: the final P(L3) rises from 0.8554 to roughly 0.9444, a far thicker buffer above the threshold despite absorbing the identical incorrect response, because more transition credit was already banked into P(L2) before attempt 3 ever happened.
Think About It
Think about this: How would you explain ai for education: adaptive learning and tutoring systems to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind ai for education: adaptive learning and tutoring systems, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.