The Bar That Moves With Every Ball
Open a live cricket score app during the last five overs of a tight T20 run chase, and a thin bar usually sits near the scorecard: a live win probability, updating after almost every ball — say, 62% for the batting side, 38% for the bowling side. A dot ball nudges it. A wicket swings it hard. A six swings it back. Behind that bar sits a model trained on thousands of earlier matches to turn a match situation — runs needed, balls remaining, wickets in hand — into a probability. Nobody typed cricket rules into it by hand. The model was shown a situation, made a guess, was told the real outcome, and was nudged a little closer to the truth. Deciding exactly how much to nudge each internal number, so the next guess improves, rests on one idea from calculus: the chain rule. It is the same idea, applied by hand below to a miniature win-probability predictor and then handed to software, that lets every modern deep learning system learn anything at all.
Composite Functions: When One Quantity Hides Inside Another
A derivative measures how fast one quantity changes as another changes — the slope of a graph at a point. Many functions act on their input directly, like x^2 or sin(x). But plenty of useful quantities are built by feeding the output of one function into another. Take y = (3x + 1)^2. Here y does not depend on x directly; it depends on an intermediate quantity u = 3x + 1, and y = u^2 depends on u. This is a composite function — a function built by nesting one function inside another — and differentiating it needs the chain rule: how fast y changes with x equals how fast y changes with u, multiplied by how fast u changes with x.
In symbols, if y = f(u) and u = g(x), then:
dy/dx = dy/du * du/dxFor y = (3x + 1)^2, set u = 3x + 1 so y = u^2. Then dy/du = 2u and du/dx = 3, so dy/dx = 2u * 3 = 6u = 6(3x + 1). At x = 1, u = 4, so dy/dx = 6 * 4 = 24. Expanding directly confirms it: y = 9x^2 + 6x + 1, so dy/dx = 18x + 6, which also gives 18(1) + 6 = 24 at x = 1. Both routes agree, and that is the point — the chain rule is not a shortcut that sometimes works. It is the exact answer, reached by tracking how a change propagates through each layer of nesting, one local derivative at a time. For a function nested five or ten layers deep, expanding it out algebraically stops being realistic; multiplying local derivatives one link at a time still works perfectly.
Chain Rule Strikes Again: Differentiating the Sigmoid
Neural networks rarely pass a number straight through unchanged; they squash it through an activation function so the network can represent curved, non-linear relationships instead of only straight lines. A common choice, especially for outputs that should read as probabilities, is the sigmoid function:
sigmoid(z) = 1 / (1 + e^-z)It squashes any real number z into a value strictly between 0 and 1. Using it inside a trainable network requires its derivative, and finding that derivative is itself a chain rule exercise nested two levels deep. Write sigmoid(z) as u^-1, where u = 1 + e^-z. By the power rule, d/du(u^-1) = -u^-2. By the chain rule, du/dz = -e^-z — the derivative of e^-z with respect to z is e^-z multiplied by the derivative of -z, which is -1, so the chain rule is at work even inside the exponent. Multiplying the two local derivatives together:
d/dz sigmoid(z) = (-u^-2) * (-e^-z) = e^-z / (1 + e^-z)^2A short algebraic rearrangement shows this is exactly sigmoid(z) * (1 - sigmoid(z)):
sigmoid'(z) = sigmoid(z) * (1 - sigmoid(z))That compact formula does a lot of work later. It means that once a sigmoid neuron's output is known, its local derivative is already known too, with no extra exponentials left to compute.
Building the Win-Probability Network
To see the chain rule do real work, build the smallest network that can still learn something: two inputs, one hidden neuron, one output neuron. Feed it a simplified, normalized description of a run chase:
x1 = 0.5— a pressure index: how far the required run rate has pulled ahead of the current scoring rate, scaled to sit between 0 and 1.x2 = 0.8— wickets in hand, scaled: 8 wickets remaining out of 10 becomes 0.8.
The hidden neuron combines both inputs with weights and a bias, then squashes the result through sigmoid; the output neuron does the same to the hidden neuron's output, producing a final win probability. Every connection needs a starting number before training begins, so give the network some initial, essentially arbitrary weights:
w1 = 0.6, w2 = -0.3, b1 = 0.1— hidden layerw3 = 0.9, b2 = -0.2— output layer
These do not yet look like sensible cricket knowledge — w2 is negative even though more wickets in hand should intuitively raise the win probability. That is expected, not a mistake. Before training, a network's weights are typically initialized to small, essentially random numbers with no real understanding baked in. Everything that follows exists to let the network correct itself, one match situation at a time, using the chain rule to work out exactly which weight is responsible for an inaccurate guess, and by how much.
Forward Pass: From Match Situation to Prediction
Suppose, in this hypothetical run chase, the batting side went on to win — the true outcome is y = 1. Before the network can learn anything it must first make a prediction. Push the inputs through exactly as wired above:
Hidden neuron
z1 = w1*x1 + w2*x2 + b1
= (0.6)(0.5) + (-0.3)(0.8) + 0.1
= 0.30 - 0.24 + 0.10 = 0.1600
h = sigmoid(z1) = 1 / (1 + e^-0.16) = 0.5399
Output neuron
z2 = w3*h + b2
= (0.9)(0.5399) + (-0.2)
= 0.4859 - 0.2 = 0.2859
y_hat = sigmoid(z2) = 1 / (1 + e^-0.2859) = 0.5710
Loss (squared error against the true outcome y = 1)
L = 0.5 * (y_hat - y)^2
= 0.5 * (0.5710 - 1)^2
= 0.5 * 0.1840 = 0.0920The network predicted a 57.1% chance of winning for a side that, in truth, won with certainty. That gap between 0.5710 and 1 is the loss: a single number measuring how wrong the prediction was. Training means adjusting w1, w2, b1, w3, b2 to make that number smaller the next time the network sees a similar situation — and the chain rule is what determines, precisely, which direction and how far to move each one.
Backward Pass: The Chain Rule, Link by Link
The loss was built by composing several functions in sequence: L depends on y_hat, which depends on z2, which depends on h, w3, and b2; and h depends on z1, which depends on w1, w2, b1, x1, and x2. To find how L changes with respect to any weight buried inside that chain, apply the chain rule repeatedly, one link at a time, always multiplying local derivatives together. This is exactly the mechanism behind backpropagation: propagating the loss's gradient backward through the network, layer by layer.
Start at the loss and work backward:
Step 1 - loss with respect to the prediction
dL/dy_hat = y_hat - y = 0.5710 - 1 = -0.4290
Step 2 - through the output sigmoid
dy_hat/dz2 = y_hat*(1 - y_hat) = 0.5710 * 0.4290 = 0.2450
dL/dz2 = dL/dy_hat * dy_hat/dz2 = (-0.4290)(0.2450) = -0.1051
Step 3 - into the output weight, bias, and hidden output
dz2/dw3 = h = 0.5399 -> dL/dw3 = -0.1051 * 0.5399 = -0.0567
dz2/db2 = 1 -> dL/db2 = -0.1051 * 1 = -0.1051
dz2/dh = w3 = 0.9 -> dL/dh = -0.1051 * 0.9 = -0.0946
Step 4 - through the hidden sigmoid
dh/dz1 = h*(1 - h) = 0.5399 * 0.4601 = 0.2484
dL/dz1 = dL/dh * dh/dz1 = (-0.0946)(0.2484) = -0.0235
Step 5 - into the hidden weights and bias
dz1/dw1 = x1 = 0.5 -> dL/dw1 = -0.0235 * 0.5 = -0.0117
dz1/dw2 = x2 = 0.8 -> dL/dw2 = -0.0235 * 0.8 = -0.0188
dz1/db1 = 1 -> dL/db1 = -0.0235 * 1 = -0.0235dL/dw1 was never computed directly — there is no simple textbook formula for "the derivative of a squared-error loss, through two sigmoids, with respect to a first-layer weight." Instead it fell out of five short multiplications: dL/dy_hat, dy_hat/dz2, dz2/dh, dh/dz1, and dz1/dw1, chained together. Each of those five factors is a derivative of one simple, elementary operation — a multiply, an add, or a sigmoid — with respect to its immediate input. That is the chain rule's real power: it turns one intractable derivative into a product of easy ones, provided every intermediate quantity created during the forward pass is kept around.
Updating the Weights
Each of the five numbers just computed is a gradient: how much the loss would change if that one parameter increased slightly, holding everything else fixed. Gradient descent uses these gradients to improve the network — move every parameter a small step in the opposite direction of its gradient, scaled by a learning rate that controls the step size. With a learning rate of 0.1 (call it eta):
new value = old value - eta * gradient
w1: 0.6000 - 0.1*(-0.0117) = 0.6012
w2: -0.3000 - 0.1*(-0.0188) = -0.2981
w3: 0.9000 - 0.1*(-0.0567) = 0.9057
b1: 0.1000 - 0.1*(-0.0235) = 0.1023
b2: -0.2000 - 0.1*(-0.1051) = -0.1895Every gradient here was negative, so every parameter moved up — which makes sense, since the true outcome (y = 1) was higher than the prediction (0.5710), and increasing any one of these five parameters pushes the prediction up too. Running the forward pass again with the updated numbers sends y_hat to about 0.5746 — still far from a confident prediction, but a genuine step closer to the truth after a single example. Repeat this predict-compare-backpropagate-update cycle across thousands of ball-by-ball situations from real matches, and the weights stop being arbitrary starting numbers and start encoding something like cricket knowledge: a high required run rate with few wickets in hand pulls the prediction down; a settled chase with wickets in hand still standing pushes it up.
Why We Don't Do This by Hand: Automatic Differentiation
Five parameters is comfortable by hand. A small image classifier can have a few million. A modern large language model can have tens of billions. Nobody sits down and derives billions of individual chain-rule expressions on paper, yet every one of those parameters gets an exact gradient at every single training step. The technique that makes this possible is automatic differentiation (often shortened to autodiff, or AD): a way for software to compute exact derivatives of any function built from elementary operations, by mechanically applying the chain rule across a computational graph — a record of every elementary operation used to compute an output, and which earlier values fed into it — the same way the five steps above were worked out by hand.
One naive alternative is numerical differentiation: nudge a single parameter by a tiny amount, rerun the forward pass, and estimate the slope from how much the output changed, roughly (f(w+h) - f(w-h)) / (2h). This works, but it only estimates the gradient with respect to one parameter per pair of forward passes. For a model with a billion parameters, that means roughly two billion forward passes just to get one gradient step — hopelessly slow, and only approximate, sensitive to the choice of h. Automatic differentiation avoids both problems, and comes in two flavours. Forward-mode AD propagates derivatives alongside values as the computation runs forward, and is efficient when a function has few inputs and many outputs. Reverse-mode AD does the opposite: run the computation forward once, remembering every intermediate value, then sweep backward exactly once, applying the chain rule at each step to send the loss's gradient to every parameter — precisely the five-step walk done by hand above. Reverse-mode AD computes the gradient with respect to every parameter in roughly two to three times the cost of a single forward pass — a constant factor, completely independent of how many parameters there are. Since neural networks have exactly this shape, millions or billions of parameters feeding into one scalar loss, reverse-mode AD is the mode deep learning relies on. It has another, more familiar name: backpropagation, popularized for training neural networks in a widely cited 1986 paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams, though the underlying mathematical idea has roots in earlier optimization and control-theory research.
A Tiny Autodiff Engine in Python
The five-step backward walk from earlier can be written as a program instead of a page of arithmetic. The idea: wrap every number in an object that remembers which operation created it and from which inputs, so that once the final loss is computed, the program can walk that record backward and apply the chain rule automatically.
import math
class Value:
"""A single number that remembers how it was computed,
so gradients can flow back through it automatically."""
def __init__(self, data, children=(), op=""):
self.data = data
self.grad = 0.0
self._backward = lambda: None
self._prev = children
self._op = op
def __add__(self, other):
other = other if isinstance(other, Value) else Value(other)
out = Value(self.data + other.data, (self, other), "+")
def _backward():
self.grad += out.grad
other.grad += out.grad
out._backward = _backward
return out
def __mul__(self, other):
other = other if isinstance(other, Value) else Value(other)
out = Value(self.data * other.data, (self, other), "*")
def _backward():
self.grad += other.data * out.grad
other.grad += self.data * out.grad
out._backward = _backward
return out
def __pow__(self, power):
out = Value(self.data ** power, (self,), f"**{power}")
def _backward():
self.grad += (power * self.data ** (power - 1)) * out.grad
out._backward = _backward
return out
def __neg__(self):
return self * -1
def __sub__(self, other):
return self + (-other)
def sigmoid(self):
s = 1 / (1 + math.exp(-self.data))
out = Value(s, (self,), "sigmoid")
def _backward():
self.grad += s * (1 - s) * out.grad
out._backward = _backward
return out
def backward(self):
topo, visited = [], set()
def build(v):
if v not in visited:
visited.add(v)
for child in v._prev:
build(child)
topo.append(v)
build(self)
self.grad = 1.0
for v in reversed(topo):
v._backward()Each operator method does two jobs: it computes the ordinary forward result, and it attaches a small _backward function that knows the local derivative for that one operation — exactly the "dz2/dh = w3" style facts used in the hand trace. The backward() method first orders every node so that none is processed before everything it feeds into (a topological sort of the computation graph), then walks that order in reverse, calling each node's stored local derivative and accumulating results into .grad with +=, because a value can feed into more than one later computation, and its incoming gradients must add up.
Now rebuild the exact win-probability network and let the engine handle the backward pass on its own:
x1, x2 = Value(0.5), Value(0.8)
w1, w2, b1 = Value(0.6), Value(-0.3), Value(0.1)
w3, b2 = Value(0.9), Value(-0.2)
y = Value(1.0)
z1 = w1 * x1 + w2 * x2 + b1
h = z1.sigmoid()
z2 = w3 * h + b2
y_hat = z2.sigmoid()
diff = y_hat - y
loss = diff ** 2 * 0.5
loss.backward()
print(f"w1.grad = {w1.grad:.4f}") # -0.0117
print(f"w2.grad = {w2.grad:.4f}") # -0.0188
print(f"w3.grad = {w3.grad:.4f}") # -0.0567
print(f"b1.grad = {b1.grad:.4f}") # -0.0235
print(f"b2.grad = {b2.grad:.4f}") # -0.1051Every printed value matches the hand-computed gradients exactly. The program was never told any sigmoid derivative formula directly, nor which weight feeds which neuron — it only knew how to differentiate three elementary operations (add, multiply, sigmoid) and how to chain them together correctly. That is automatic differentiation working exactly as intended: the chain rule, applied mechanically and exactly, however deep or tangled the computational graph becomes.
From Fifty Parameters to Fifty Billion
Every major deep learning framework is built around this exact pattern, engineered for speed and scaled up enormously. PyTorch's tensors track operations the same way the Value class above does, and calling .backward() on a loss tensor triggers the identical reverse traversal. TensorFlow's GradientTape and JAX's grad follow the same reverse-mode idea through different interfaces. One detail carries over directly from the toy engine above: because gradients accumulate with += rather than overwrite, PyTorch parameters keep adding new gradients onto old ones across multiple backward passes unless they are explicitly cleared with optimizer.zero_grad() before each new step. Forgetting that line is a well-known early stumbling block for anyone learning PyTorch — and it is easy to see exactly why it happens, since it is the same accumulation rule visible in the _backward closures above.
Because reverse-mode automatic differentiation is exact, not approximate like the finite-difference method described earlier, engineers occasionally use finite differences for the opposite purpose: as a sanity check called gradient checking, comparing an autodiff-computed gradient against a slow numerical estimate on a small test case to catch bugs in a newly written operation. It is worth noting, too, that the squared-error loss used above was chosen because every step stays visible; real classifiers usually pair a sigmoid or softmax output with cross-entropy loss instead, and that pairing has a well-known property — the gradient flowing into the output layer's pre-activation simplifies to exactly y_hat - y, with the messy sigmoid derivative cancelling out algebraically. That is one reason the pairing is close to universal in classification networks.
Back to the Scoreboard
Return to that live win-probability bar. It is not one tiny two-input network trained on a single example — it is a model with far more inputs (overs bowled, venue, batting depth, recent form) and probably several hidden layers, trained on ball-by-ball data from thousands of matches. But the training loop underneath is not conceptually different from the one traced by hand above: predict, compare against the real outcome, apply the chain rule backward through every layer to find each parameter's share of the blame, nudge every parameter a small step, repeat. The same loop, unchanged in principle, trains the fraud-detection model that flags an unusual UPI payment before it clears, the demand-forecasting model a travel or e-commerce platform uses to plan capacity, and the large language models behind modern AI assistants — the last of these simply running the loop an astronomical number of times, across an astronomical number of parameters, with reverse-mode automatic differentiation keeping every single step tractable. Trace five gradients by hand through two sigmoids once, and the essential mechanism behind all of them stops being a mystery — the only thing that changes at scale is who does the arithmetic: not a person with a pencil, but the chain rule, wired directly into the software.
Think About It
Think about this: How would you explain chain rule and automatic differentiation to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind chain rule and automatic differentiation, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.