The Payment That Should Never Have Been Blocked
It is the last week of the month. You are paying a shopkeeper ₹850 through UPI for groceries you buy every week, and instead of the usual confirmation, your phone shows "Transaction declined — flagged as suspicious." You try again. Declined again. Your account balance is fine, your phone has not changed, and this is a shop you have paid a dozen times before. Somewhere on a bank's server, a neural network looked at this transaction and decided, with high confidence, that it resembled fraud.
An engineer on the fraud team pulls up the logs the next morning and confirms the model made a mistake. Fixing it does not mean hand-writing a new rule such as "never block payments under ₹1000 to known merchants." It means showing the network this exact transaction, telling it the correct answer was not fraud, and letting the network adjust itself. But a production fraud-detection model can have millions of internal numbers called weights, arranged across dozens of layers. The transaction amount does not touch most of those weights directly — it passes through neuron after neuron before it ever influences the final decision. So when the network is told it got this one prediction wrong, how does it know which of its millions of weights to nudge, in which direction, and by how much?
That question — tracing a single wrong answer backward through every layer of a network to work out precisely how much each weight is to blame — is what backpropagation answers. It is not a vague idea or a loose metaphor. It is one tool from calculus, applied mechanically and repeatedly: the chain rule. By the end of this chapter you will have traced that blame by hand, number by number, through a small but real fraud-detection network, watched its prediction move measurably closer to the truth after a single correction, and seen the exact code that performs the same arithmetic.
What the Network Computes on the Way Forward
Before assigning blame, we need to be precise about how the network produced its answer in the first place — the forward pass. Every neuron in the network runs the same two-step routine. First, it takes a weighted sum of its inputs and adds a bias term: given inputs x1 and x2 with weights w1 and w2, the neuron computes z = w1*x1 + w2*x2 + b. Second, it passes z through an activation function, which bends that number into a bounded, non-linear output. Without this bending step, stacking layers would achieve nothing — a chain of purely linear operations always collapses back into one linear operation, no matter how many layers you stack.
This chapter uses the sigmoid function, one of the earliest activation functions used in neural networks and still a natural choice for a final layer that must output a probability: sigmoid(z) = 1 / (1 + e^(-z)). It squeezes any real number into the open interval between 0 and 1, which is just what we want from a neuron whose job is to output "probability this transaction is fraud." A large positive z pushes the output close to 1, a large negative z pushes it close to 0, and z = 0 gives exactly 0.5.
Once the network produces its final output, call it y_hat, we compare it to the true label y with a loss function — a single number measuring how wrong the prediction was. This chapter uses squared error: L = 0.5 * (y_hat - y)^2. The one-half in front looks arbitrary until you differentiate it: it exactly cancels the 2 that the power rule brings down, leaving the clean derivative dL/dy_hat = (y_hat - y), which is precisely where every backpropagation calculation in this chapter begins.
The Real Question: Whose Fault Is the Error?
Training a neural network means repeating one loop many times over: run a forward pass, measure the loss, adjust every weight slightly in the direction that reduces the loss, and repeat. That adjustment step is called gradient descent, and it needs one thing for every weight in the network: the gradient of the loss with respect to that weight — how much the loss would change if that one weight moved by a tiny amount, with everything else held fixed. In calculus terms this is a partial derivative, written dL/dw.
For a weight sitting in the very last layer, right next to the loss, this is easy to picture — you can almost see the weight's fingerprints on the answer. The difficulty is every weight several layers earlier. A weight in the first hidden layer of a deep network never touches the loss directly; its effect has to travel forward through every layer between it and the output before it can be measured. Computing each such gradient by brute force — nudging one weight slightly, rerunning the entire forward pass, and checking how much the loss moved — would demand a fresh forward pass for every single weight in the network, which is hopelessly slow once a network has millions of them. Backpropagation avoids that entirely: one forward pass followed by one backward pass produces the exact gradient for every weight at once.
The Chain Rule, One Link at a Time
The tool that makes this possible is one you likely already know from single-variable calculus: the chain rule. If y is a function of u, and u is itself a function of x, then dy/dx = (dy/du) * (du/dx). You differentiate the outer function with respect to the thing it directly depends on, differentiate the inner function with respect to x, and multiply the two results.
Take y = (3x + 1)^2. Let u = 3x + 1, so y = u^2. Then dy/du = 2u and du/dx = 3, giving dy/dx = 2u * 3 = 6(3x + 1) = 18x + 6. At x = 2, that is 18(2) + 6 = 42 — the same answer you get by expanding y = 9x^2 + 6x + 1 first and differentiating directly. The chain rule is not a shortcut that trades away accuracy; it is an exact restatement of ordinary differentiation, organized so you never have to expand a long composite expression before differentiating it.
A neural network's output is nothing but a long composition of functions: a weighted sum, through a sigmoid, into another weighted sum, through another sigmoid, and so on. Backpropagation is the chain rule applied to that composition, one link at a time, from the loss all the way back to every weight.
When One Change Travels Through Multiple Paths
Real networks add one complication the simple chain rule above does not cover: a single quantity often feeds forward into more than one path before those paths reunite. In the fraud-detection network we are about to build, the transaction-amount input feeds every neuron in the hidden layer, not just one. When several paths carry a variable's influence forward, you need the multivariable chain rule: sum the contribution of every path separately.
Concretely, if z depends on x and y, and both x and y depend on t, then dz/dt = (∂z/∂x)(dx/dt) + (∂z/∂y)(dy/dt) — one term per path, added together. Try z = x^2 y with x = 2t and y = t + 1, evaluated at t = 1. Direct substitution gives x = 2, y = 2, so z = (2^2)(2) = 8. Along the x-path, ∂z/∂x = 2xy = 2(2)(2) = 8, and dx/dt = 2, contributing 8 × 2 = 16. Along the y-path, ∂z/∂y = x^2 = 4, and dy/dt = 1, contributing 4 × 1 = 4. Summing the two paths gives dz/dt = 16 + 4 = 20. Check this by substituting first: z(t) = (2t)^2(t + 1) = 4t^3 + 4t^2, so dz/dt = 12t^2 + 8t, which at t = 1 gives 12 + 8 = 20. Both routes agree.
This sum-over-every-path rule is the second half of what makes backpropagation work. It is why a single weight deep inside a network can still receive a well-defined, exact gradient even when its influence on the loss fans out and recombines many times before it gets there.
The Computational Graph: Multiply Along a Path, Add Across Branches
It helps to picture the forward pass as a computational graph — a diagram where every operation (a multiplication, an addition, a sigmoid) is a node, and arrows show which values feed into which. Backpropagation walks this graph in reverse. At every node it computes a local gradient — the derivative of that one operation's output with respect to its own input, using only values already produced during the forward pass. To find the gradient of the loss with respect to any earlier quantity, multiply local gradients along the path connecting the two; where paths branch, add the results. These are the same two rules from the last two sections, applied mechanically at every node in the graph.
This is also what makes backpropagation efficient rather than merely correct. A network with many layers has an enormous number of distinct paths from an early weight to the final loss, and re-multiplying an entire path from scratch for every weight would take time that explodes with depth. Backpropagation avoids this by working backward one layer at a time and reusing an intermediate result — usually written delta (δ) — computed once at each neuron; every weight feeding into that neuron then reuses the same delta instead of recomputing it. This reuse is what turns an exponential problem into one costing roughly the same amount of arithmetic as the forward pass itself, and it is precisely the algorithm that David Rumelhart, Geoffrey Hinton, and Ronald Williams described in their influential 1986 paper, which showed for the first time that this method could train multi-layer networks to discover useful internal representations on their own.
A Complete Worked Example: Backpropagating Through a Fraud-Detection Network
Now trace all of this through actual numbers, using a deliberately small stand-in for the fraud-detection network: two inputs, one hidden layer of two neurons, and one output neuron.
- Inputs:
x1 = 0.6(a normalized transaction amount) andx2 = 0.9(a normalized risk score combining device and time-of-day signals) - Hidden neuron h1 receives x1 through weight w1 = 0.4 and x2 through weight w2 = 0.3, plus bias b1 = 0.1
- Hidden neuron h2 receives x1 through weight w3 = 0.2 and x2 through weight w4 = 0.5, plus bias b2 = 0.1
- Output neuron y_hat receives h1 through weight w5 = 0.5 and h2 through weight w6 = 0.6, plus bias b3 = 0.1
- True label: y = 0, because this transaction is genuine, not fraud
Forward pass. First, the pre-activation sums:
h1_in = 0.4(0.6) + 0.3(0.9) + 0.1 = 0.24 + 0.27 + 0.1 = 0.61
h2_in = 0.2(0.6) + 0.5(0.9) + 0.1 = 0.12 + 0.45 + 0.1 = 0.67
Passing these through the sigmoid function gives h1 = sigmoid(0.61) ≈ 0.6479 and h2 = sigmoid(0.67) ≈ 0.6615. These feed the output neuron:
y_in = 0.5(0.6479) + 0.6(0.6615) + 0.1 ≈ 0.8209
so y_hat = sigmoid(0.8209) ≈ 0.6944. The network is 69.4% confident this genuine transaction is fraud — the kind of false positive that blocked the payment at the start of this chapter. The loss is L = 0.5 × (0.6944 − 0)^2 ≈ 0.2411.
Backward pass, output layer. We need dL/dw5, dL/dw6, and dL/db3. Each is a product of local gradients along the path from L back to that weight. Start with the part every one of those paths shares, at the output neuron itself:
dL/dy_hat = y_hat − y = 0.6944 − 0 = 0.6944
dy_hat/dy_in = y_hat(1 − y_hat) = 0.6944 × 0.3056 ≈ 0.2122
This second line is the derivative of the sigmoid function, conveniently expressed using its own output. Multiplying these two local gradients gives the error signal at the output neuron's pre-activation, delta_out = 0.6944 × 0.2122 ≈ 0.1474. Every weight feeding into y_hat reuses this exact number:
dL/dw5 = delta_out × h1 = 0.1474 × 0.6479 ≈ 0.0955
dL/dw6 = delta_out × h2 = 0.1474 × 0.6615 ≈ 0.0975
dL/db3 = delta_out ≈ 0.1474 (a bias's local gradient with respect to its own pre-activation is always 1, so it simply inherits delta_out unchanged)
Backward pass, hidden layer. To reach w1 through w4, the gradient must pass through the hidden neurons first. In this small network, h1's only downstream connection is to y_hat through w5, so:
dL/dh1 = delta_out × w5 = 0.1474 × 0.5 ≈ 0.0737
dh1/dh1_in = h1(1 − h1) = 0.6479 × 0.3521 ≈ 0.2281
delta_h1 = 0.0737 × 0.2281 ≈ 0.0168
and symmetrically for h2, whose only downstream connection is through w6:
dL/dh2 = delta_out × w6 = 0.1474 × 0.6 ≈ 0.0884
dh2/dh2_in = h2(1 − h2) = 0.6615 × 0.3385 ≈ 0.2239
delta_h2 = 0.0884 × 0.2239 ≈ 0.0198
The weight gradients for the first layer now simply multiply each hidden neuron's delta by whichever input fed it:
dL/dw1 = delta_h1 × x1 = 0.0168 × 0.6 ≈ 0.0101
dL/dw2 = delta_h1 × x2 = 0.0168 × 0.9 ≈ 0.0151
dL/dw3 = delta_h2 × x1 = 0.0198 × 0.6 ≈ 0.0119
dL/dw4 = delta_h2 × x2 = 0.0198 × 0.9 ≈ 0.0178
Notice that h1 and h2 each have only one downstream path here, so no summing was needed yet. The sum-over-paths rule would appear immediately if you kept walking one step further back, to the input x1 itself, which feeds both h1 and h2: dL/dx1 = (delta_h1 × w1) + (delta_h2 × w3) = (0.0168 × 0.4) + (0.0198 × 0.2) ≈ 0.0107, the two contributions added together just as in the z(t) example earlier. A network with another layer behind x1 would repeat this same branching pattern for its weights, which is precisely why depth is not free: every extra layer means one more round of multiplying and summing local gradients before the earliest weights receive their instructions.
The update. With every gradient in hand, gradient descent nudges each parameter against its gradient, scaled by a learning rate η (eta). Using η = 0.5, the rule is w_new = w − η × (dL/dw). Applying this to all nine parameters gives w1 ≈ 0.3950, w2 ≈ 0.2924, w3 ≈ 0.1941, w4 ≈ 0.4911, w5 ≈ 0.4523, w6 ≈ 0.5513, b1 ≈ 0.0916, b2 ≈ 0.0901, and b3 ≈ 0.0263. Run the forward pass again with these updated numbers, and the network now outputs y_hat ≈ 0.6636 with loss L ≈ 0.2202 — both lower than before. One correction, one backward pass, and the network already looks less certain that a genuine ₹850 grocery payment is fraud.
Same Computation, In Code
Every number above comes directly from this program — no library, no shortcuts, just the chain rule written out as code:
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
# Inputs and true label
x1, x2 = 0.6, 0.9
y_true = 0.0
# Initial weights and biases
w1, w2, w3, w4 = 0.4, 0.3, 0.2, 0.5
b1, b2 = 0.1, 0.1
w5, w6 = 0.5, 0.6
b3 = 0.1
# ---- Forward pass ----
h1 = sigmoid(w1 * x1 + w2 * x2 + b1)
h2 = sigmoid(w3 * x1 + w4 * x2 + b2)
y_hat = sigmoid(w5 * h1 + w6 * h2 + b3)
loss = 0.5 * (y_hat - y_true) ** 2
print(f"h1={h1:.4f} h2={h2:.4f} y_hat={y_hat:.4f} loss={loss:.4f}")
# ---- Backward pass ----
delta_out = (y_hat - y_true) * y_hat * (1 - y_hat)
grad_w5 = delta_out * h1
grad_w6 = delta_out * h2
grad_b3 = delta_out
delta_h1 = delta_out * w5 * h1 * (1 - h1)
delta_h2 = delta_out * w6 * h2 * (1 - h2)
grad_w1 = delta_h1 * x1
grad_w2 = delta_h1 * x2
grad_b1 = delta_h1
grad_w3 = delta_h2 * x1
grad_w4 = delta_h2 * x2
grad_b2 = delta_h2
# ---- Gradient descent step ----
eta = 0.5
w1 -= eta * grad_w1; w2 -= eta * grad_w2
w3 -= eta * grad_w3; w4 -= eta * grad_w4
w5 -= eta * grad_w5; w6 -= eta * grad_w6
b1 -= eta * grad_b1; b2 -= eta * grad_b2; b3 -= eta * grad_b3
# ---- Confirm the loss dropped ----
h1 = sigmoid(w1 * x1 + w2 * x2 + b1)
h2 = sigmoid(w3 * x1 + w4 * x2 + b2)
y_hat = sigmoid(w5 * h1 + w6 * h2 + b3)
new_loss = 0.5 * (y_hat - y_true) ** 2
print(f"After one update: y_hat={y_hat:.4f} loss={new_loss:.4f}")
Running this prints h1=0.6479 h2=0.6615 y_hat=0.6944 loss=0.2411 on the first line and After one update: y_hat=0.6636 loss=0.2202 on the second — matching the hand calculation exactly. Notice how closely the code mirrors the mathematics: delta_out, delta_h1, and delta_h2 are each computed once and reused for every weight that needs them, which is the reuse trick that keeps backpropagation fast, written directly into the program's structure. Frameworks such as PyTorch and TensorFlow automate this same process — building the computational graph as the forward pass runs, then walking it backward — under a feature usually called autodiff or autograd, so in practice you rarely write these gradient formulas by hand for a real model.
Why Depth Makes Gradients Shrink
The hidden-layer gradients in the worked example (0.0168 and 0.0198) are noticeably smaller than the output-layer gradients (0.0955 and 0.0975), even though every local gradient involved is a perfectly ordinary number. This is not a coincidence, and it points to a genuine limitation of the sigmoid function: its derivative, sigmoid(z)(1 − sigmoid(z)), tops out at exactly 0.25, reached only when z = 0, and is smaller everywhere else. Every layer you walk backward through multiplies the running gradient by another one of these fractional local derivatives. Stack ten sigmoid layers, and even in the best case where every single one sits at its maximum of 0.25, the gradient reaching the earliest layer has been multiplied by roughly 0.25^10 — about one part in a million. In practice the shrinkage is usually worse, because neurons rarely sit exactly at z = 0.
This effect, known as the vanishing gradient problem, means the earliest layers of a deep sigmoid network learn extremely slowly — their instructions arrive as a signal so faint it barely moves the weights at all. Researchers in the early 1990s, most notably Sepp Hochreiter, identified this as a fundamental obstacle to training deep networks, and it is a major reason modern networks favor activation functions such as ReLU (f(z) = max(0, z)), whose derivative is a flat 1 for every positive input rather than a fraction shrinking toward zero. The chain rule itself never changes — it is exact, always — but the size of the numbers being multiplied along that chain determines whether the gradient reaching an early weight is still useful or has faded into numerical noise.
Back to the Blocked Payment
The single update traced above moved one small network's confidence from 69.4% fraud to 66.4% fraud on one transaction, after being told once that it was wrong. A real fraud-detection system repeats this exact loop — forward pass, loss, backward pass, update — across millions of labeled transactions, many times over, until false positives like the blocked ₹850 grocery payment become rare instead of routine. Nothing about that process is mysterious once you can see inside it: it is the chain rule, applied link by link from a single loss value back to every weight that contributed to it, multiplying local gradients along each path and summing wherever paths branch. Backpropagation does not guess which weights are at fault. It calculates it, exactly, every time — which is precisely why an engineer can trust the fix, and why the next payment like yours goes through.
Think About It
Think about this: How would you explain backpropagation: the calculus deep dive to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind backpropagation: the calculus deep dive, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.