A Fraud Check That Has to Run in Your Pocket
Suppose a UPI app wants to catch a fraudulent payment the instant you tap "Pay," before the money has actually left your account. Sending every transaction to a distant server and waiting for a fraud score to come back would add a noticeable delay, and in a small town where the mobile signal drops to 2G near a shop's concrete counter, that round trip might not even complete. So the app ships a small neural network inside itself: a model that runs directly on the phone, scores the transaction in a few milliseconds using signals like the amount, how recently you last paid this payee, and whether you are far from your usual location, and only then lets the payment through.
That plan runs into a wall the moment it meets a real device. Plenty of smartphones sold in India, especially in the budget and entry-level segments, still ship with 3 to 4 GB of RAM, shared across a browser, a couple of social apps, the camera, and the operating system itself. A neural network with a few hundred thousand parameters, stored as 32-bit numbers, can easily weigh several megabytes. That is small by desktop standards, but big enough to make an app slower to install, slower to open, and noticeably heavier on the battery every time it runs. The model was trained on powerful GPUs with no such constraints, so it was never designed to be small. It was designed to be accurate. Making it small enough to live comfortably on a modest phone, without giving up the accuracy it took so much data and compute to earn, is the problem this chapter solves.
Why Trained Networks Carry Dead Weight
Every layer in a neural network computes a weighted sum. Given inputs x1, x2, x3, ... and learned weights w1, w2, w3, ..., a single neuron computes w1*x1 + w2*x2 + w3*x3 + ... + b and passes the result through an activation function. Training adjusts every one of these weights, layer after layer, so the network's final output matches the correct answer as closely as possible across thousands or millions of examples.
Training does not, however, try to use the fewest weights possible. It only tries to minimize error, using whatever architecture it was handed. Most modern architectures are deliberately built with far more weights than the training data strictly needs, because over-provisioning makes optimization easier. Gradient descent finds a good solution more reliably in a large, flexible network than in a small, cramped one. The network ends up over-parameterized: it has many more weights than are actually doing useful work. After training, if you inspect the weight values layer by layer, a large fraction of them turn out to be very close to zero. A weight near zero contributes almost nothing to the weighted sum, no matter what the input is. The connection still exists, and it still costs memory and a multiply-add operation at inference time, but it barely changes the answer.
This is not a strange coincidence. Something similar happens inside the human brain, which is where the vocabulary of "neurons" and "connections" in deep learning originally came from. A child's brain forms far more synaptic connections than it will end up needing. Through childhood and adolescence, the brain does not just add connections — it selectively removes the weak or rarely used ones, a process neuroscientists call synaptic pruning, while reinforcing the connections that get used often. The result, by early adulthood, is a brain that runs on less energy and reacts faster, not a worse one. Pruning in deep learning borrows this idea directly: train a large, flexible network first, then cut away the connections that turned out not to matter.
Defining Pruning
Pruning is the process of identifying weights, neurons, or filters in a trained network that contribute little to its output, and removing them — either by deleting them outright or by permanently setting them to zero — so that the resulting network is smaller and cheaper to run, while its accuracy stays as close as possible to the original.
The simplest and most widely used way to decide what "contributes little" means is magnitude-based pruning: rank every weight by its absolute value |w|, and remove the ones closest to zero first. The reasoning is direct. In the term w*x of a weighted sum, if |w| is tiny, that term stays small across the typical range of input values a trained network sees, so setting it to zero changes the neuron's output only slightly. A weight with a large magnitude, positive or negative, is doing real work in the computation and is far riskier to remove.
Magnitude is not the only criterion researchers use: some methods estimate how much the loss would increase if a given weight were removed, or track how consistently a weight matters across many training batches. But magnitude pruning is the natural starting point, because it needs nothing beyond the trained weights themselves: no extra computation, no retraining loop, just a sort.
Unstructured Pruning vs. Structured Pruning
Once you have decided which weights to remove, there are two very different ways to do the removal, and the difference matters enormously for whether the pruned network actually runs faster.
Unstructured pruning zeroes out individual weights, wherever they happen to fall inside a weight matrix, without any regard for rows or columns. The matrix keeps its original shape, but it becomes sparse, full of zeros scattered through it. This gives the finest possible control: you can remove the 30% of weights with the smallest magnitude across the whole network, no matter which neuron each one belongs to. The catch is that a standard GPU or CPU multiplies matrices as dense blocks. Unless the hardware or software specifically knows how to skip the zero entries, a sparse matrix full of scattered zeros still takes the same time to multiply as a dense one. You save storage, since a sparse matrix compresses well when saved to disk, but you do not automatically save computation time.
Structured pruning removes whole, regularly shaped chunks instead: an entire neuron (a full row or column of a weight matrix), an entire convolutional filter, or even an entire layer. Because the removed pieces line up with the matrix's actual structure, what remains is a smaller dense matrix that ordinary matrix multiplication understands immediately: a layer that had 512 output neurons and now has 400 is simply a smaller dense layer, and every standard library computes it faster with no special sparse-matrix support required. The tradeoff is that structured pruning is coarser: removing a whole neuron removes every one of its weights together, even the few that were individually large and useful, so it typically needs a larger pruning budget or more careful fine-tuning to reach the same accuracy as unstructured pruning at the same sparsity level.
In practice, teams choose based on where the model will run. With specialized sparse-matrix hardware or software support, unstructured pruning at very high sparsity gives the best accuracy-per-parameter tradeoff. Targeting an ordinary phone CPU with a standard inference library, structured pruning is usually what actually makes the app faster and lighter, because it shrinks the matrices themselves rather than just filling them with zeros.
A Worked Example: Pruning the Fraud-Detection Layer
Return to the on-device fraud model. Suppose its first hidden layer takes four normalized input features from a transaction:
x1= transaction amount, normalized to 0.9 (a large payment)x2= time since the last transaction, normalized to 0.1 (very recent — two payments in quick succession)x3= new-payee flag, 1.0 (this payee has never been paid before)x4= distance from the user's usual location, normalized to 0.7 (fairly far from home)
and feeds them into a hidden layer of three neurons, with this weight matrix and these biases:
Neuron h1: weights = [ 0.02, -0.01, 0.03, -0.04] bias = 0.10
Neuron h2: weights = [ 0.85, 0.02, 0.91, 0.78] bias = -0.20
Neuron h3: weights = [-0.03, 0.65, 0.05, 0.02] bias = 0.05Before touching anything, compute the original forward pass, using ReLU as the activation (ReLU keeps positive values as they are and clips negative values to zero):
h1 = 0.02(0.9) + (-0.01)(0.1) + 0.03(1.0) + (-0.04)(0.7) + 0.10
= 0.018 - 0.001 + 0.030 - 0.028 + 0.10 = 0.119 -> ReLU(h1) = 0.119
h2 = 0.85(0.9) + 0.02(0.1) + 0.91(1.0) + 0.78(0.7) + (-0.20)
= 0.765 + 0.002 + 0.910 + 0.546 - 0.20 = 2.023 -> ReLU(h2) = 2.023
h3 = -0.03(0.9) + 0.65(0.1) + 0.05(1.0) + 0.02(0.7) + 0.05
= -0.027 + 0.065 + 0.050 + 0.014 + 0.05 = 0.152 -> ReLU(h3) = 0.152Now apply magnitude-based unstructured pruning with a threshold of 0.05: every weight whose absolute value is below 0.05 gets set to zero. Checking each of the twelve weights against that threshold:
- Neuron h1: 0.02, -0.01, 0.03, -0.04 — all four fall below 0.05, so all four are pruned.
- Neuron h2: 0.85, 0.91, 0.78 survive; 0.02 is pruned.
- Neuron h3: 0.65 and 0.05 survive; -0.03 and 0.02 are pruned.
Seven of the twelve weights are removed, for a sparsity of 7/12 ≈ 58.3%. Recomputing the forward pass with the pruned weights (biases untouched):
h1_pruned = 0 + 0 + 0 + 0 + 0.10 = 0.100 (was 0.119)
h2_pruned = 0.765 + 0 + 0.910 + 0.546 - 0.20 = 2.021 (was 2.023)
h3_pruned = 0 + 0.065 + 0.050 + 0 + 0.05 = 0.165 (was 0.152)Cutting 58.3% of the weights in this layer moved each neuron's output by less than 0.02 in every case. That difference is small enough that, after a brief round of fine-tuning on real transaction data, the rest of the network can easily absorb it. This gap between "removed most of the weights" and "barely changed the answer" is the entire payoff of magnitude pruning: the weights that got cut genuinely were not doing much.
Notice something else about neuron h1: every one of its incoming weights was pruned, so its output no longer depends on the input at all. It always outputs ReLU(0.10) = 0.10, a constant. A neuron whose output never changes is not computing anything useful; it is only adding its fixed bias to whatever comes next. This is the signal structured pruning looks for: instead of leaving h1 as a "zeroed but still present" row, a structured pruning pass would delete neuron h1 entirely, folding its constant contribution into the next layer's bias, and shrink the hidden layer from three neurons to two. The weight matrix that follows this layer would then also shrink, from expecting three inputs to expecting two. That is a real reduction in the size of a matrix multiplication, not just a matrix with more zeros in it.
Pruning in Code
PyTorch exposes pruning through torch.nn.utils.prune. The function l1_unstructured implements the magnitude-based unstructured pruning worked through above: it ranks weights by their L1 norm, which for a single weight is just its absolute value, and zeroes out the smallest fraction requested.
import torch
import torch.nn as nn
import torch.nn.utils.prune as prune
torch.manual_seed(42)
# A tiny fraud-detection layer: 4 input features -> 3 hidden neurons
layer = nn.Linear(in_features=4, out_features=3)
total_weights = layer.weight.nelement()
print(f"Total weights before pruning: {total_weights}")
# Magnitude-based (L1) unstructured pruning: remove the 50% of
# individual weights with the smallest absolute value
prune.l1_unstructured(layer, name="weight", amount=0.5)
zero_weights = torch.sum(layer.weight == 0).item()
sparsity = 100.0 * zero_weights / total_weights
print(f"Weights pruned: {zero_weights} of {total_weights}")
print(f"Sparsity: {sparsity:.1f}%")
# layer.weight_mask holds the 0/1 mask that l1_unstructured applied
print("Mask:\n", layer.weight_mask)
# Bake the mask into the weight tensor permanently and drop the
# extra bookkeeping (weight_orig, weight_mask)
prune.remove(layer, "weight")nn.Linear(in_features=4, out_features=3) stores its weight as a (3, 4) tensor, one row of four weights per output neuron, matching the layer worked through by hand above, so total_weights always prints as 12. Because amount=0.5 is a fraction of that fixed count, the number of weights pruned is fully determined before the code even runs: 0.5 × 12 = 6, so the sparsity line always prints exactly Sparsity: 50.0%, regardless of which six weights the random initialization happens to make smallest. What is random is which weights get chosen, not how many.
Before prune.remove is called, the layer keeps two things side by side: weight_orig, the untouched original weights, and weight_mask, a tensor of 1s and 0s marking which weights survive. Every time the layer runs, PyTorch multiplies them together to get the effective weight. Keeping the mask separate like this makes it easy to see which connections were cut, or even undo the pruning, before prune.remove commits to it permanently by folding the mask into the weight tensor and deleting the extra bookkeeping.
Structured pruning uses a related function, ln_structured, which ranks and removes entire rows or columns instead of individual entries:
# Structured pruning removes whole neurons instead of individual weights.
# dim=0 targets output neurons (rows of the weight matrix); n=2 ranks
# each neuron by its L2 norm and removes the weakest one.
layer2 = nn.Linear(in_features=4, out_features=3)
prune.ln_structured(layer2, name="weight", amount=1, n=2, dim=0)
print(layer2.weight) # one full row is now entirely zeroHere amount=1 means "remove 1 neuron" out of the 3 in this layer, and dim=0 tells PyTorch that rows, not columns, are the units being compared. That is the same structural cut illustrated by hand with neuron h1 a moment ago. TensorFlow offers comparable tools through the TensorFlow Model Optimization Toolkit, so the same ideas transfer across frameworks.
Pruning Is Iterative, Not a One-Shot Cut
Removing a large fraction of a network's weights in a single pass, then never touching the network again, tends to hurt accuracy more than necessary. The standard recipe instead cycles through three steps: prune a modest fraction of weights, fine-tune the surviving weights by continuing to train the network for a few more epochs so they can compensate for what was removed, and then check accuracy on a held-out validation set. If the accuracy loss is acceptable and the target size has not been reached yet, the cycle repeats — prune a bit more, fine-tune again — until the model is small enough or accuracy starts dropping too fast.
This iterative approach usually reaches far higher sparsity than a single aggressive cut ever could. In a widely cited 2015 paper on network pruning, Song Han and colleagues at Stanford showed that AlexNet, an image classifier with roughly 61 million parameters, could be pruned down to under 7 million — about a 9-fold reduction — without any loss of accuracy on ImageNet, by pruning and fine-tuning repeatedly instead of all at once. VGG-16, a deeper network with around 138 million parameters, was reduced by roughly 13-fold using the same approach. A follow-up paper, "Deep Compression," layered two further compression steps on top of pruning (sharing weight values across clusters, then Huffman-coding the result) and reported shrinking AlexNet's stored size from 240 MB to about 6.9 MB, and VGG-16's from 552 MB to about 11.3 MB, again without losing accuracy.
A related and, at first, surprising finding is the lottery ticket hypothesis, put forward by Jonathan Frankle and Michael Carbin in 2019. They showed that inside a large, randomly initialized network there often exists a much smaller subnetwork which, if reset back to its original initial weights and trained by itself, reaches accuracy comparable to the full network in a similar number of training steps. They called this small subnetwork a "winning ticket." It was present from the very first random draw, and pruning is largely a way of finding it, not just a way of cleaning up after training. That reframes pruning: it is not simply deleting waste, it is uncovering a smaller architecture that was capable of learning the task all along.
How Much Can You Prune?
There is no fixed answer, and it depends heavily on how over-parameterized the starting network was. A large, deliberately over-built network like VGG-16 has enormous room to spare, so removing 90% or more of its weights with careful iterative fine-tuning can leave accuracy almost unchanged. A network that was already designed to be compact and efficient — the kind meant for phones in the first place — has far less slack, and aggressive pruning on top of it degrades accuracy much faster, because there was little redundancy left to remove.
The tolerance for pruning also varies unevenly within a single network: earlier layers and later layers, or different neurons within the same layer, respond very differently to being cut. That is why the fraud-detection example checked every one of the twelve weights individually rather than removing an arbitrary fixed number from each neuron: h1's weights all happened to be small together, while h2 kept three of its four. Good pruning always tracks accuracy on validation data as sparsity increases, rather than committing to a target percentage in advance and hoping it works out.
Check Your Understanding
- Why does removing a weight with a large positive value hurt accuracy more than removing one close to zero, even though both are "just one weight"?
- If a mobile inference library has no special support for sparse matrices, would unstructured pruning or structured pruning make the fraud-detection layer run faster on that phone? Why?
- In the worked example, neuron h3 kept a weight of exactly 0.05 while pruning one of -0.03. What does that tell you about how the 0.05 threshold was applied?
Back to the Phone in Your Pocket
Push the fraud-detection network through several rounds of pruning and fine-tuning — cutting individual low-magnitude weights where fine control matters, removing whole dead neurons like h1 where a smaller matrix matters more, and re-training in between each cut — and a model that once needed several megabytes and a noticeable pause can shrink dramatically, with accuracy nearly unchanged. That is the difference between a fraud check that only works well on a flagship phone with fast internet, and one that works identically for a shopkeeper on a modest device in an area where the network drops out entirely. UPI handles billions of transactions every month across India, on every tier of hardware at once; pruning is a large part of what lets the same intelligence run instantly on all of them, without ever leaving the phone.
Think About It
Think about this: How would you explain pruning: removing unnecessary weights to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind pruning: removing unnecessary weights, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.