Neuromorphic Computing: Brain-Inspired Architectures

Training Spiking Neural Networks: Surrogate Gradients and BPTT

In 2017, an IBM research team paired a 128×128-pixel Dynamic Vision Sensor — a camera that reports only per-pixel brightness changes, not frames — with a TrueNorth neuromorphic chip and trained it to recognize eleven hand gestures in real time, drawing under 200 milliwatts (Amir et al., 2017, CVPR). The chip side of that story — why spiking hardware is so power-efficient, how a neuron core like Loihi's routes spikes instead of multiplying dense matrices — is the subject of this course's companion chapter on neuromorphic chips. This chapter answers the question that story leaves open: how do you get a network of spiking neurons to actually learn to tell "swipe left" from "clockwise circle" with competitive accuracy? Spike-timing-dependent plasticity (STDP), the local, unsupervised, biologically literal learning rule you may already know, cannot get you there on its own. The gesture-recognition result — and essentially every strong result on neuromorphic benchmarks since — comes from a different technique: training spiking networks with backpropagation, despite the fact that spikes look like they should make backpropagation impossible.

Why you can't just call backward() on a spike

A spiking neuron fires when its membrane potential V crosses a threshold V_th. Formally, the spike is a Heaviside step function of the distance to threshold:

S = H(V - V_th) = 1 if V >= V_th else 0

Gradient-based learning needs dS/dV, the sensitivity of the output to a small change in membrane potential. For the Heaviside function, that derivative is exactly zero everywhere except at V = V_th, where it is an infinite spike (a Dirac delta). Plugging this into the chain rule is fatal in two ways at once: almost every neuron, almost all the time, reports "changing your weights has zero effect on my output" — even though changing the weights by enough clearly does change whether the neuron fires. And at the single instant where the true derivative is nonzero, it is infinite, which is equally useless in a numerical gradient step. A network trained this way never updates, because the gradient is dead almost everywhere and at the one point that matters. This is a categorically different failure from the vanishing-gradient problem you met in deep feedforward and recurrent networks, where the gradient shrinks but stays defined. Here, for all but a measure-zero set of membrane potentials, there is no gradient information to propagate at all.

The surrogate gradient trick

The fix, formalized by Bohte, Kok and La Poutré's SpikeProp (2002) and made practical at scale by Zenke and Ganguli's SuperSpike (2018) and surveyed comprehensively by Neftci, Mostafa and Zenke (2019, IEEE Signal Processing Magazine), is to use two different functions for the same operation, one for each direction of the computation. The forward pass keeps the true Heaviside step — the network still emits real, binary, sparse spikes, identical to what would run on neuromorphic silicon. Only the backward pass, which exists purely inside the training procedure and never touches the deployed chip, substitutes a smooth proxy for the derivative. A common choice, the one SuperSpike introduced, is a scaled derivative of a fast sigmoid:

sigma'(x) = 1 / (1 + k*|x|)^2,   where x = V - V_th

k controls how sharply the surrogate is peaked around threshold. This function is nonzero everywhere, symmetric, and looks roughly like the true derivative's shape (a bump centered on threshold) without its two pathologies: it never collapses to exactly zero, and it never blows up to infinity. Training substitutes sigma'(V - V_th) for dS/dV only when computing gradients; the forward-pass spike itself is untouched. This is not cheating or approximating the network's behavior — the function the hardware executes is still the exact step function. It is a deliberate mismatch, forward function and backward derivative disagreeing on purpose, so that a learning signal can exist at all.

Worked example: backpropagation through time on a leaky integrate-and-fire neuron

Take a single leaky integrate-and-fire (LIF) neuron with leak beta = 0.9, threshold V_th = 1.0, weight w = 0.6, and a reset-to-zero rule applied by multiplying the carried voltage by (1 - S[t-1]). Drive it with the input spike train x = [1, 0, 1, 1] over four timesteps, V[0] = 0:

V[t] = beta * V[t-1] * (1 - S[t-1]) + w * x[t]
S[t] = 1 if V[t] >= V_th else 0

Stepping through by hand:

t=1: V = 0.9*0*(1-0) + 0.6*1 = 0.600   -> S=0
t=2: V = 0.9*0.600*(1-0) + 0.6*0 = 0.540 -> S=0
t=3: V = 0.9*0.540*(1-0) + 0.6*1 = 1.086 -> S=1 (fires!)
t=4: V = 0.9*1.086*(1-1) + 0.6*1 = 0.600 -> S=0

The neuron fires exactly once, at t=3. Now suppose the target behavior is silence — zero spikes over the window — with squared-error loss L = 0.5*(sum(S) - 0)^2. Since the network actually produced one spike, L = 0.5*(1)^2 = 0.5, and we want dL/dw to know which way to push the weight.

By the chain rule, dL/dw = sum_t (dL/dS[t]) * (dS[t]/dV[t]) * (dV[t]/dw). Each factor is computed separately:

dL/dS[t]: since L depends on the sum of spikes, dL/d(sum S) = (sum S - target) = 1, and because the sum is linear in each S[t], dL/dS[t] = 1 for every t.

dS[t]/dV[t]: the surrogate derivative 1/(1+5|V[t]-1.0|)^2, evaluated at each timestep's actual membrane potential (k=5):

t=1: V=0.600, |V-1|=0.400 -> sigma' = 1/(1+5*0.400)^2 = 1/9.00   = 0.1111
t=2: V=0.540, |V-1|=0.460 -> sigma' = 1/(1+5*0.460)^2 = 1/10.89  = 0.0918
t=3: V=1.086, |V-1|=0.086 -> sigma' = 1/(1+5*0.086)^2 = 1/2.0449 = 0.4890
t=4: V=0.600, |V-1|=0.400 -> sigma' = 1/9.00               = 0.1111

Notice t=3, the timestep that actually spiked, gets by far the strongest gradient signal (0.489 vs. ~0.11) — because it sat closest to threshold. This is exactly the useful property a surrogate is supposed to have: neurons near the decision boundary get the loudest learning signal, matching the intuition that these are the "close calls" worth adjusting.

dV[t]/dw: this is where the recurrence matters, and where implementations make a deliberate simplification. Differentiating the update rule naively would make dV[t]/dw depend on dS[t-1]/dw through the reset term (1-S[t-1]) — a second, competing gradient path through the same spike nonlininearity that empirically destabilizes training. Standard practice, documented as the default in the snnTorch library (Eshraghian et al., 2023, Proc. IEEE), is to detach the reset term: treat S[t-1] as a constant (its already-computed forward-pass value) when differentiating, rather than differentiating back through it. With that convention:

dV[t]/dw = beta*(1-S[t-1]) * dV[t-1]/dw + x[t],   dV[0]/dw = 0

t=1: dV/dw = 0.9*(1-0)*0     + 1 = 1.000
t=2: dV/dw = 0.9*(1-0)*1.000 + 0 = 0.900
t=3: dV/dw = 0.9*(1-0)*0.900 + 1 = 1.810
t=4: dV/dw = 0.9*(1-1)*1.810 + 1 = 1.000   <- history wiped by the reset

The jump at t=4 is the important, non-obvious part. Because the neuron spiked at t=3, the reset gate (1-S[3]) is exactly zero, and it multiplies away all of dV[3]/dw = 1.81 — three timesteps of accumulated gradient history vanish at that instant, and dV[4]/dw starts over from just the direct input term. A spike doesn't just reset voltage; under the detached-reset convention, it also severs the gradient's memory of everything before it. This is a genuine, hardware-relevant consequence of event-driven dynamics on the training side, not a simplification for the sake of the example.

Combining all three factors per timestep and summing:

dL/dw = 1*0.1111*1.000 + 1*0.0918*0.900 + 1*0.4890*1.810 + 1*0.1111*1.000
      = 0.1111 + 0.0826 + 0.8851 + 0.1111
      = 1.190

The gradient is positive, meaning gradient descent (w <- w - eta*dL/dw) will decrease w — exactly right, since a smaller weight makes the membrane potential lower and less likely to cross threshold, moving the network toward its target of silence. The sign checks out even though the computation used an approximate derivative at every step.

Implementing it: a custom autograd function

Every major SNN training library — snnTorch, Norse, Lava-DL — implements exactly this forward/backward split using a custom autograd function. In PyTorch, that means overriding forward (which computes the real step function and is what actually runs) and backward (which computes the surrogate and is what only training ever sees):

import torch

class SurrGradSpike(torch.autograd.Function):
    @staticmethod
    def forward(ctx, v_minus_vth):
        ctx.save_for_backward(v_minus_vth)
        return (v_minus_vth >= 0).float()   # true Heaviside

    @staticmethod
    def backward(ctx, grad_output):
        (v_minus_vth,) = ctx.saved_tensors
        k = 5.0
        surrogate = 1.0 / (1.0 + k * v_minus_vth.abs()) ** 2
        return grad_output * surrogate       # surrogate derivative

spike_fn = SurrGradSpike.apply

beta, V_th = 0.9, 1.0
w = torch.tensor(0.6, requires_grad=True)
x = torch.tensor([1.0, 0.0, 1.0, 1.0])

V, S_prev, spikes = torch.tensor(0.0), torch.tensor(0.0), []
for t in range(4):
    V = beta * V * (1 - S_prev.detach()) + w * x[t]
    S = spike_fn(V - V_th)
    spikes.append(S)
    S_prev = S

loss = 0.5 * (torch.stack(spikes).sum() - 0.0) ** 2
loss.backward()
print(w.grad.item())

Running this prints 1.1899958848953247 — matching the hand-derived 1.190 to floating-point precision, because the code implements exactly the same recurrence, the same detached reset, and the same surrogate we traced by hand. The .detach() call on S_prev when it feeds back into the voltage update is the line that enforces the detached-reset convention; deleting it would let gradient flow back through the reset gate and change the answer.

Why training doesn't happen on the neuromorphic chip itself

There is an asymmetry worth naming explicitly: inference on a spiking network is event-driven and can run on Loihi-class hardware, but training it with surrogate gradients, as done above, is not event-driven at all. Backpropagation through time unrolls the recurrence across every timestep — the code above keeps V and every intermediate spike in memory for the full window before loss.backward() can run a single step backward. For a real network with hundreds of neurons over hundreds of timesteps, that is the same O(timesteps × neurons) activation-memory cost that made truncated BPTT necessary for ordinary RNNs, and it is why practical surrogate-gradient training is done densely, on GPUs, exactly like training an RNN — not on the sparse, asynchronous, milliwatt-scale chip the network will eventually run on. This is a genuine open problem, not a footnote: it means the chip's efficiency advantage applies only after training finishes. One proposed way around it is eligibility propagation (Bellec et al., 2020, Nature Communications), which approximates the same gradient using only local, forward-running eligibility traces per synapse — avoiding backward-through-time entirely, at the cost of being an approximation to the true BPTT gradient rather than an exact match to it. Whether that approximation is close enough matters directly for whether a chip could ever learn online, on-device, the way it infers.

A misconception worth correcting directly

It is natural, having learned that STDP is "how biological synapses learn" and that spiking networks are meant to be biologically inspired, to assume STDP is also how state-of-the-art spiking networks are trained in practice. It mostly is not. STDP is a local, unsupervised rule: each synapse adjusts itself based only on the relative timing of its own pre- and post-synaptic spikes, with no access to a global error signal or a specific target output. That locality is exactly what makes it biologically plausible and hardware-cheap to implement on-chip — and exactly what makes it weak at solving a specific, hard classification objective like "these ten neurons, one per gesture, should each fire only for their own gesture." The competitive results on real neuromorphic benchmarks — DVS-Gesture, N-MNIST, CIFAR10-DVS — are overwhelmingly produced by supervised training with surrogate gradients, computed with a real, chosen loss function and backpropagated end to end, exactly as this chapter derives. STDP remains an active research direction for online, label-free, on-chip adaptation, but "STDP" and "how you train a spiking network" are not synonyms, and conflating them is the single most common conceptual error at this stage.

Active recall

Attempt each question before reading its answer. Question 3 modifies a parameter used throughout the worked example above — trace every value that changes, not just the total.

1. Why is dS/dV for a true spiking neuron unusable for gradient descent, even though it is technically well-defined at one point?
2. In the surrogate sigma'(x) = 1/(1+k|x|)^2, what happens to the gradient at t=3 (near threshold) versus t=1 (far from threshold), and why is that the desired behavior rather than an artifact?
3. Suppose w increases from 0.6 to 0.7 (everything else unchanged). Recompute the forward pass, the spike train, and dL/dw. Does the spike train change? Does the gradient?
4. What is dL/dw if the target spike count is 1 instead of 0, keeping w=0.6?
5. If the surrogate steepness k is increased from 5 to 20 (a sharper, more "accurate" approximation of the true Heaviside derivative), does |dL/dw| at w=0.6 go up or down? What does that imply about making the surrogate "more correct"?
6. Why does gradient-based training of a spiking network require far more memory during training than the deployed chip uses during inference?

Answers

1. The true derivative of the Heaviside step is zero everywhere except exactly at V = V_th, where it is an /infinite spike (a Dirac delta). A near-zero derivative tells the optimizer "this weight has no effect," which is false — nudging the weight enough clearly changes whether the neuron fires, the derivative just cannot see that because it only measures an infinitesimal change. And the one point where the derivative is technically nonzero gives an infinite, unusable step. Both failure modes make the exact derivative useless for numerical optimization, which is why a surrogate is substituted only for this one operation.

2. At t=3, V=1.086 is only 0.086 away from V_th=1.0, so sigma' = 1/(1+5*0.086)^2 = 0.489 — large. At t=1, V=0.600 is 0.400 away, giving sigma' = 1/(1+5*0.400)^2 = 0.111 — over four times smaller. This is desired: a neuron sitting right at the decision boundary is the one whose behavior a small weight change is most likely to actually flip, so it should carry the strongest learning signal. A neuron sitting far from threshold (comfortably not-firing or comfortably firing) is less sensitive to a small weight nudge, and the surrogate correctly reflects that.

3. Recomputing forward with w=0.7: V=[0.700, 0.630, 1.267, 0.700], giving the same spike train S=[0,0,1,0] — the neuron still fires only at t=3, because 0.630 is still below threshold and 1.267 still clears it. So the forward behavior is unchanged. But the gradient is not: dV/dw at each step is unchanged in form ([1.0, 0.9, 1.81, 1.0], since that recurrence doesn't depend on w's value directly) — the ripple instead lands entirely on the surrogate terms, because V[3] moved from 1.086 to 1.267, now 0.267 past threshold instead of 0.086: sigma'(t=3) = 1/(1+5*0.267)^2 = 0.183, down from 0.489. The other three surrogate values also shift slightly (t=1,4: 0.160; t=2: 0.123). Recombining: dL/dw = 1*0.160*1 + 1*0.123*0.9 + 1*0.183*1.81 + 1*0.160*1 = 0.160+0.111+0.332+0.160 = 0.763. The gradient shrank by roughly 36% (1.190 to 0.763) with the output completely unchanged — a concrete demonstration that surrogate-gradient magnitude can drift a great deal even while forward accuracy stays flat, purely because the surrogate function saturates the further a neuron's voltage sits from threshold.

4. With target 1 and an actual spike count of 1, the loss is already at its minimum: L = 0.5*(1-1)^2 = 0, so dL/d(sum S) = (1-1) = 0, and every per-timestep gradient term is multiplied by zero. dL/dw = 0 exactly, regardless of the surrogate or the dV/dw values — there is no learning signal, correctly, because the network already does what was asked.

5. At k=20, recomputing the surrogate values gives roughly [0.012, 0.010, 0.135, 0.012] at the same four voltages, all noticeably smaller than the k=5 values, and the total gradient drops to about 0.278 — less than a quarter of the k=5 result. A sharper k makes the surrogate a closer approximation to the true Heaviside derivative, but the true derivative is exactly the pathological function this whole technique exists to avoid (zero almost everywhere, infinite at one point). So making the surrogate "more accurate" pushes it back toward being useless, reintroducing the vanishing-gradient problem it was invented to solve. The steepness k is therefore a genuine hyperparameter trading fidelity against trainability, not a knob to be maximized.

6. Backpropagation through time requires the full forward trajectory — every timestep's membrane potential and spike — to be kept in memory before a single backward step can be computed, because the backward pass walks the recurrence in reverse. That memory scales with the number of timesteps times the number of neurons, exactly as it does for an ordinary RNN trained with BPTT. Inference on deployed neuromorphic hardware needs none of this: each neuron only needs its own current membrane potential and reacts to incoming spikes as they arrive, discarding history it doesn't need. The efficiency neuromorphic chips are built for is therefore an inference-time property; the training procedure that produces the weights currently costs GPU memory and dense compute proportional to sequence length, which is precisely the gap approaches like eligibility propagation (Bellec et al., 2020) attempt to close.

Think About It

Think about this: How would you explain neuromorphic computing: brain-inspired architectures to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind neuromorphic computing: brain-inspired architectures, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.