Continual Learning: Learning Without Catastrophic Forgetting

A large private bank runs a transformer-based sequence model over UPI transactions to flag fraud in real time. Every few weeks a new scam pattern shows up — screen-sharing app fraud in one quarter, fake customer-care QR codes in the next, SIM-swap-triggered mule transfers after that. The model has to learn each new pattern without losing what it already knows about the old ones, because old scam patterns do not retire; they recur in waves. The obvious fix — keep a buffer of old fraudulent transactions and replay them while training on the new pattern — runs straight into RBI data-retention and data-minimisation rules: a bank cannot indefinitely warehouse raw customer transaction records purely to prevent a neural network from forgetting. A method that needs zero stored raw data, and still stops the new update from overwriting the old knowledge, is not a nice-to-have here — it is the only option regulation leaves on the table.

This chapter builds exactly such a method: orthogonal gradient descent via Gradient Projection Memory (GPM), introduced by Gobinda Saha, Isha Garg, and Kaushik Roy at ICLR 2021 ("Gradient Projection Memory for Continual Learning", arXiv:2103.09762, ICLR 2021 Oral). It does not touch the loss function, and it does not freeze any parameters or add new ones per task — both of those are other chapters' territory. It works one level lower, on the geometry of the gradient itself, at the exact moment before an optimiser step is taken.

Where interference actually comes from

Take the simplest possible layer: a single linear neuron with weight vector w ∈ ℝ², producing y = w·x for an input x ∈ ℝ². Task A trains this neuron until it settles on some w. Now task B arrives, gradient descent computes a gradient g from task B's loss, and applies w_new = w − ηg. What happens to the output on an old, task-A input x_old?

Δy = w_new·x_old − w·x_old = −η(g·x_old)

That single line is the entire mechanism of catastrophic forgetting in a linear layer: the output on an old input shifts exactly in proportion to how much the new gradient overlaps, via a dot product, with that old input direction. If g happens to be orthogonal to x_old, the old output does not move at all — regardless of how large η or ‖g‖ are. Forgetting is not really about "the weights changing" in the abstract; it is about the weight change having a nonzero component along directions the old task actually used. This is the fact GPM exploits directly, rather than fighting indirectly through a penalty term the way Elastic Weight Consolidation does.

Building the memory: SVD of activations, not storage of data

The question becomes: which directions in input/activation space did an old task actually use? GPM answers this empirically, per layer, without keeping a single training example. After finishing task A, run a modest sample of task-A inputs forward through the network one more time and collect the layer's input activations as columns of a matrix R_l (rows = feature dimension of layer l, columns = samples). Compute its SVD:

R_l = UΣV^T

The columns of U are ranked by how much of the activation energy (Σ of squared singular values) they explain. Keep the smallest set of leading columns u₁, …, u_k whose squared singular values sum to at least a threshold fraction ε_th of the total: this is the core gradient space, and it is stored as the memory basis M_l = [u₁ … u_k]. Note what is and is not stored: M_l is a d × k matrix of orthonormal directions — never a raw transaction, never a label, never anything traceable to an individual. This is the property that clears the UPI bank's regulatory bar: GPM's "memory" is a geometric summary, not a data cache, which is a different kind of object from the rehearsal buffer covered elsewhere in this course.

When task B arrives, its raw gradient at that layer, g_l, is projected before the optimiser ever sees it:

g_orth = g_l − M_lM_l^Tg_l

and the update uses g_orth in place of g_l. Because M_lM_l^T is the orthogonal projector onto the memory subspace, g_orth is guaranteed — by the algebra of projection, not by hoping a penalty term wins a tug-of-war — to be orthogonal to every direction in M_l. Combined with the Δy = −η(g·x_old) identity above, this means: for any old-task input direction captured in memory, the output cannot move, because the dot product that would move it is forced to zero by construction.

Worked example: projecting a gradient by hand

Layer l has a single output neuron, w ∈ ℝ². Task A produced two activation samples at this layer, x_a = (2, 2) and x_b = (1, −1) (chosen orthogonal to each other so their individual contributions stay separable). As columns of R:

R = [[2, 1], [2, −1]]

Its singular values are √8 ≈ 2.8284 and √2 ≈ 1.4142 (squared: 8 and 2, summing to 10 — the total activation energy). The top singular value alone captures 8/10 = 80% of the energy, with left singular vector u₁ = (1, 1)/√2. Set ε_th = 0.8: keep only u₁, so M = [u₁], a 2×1 matrix, and the projector is

MM^T = u₁u₁^T = [[0.5, 0.5], [0.5, 0.5]]

Task B's raw gradient at this layer is g = (3, 1). Project it:

g_proj = MM^Tg = [[0.5, 0.5], [0.5, 0.5]]·(3, 1) = (2, 2)
g_orth = g − g_proj = (3, 1) − (2, 2) = (1, −1)

Two coincidences worth noticing, both consequences of the numbers chosen, not general facts: g_proj lands exactly on x_a, because x_a = (2,2) is itself a pure multiple of u₁; and g_orth lands exactly on x_b. Now measure what actually happens to the old outputs, with η = 0.1. Using the naive (unprojected) update, the shift on x_a would be −η(g·x_a) = −0.1(6+2) = −0.8 — a large, uncontrolled change. Using g_orth instead: Δy_a = −η(g_orth·x_a) = −0.1(2 − 2) = 0 exactly, because x_a lies in the protected subspace. But check x_b, which was not kept in memory: Δy_b = −η(g_orth·x_b) = −0.1(1+1) = −0.2, a nonzero shift. GPM protected the 80%-energy direction perfectly and left the 20%-energy direction completely exposed. That gap is not a bug to be fixed later — it is the explicit price of the ε_th = 0.8 threshold, and it is the deliberate stability/plasticity dial the algorithm hands the engineer.

Tracing it in code

The following reproduces the hand derivation with NumPy's SVD, so the numbers can be checked rather than trusted.

import numpy as np
np.set_printoptions(suppress=True)

# Task A: two old-task activation samples at this layer (columns of R)
R = np.array([[2.0, 1.0],
              [2.0, -1.0]])          # x_a = (2,2), x_b = (1,-1)

U, S, Vt = np.linalg.svd(R)
energy = S**2
cum_ratio = np.cumsum(energy) / energy.sum()
print(np.round(S, 4))                # [2.8284 1.4142]
print(np.round(cum_ratio, 4))        # [0.8  1.  ]  -> u1 alone hits eps_th=0.8

k = 1
M = U[:, :k]                         # memory basis, shape (2,1)
P = M @ M.T                          # projector onto memory subspace
print(np.round(P, 4))                # [[0.5 0.5]
                                      #  [0.5 0.5]]

g = np.array([3.0, 1.0])             # task-B raw gradient at this layer
g_orth = g - P @ g
print(g_orth)                        # [ 1. -1.]

eta = 0.1
x_a, x_b = np.array([2.0, 2.0]), np.array([1.0, -1.0])
print(-eta * g.dot(x_a))             # -0.8   (naive update: old output moves)
print(-eta * g_orth.dot(x_a))        # -4.440892098500626e-16 (== 0, up to float roundoff)
print(-eta * g_orth.dot(x_b))        # -0.2   (unprotected direction still moves)

Every printed value matches the hand derivation, with one honest caveat on the third-to-last line: exact arithmetic gives Δy_a = 0, but floating-point SVD returns something like −4.44 × 10⁻¹⁶. That residual is 15 orders of magnitude below the naive update's −0.8 and is numerical noise from how numpy.linalg.svd computes U, not evidence that the projection failed. Also note that numpy.linalg.svd may return u₁ as −(1,1)/√2 instead of (1,1)/√2 depending on its internal sign convention — this never matters here, because MM^T = u₁u₁^T is unchanged if every entry of u₁ is negated. The projector, and therefore the whole method, is sign-invariant by construction.

Why "by construction" is a stronger claim than "by regularisation"

Elastic Weight Consolidation adds a quadratic penalty term λΣF_i(θ_i − θ*_i)² to the task-B loss, where F is an estimated importance weighting. It is a soft constraint: it competes with the task-B loss term inside the same scalar objective, and if the task-B gradient signal is large enough, or λ is tuned too low, the optimiser can still walk straight through it — nothing stops it, it just gets discouraged. GPM does not add anything to the loss at all. It intercepts the gradient after it has been computed and before the optimiser step is applied, and it removes the offending component entirely via a linear projection. There is no scalar tug-of-war for a large task-B gradient to win, because the removed component is exactly zero regardless of how large the rest of the gradient is — Δy_a was exactly 0 in the worked example above however large η or ‖g‖ get, since η(g_orth·x_a) stays 0 for any scalar η. This is a hard geometric guarantee for directions inside the memory subspace, not a statistical discouragement — and it is also a fundamentally different shape of guarantee from freezing a base model and routing each task through its own isolated low-rank adapter, which protects old tasks by never touching the shared weights at all rather than by constraining how they are allowed to move.

The guarantee does have real limits, and they are worth being precise about rather than glossing over. It is exact for a single linear layer's pre-activation. Once you stack layers with nonlinearities and a final cross-entropy loss, a per-layer zero-interference guarantee on pre-activations does not compose into an exact zero-interference guarantee on the final loss — a nonlinearity applied to a value that changed by a numerically tiny amount can still perturb what happens downstream, and errors from multiple layers can accumulate. In practice this residual is small when ε_th is set high enough, which is exactly why the GPM paper reports it as reducing forgetting to near-zero on their benchmark suite, not as an algorithm with a formally verified zero-forgetting bound end-to-end.

The tradeoff GPM cannot avoid: capacity saturation

Raising ε_th protects more of the old task, but every direction added to M_l is a direction removed from the space task B's gradient is allowed to move in. Push ε_th to 1.0 in the worked example — capturing 100% of the two-dimensional activation energy needs both singular vectors, so k = 2 and M becomes a full orthonormal basis of ℝ². Then MM^T = I, and g_orth = g − Ig = (0, 0) for any task-B gradient g at that layer: task B receives zero learning signal there. This is confirmed by running the same NumPy code with k = 2: P2 comes out as the identity matrix and g_orth2 comes out as [0. 0.]. In a real network, each layer's activation space has far more than 2 dimensions, so a handful of tasks rarely saturate it — but the space is finite, and every task after the first is spending some of it. This is GPM's version of the plasticity-stability tradeoff, and it is exactly the tradeoff a per-task LoRA adapter sidesteps by giving every new task fresh, isolated parameters instead of sharing — at the cost of that adapter chapter's own tradeoff, parameter count growing linearly with the number of tasks. GPM instead keeps the parameter count fixed and spends a shrinking, shared geometric budget.

Common misconception

The misconception students reach for almost every time: "GPM guarantees the network never forgets task A." It does not. It guarantees that the update does not move the output along whichever old-task directions were retained in memory — and that guarantee is exact only at the level of one linear layer's pre-activation, subject to the ε_th threshold choosing what counts as "retained." The worked example is the cleanest possible counter-proof: with ε_th = 0.8, the 20%-energy direction x_b was left completely unprotected and its output shifted by exactly −0.2η under the projected update, not zero. GPM does not eliminate forgetting; it converts an uncontrolled, unbounded interference into a bounded one, with the bound set by how much energy ε_th demands the memory capture and how many old-task directions the layer's activations actually spread across. Treating it as an absolute guarantee is the mistake that leads to skipping validation on old tasks after deployment — exactly the failure mode continual learning is supposed to prevent.

Active recall

Q1. In the worked example, recompute g_orth and the resulting Δy on x_a and x_b if task B's raw gradient had instead been g = (4, 0), with the same memory M = [u₁] and η = 0.1.

Q2. The chapter raised ε_th from 0.8 to 1.0 and found g_orth = (0, 0) for any g. Trace the full ripple of raising ε_th from 0.8 to exactly 0.99 in the two-dimensional worked example specifically (not the general statement) — what happens to k, to M, to the projector MM^T, to Δy on x_a and x_b, and to task B's remaining learning capacity at this layer?

Q3. A transformer block's projection layers have hidden dimension 768. GPM keeps k = 50 memory directions per layer across L = 12 layers. A rehearsal buffer alternative stores 10,000 raw examples of the same 768-dimensional activation. Compute the total stored floats for each approach and their ratio.

Q4. Why is the projector MM^T unaffected by NumPy (or any SVD routine) returning u₁ with a flipped sign?

Q5. Contrast GPM's failure mode when many tasks accumulate with a per-task LoRA adapter's failure mode under the same condition. Which one degrades old-task accuracy, and which one degrades something else entirely?

Q6. True or false, with justification: "Because M_lM_l^T is computed from activations, not gradients, GPM needs task A's original training labels to build its memory."

Worked answers

A1. g_proj = MM^Tg = [[0.5,0.5],[0.5,0.5]]·(4,0) = (2,2). g_orth = (4,0) − (2,2) = (2,−2). Check orthogonality: (2,−2)·(1,1) = 0, confirmed. Δy_a = −0.1·((2,−2)·(2,2)) = −0.1·(4−4) = 0, protected as expected. Δy_b = −0.1·((2,−2)·(1,−1)) = −0.1·(2+2) = −0.4, larger than the −0.2 seen with the original g, because this g's orthogonal remainder happens to have twice the magnitude along x_b's direction.

A2. The total energy is only 10 (8+2), so 0.99 of it requires both singular values (8/10=0.8 alone is insufficient, (8+2)/10=1.0 is the only way to clear 0.99) — k jumps from 1 to 2. M becomes the full 2×2 orthonormal matrix U, so MM^T = I (verified numerically: the projector comes out as the 2×2 identity, up to a −0.0 sign artifact from SVD). Consequently g_orth = g − Ig = (0,0) for every possible task-B gradient g at this layer — Δy is now exactly 0 on both x_a and x_b (full protection), but task B's weight at this layer receives literally zero update: its remaining learning capacity here has dropped to zero. Raising the threshold did not just change the memory and the protection level; it silently zeroed out task B's ability to learn anything new through this layer.

A3. GPM: 768 × 50 × 12 = 460,800 floats. Rehearsal buffer: 10,000 × 768 = 7,680,000 floats. Ratio: 7,680,000 / 460,800 ≈ 16.67×. GPM's memory costs roughly 1/17th the storage of the raw-exemplar alternative for this configuration, and — unlike the buffer — contains no example that could be traced back to an individual transaction.

A4. MM^T = u₁u₁^T for k=1. Replacing u₁ with −u₁ gives (−u₁)(−u₁)^T = u₁u₁^T, identical, because the two negative signs cancel in the outer product. The projector — and therefore the entire update rule — depends only on the subspace M spans, never on the sign convention used to write down its basis vectors.

A5. GPM keeps parameter count fixed and shares one weight matrix across every task; as tasks accumulate, the protected memory subspace grows and the orthogonal complement available to new tasks shrinks, so the failure mode is new-task learning capacity collapsing toward zero (as in A2) while old-task accuracy stays protected. A per-task LoRA adapter freezes the shared base weights permanently and gives each new task its own low-rank parameters, so old-task accuracy never degrades no matter how many tasks accumulate — but total parameter count grows linearly with the number of tasks, so its failure mode is unbounded storage and serving cost, not accuracy loss.

A6. False. R_l is built from the layer's input activations during a forward pass over task-A inputs — it needs no labels and no loss computation at all, only the inputs themselves flowing through the already-trained network. This is what makes it possible to build the memory from a lightly-labelled or even unlabelled replay sample, unlike EWC's Fisher-information estimate, which needs the loss (and therefore the labels) to compute gradients for importance weighting.

Think About It

Think about this: How would you explain continual learning: learning without catastrophic forgetting to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.