Why Backpropagation Isn't Always the Tool
Every network you trained in Grade 11 learned the same way: forward pass, compute a differentiable loss, backpropagate the gradient, nudge the weights. That machinery assumes something you rarely stop to check: that the path from weights to loss is a chain of differentiable operations you can actually write down. Now picture a different job. A team is building the flight-control network for an autonomous medicine-delivery drone, the kind piloted in Telangana under India's "Medicine from the Sky" program, carrying a payload across a village route with real wind, real GPS drift, and a real decision tree for obstacle avoidance running underneath. The network's "loss" is really a reward computed only after a full flight: did the package arrive, how much battery is left, did the drone violate a stability threshold and abort. That reward comes out of a simulator (or the real world) full of stochastic wind draws, discrete branching logic, and hard sensor thresholds. There is no clean symbolic expression connecting a specific weight to that final number, so there is no gradient to backpropagate through it.
A second problem is subtler and just as common. Even when a gradient exists, you might not know the right architecture to attach it to: how many hidden units the controller needs, which sensor feeds which layer, whether a recurrent connection helps stabilize hovering. Gradient descent optimizes weights for a topology you hand it in advance; it says nothing about which topology to choose. Neuroevolution answers both problems with one idea: treat the network, weights and optionally its topology together, as a genome, and search the space of genomes with the tools of evolutionary computation rather than calculus. A population of networks is evaluated on the task, the better performers are more likely to produce offspring, and mutation and crossover generate the next generation. No gradient is required anywhere in the loop, because a "fitness function" only needs to return a scalar score for a genome; it never needs to be differentiable, or even continuous, or even the same function twice (a stochastic simulator is perfectly fine).
The Evolutionary Algorithm, Specialized to Neural Networks
Strip neuroevolution to its skeleton and it is the same loop every genetic algorithm uses, with a neural network standing in for the genome:
1. Representation. Decide what a genome encodes. The simplest choice is a flat vector of real numbers, one per weight (and bias) of a fixed architecture. NEAT, covered below, instead encodes a variable list of node and connection genes, so topology itself is part of the genome.
2. Initialization. Generate a population of genomes, usually with small random weights, sometimes starting from the minimal possible topology.
3. Fitness evaluation. Decode each genome into an actual network, run it on the task, and record a scalar score: game score, tracking error, negative loss, whatever the domain calls for.
4. Selection. Genomes with higher fitness are more likely to become parents. Common schemes are truncation selection (keep the top fraction), tournament selection (sample a few genomes, keep the best), and elitism (always carry the single best genome forward unchanged, so the population's best-ever fitness can never decrease).
5. Variation. Produce offspring by mutation (perturb weights, usually by adding Gaussian noise, and possibly add or remove structure) and crossover (recombine genes from two parents).
6. Replacement. The offspring, sometimes mixed with surviving parents, become the next generation. Return to step 3.
The genome lives in the same weight space your G11 backpropagation lessons used, R^n for n parameters, but the search process is entirely different. Gradient descent takes a single point and moves it along the local slope of a differentiable loss surface. Neuroevolution maintains a whole population of points and moves the population by selection and stochastic variation, using only the fitness value at each point, never its derivative. This is exactly the tool you reach for when the surface has no usable derivative, or when you do not trust the derivative you do have (a loss surface for a long RL episode with sparse, delayed reward is a classic case where the true gradient, even if computable, is an extremely noisy estimate).
A Fully Traced Generation
The mechanics are easiest to see on a genome small enough to compute by hand. Take a tiny one-hidden-unit network with a ReLU nonlinearity: two real inputs, weight w1 and bias b1 into the hidden unit, weight w2 from the hidden unit to a single linear output. The genome is the triple [w1, b1, w2]. The task is to fit two data points drawn from the target function y = 2x + 1: the pairs (x=1, y=3) and (x=2, y=5). Fitness is the negative sum of squared errors, so a fitness of 0 is a perfect fit and more negative is worse.
def relu(z):
return max(0.0, z)
def forward(genome, x):
w1, b1, w2 = genome
hidden = relu(w1 * x + b1)
return w2 * hidden
def fitness(genome, data):
sse = 0.0
for x, target in data:
pred = forward(genome, x)
sse += (target - pred) ** 2
return -sse
data = [(1, 3), (2, 5)] # target function: y = 2x + 1
population = {
"G1": [1.0, 0.0, 1.0],
"G2": [0.5, 0.5, 2.0],
"G3": [2.0, -1.0, 1.0],
"G4": [1.5, 1.0, 1.5],
}
for name, genome in population.items():
print(name, genome, "fitness =", fitness(genome, data))Trace it by hand to see exactly what the code prints. For G1 = [1.0, 0.0, 1.0]: at x=1, hidden = relu(1.0) = 1.0, prediction = 1.0, error = 3 - 1 = 2; at x=2, hidden = relu(2.0) = 2.0, prediction = 2.0, error = 5 - 2 = 3. Sum of squares = 4 + 9 = 13, so fitness = -13.0. For G2 = [0.5, 0.5, 2.0]: at x=1, hidden = relu(1.0) = 1.0, prediction = 2.0, error = 1; at x=2, hidden = relu(1.5) = 1.5, prediction = 3.0, error = 2. SSE = 1 + 4 = 5, fitness = -5.0. For G3 = [2.0, -1.0, 1.0]: at x=1, hidden = relu(1.0) = 1.0, prediction = 1.0, error = 2; at x=2, hidden = relu(3.0) = 3.0, prediction = 3.0, error = 2. SSE = 4 + 4 = 8, fitness = -8.0. For G4 = [1.5, 1.0, 1.5]: at x=1, hidden = relu(2.5) = 2.5, prediction = 3.75, error = -0.75; at x=2, hidden = relu(4.0) = 4.0, prediction = 6.0, error = -1.0. SSE = 0.5625 + 1 = 1.5625, fitness = -1.5625. Ranked from best to worst: G4 (-1.5625), G2 (-5.0), G3 (-8.0), G1 (-13.0).
Now run one generation of selection, crossover, and mutation on this population.
def crossover(parent_a, parent_b, point):
return (parent_a[:point] + parent_b[point:],
parent_b[:point] + parent_a[point:])
def mutate(genome, deltas):
return [gene + d for gene, d in zip(genome, deltas)]
ranked = sorted(population.items(),
key=lambda item: fitness(item[1], data), reverse=True)
parent_a_name, parent_a = ranked[0]
parent_b_name, parent_b = ranked[1]
child1, child2 = crossover(parent_a, parent_b, point=1)
child1_mut = mutate(child1, deltas=[0.1, 0.0, 0.0])
for name, genome in [("child1", child1), ("child2", child2),
("child1_mutated", child1_mut)]:
print(name, genome, "fitness =", fitness(genome, data))The two fittest parents are G4 and G2. A single-point crossover at index 1 splits each three-gene genome into a first gene and a remaining pair, then swaps: child1 = [G4[0]] + G2[1:] = [1.5, 0.5, 2.0], child2 = [G2[0]] + G4[1:] = [0.5, 1.0, 1.5]. Evaluate child1: at x=1, hidden = relu(2.0) = 2.0, prediction = 4.0, error = -1; at x=2, hidden = relu(3.5) = 3.5, prediction = 7.0, error = -2. SSE = 1 + 4 = 5, fitness = -5.0. Evaluate child2: at x=1, hidden = relu(1.5) = 1.5, prediction = 2.25, error = 0.75; at x=2, hidden = relu(2.0) = 2.0, prediction = 3.0, error = 2. SSE = 0.5625 + 4 = 4.5625, fitness = -4.5625. Then mutate child1 by adding 0.1 to its first gene: child1_mutated = [1.6, 0.5, 2.0]. At x=1, hidden = relu(2.1) = 2.1, prediction = 4.2, error = -1.2; at x=2, hidden = relu(3.7) = 3.7, prediction = 7.4, error = -2.4. SSE = 1.44 + 5.76 = 7.2, fitness = -7.200000000000002 (this exact value does not print as a clean -7.2, because 1.6 is not exactly representable in binary floating point, and the rounding error propagates through the rest of the computation).
Look at what actually happened: neither unmutated child (-5.0, -4.5625) beats parent G4's -1.5625, and the mutated child (about -7.2) is worse still. This is the normal, expected outcome of a single generation on a three-gene genome, not a bug in the trace. It is also the reason real genetic algorithms almost always pair variation with elitism: G4 is copied unchanged into the next generation regardless of what crossover and mutation produce, so the population's best-found fitness can only hold steady or improve across generations, never regress, even though most individual offspring in most individual generations are neutral or worse than their parents.
Correcting a Common Misconception
That result is exactly the point where most students misread how evolutionary search makes progress. The intuitive picture is "each generation should be a bit better than the last one," as if evolution were a smoothed-out version of gradient descent that occasionally stumbles. It isn't. Recombining two good genomes has no guarantee of producing a good genome, because the pieces that made each parent work were tuned jointly with the rest of that parent's genes, and splicing in a piece from a different, independently-tuned parent can (and, as the trace above shows, often does) break that coordination. Progress in a genetic algorithm comes from the shape of the whole population's fitness distribution shifting upward over many generations under selection pressure, combined with elitism guaranteeing the record-holder is never lost, not from a promise that any single offspring event is an improvement. If you ever see "fitness only goes up from here" reasoning applied to a single mutation or a single crossover, distrust it; the guarantee is about the running best across the population and the generations, not about any one genome.
The Permutation Problem, and Why Fixed-Topology Crossover Breaks Down at Scale
The three-gene example above hides a problem that gets severe fast as networks grow. Two independently-initialized networks of the same architecture can compute the same function while labeling their hidden units in completely different orders: hidden unit 3 in network A might play the role hidden unit 7 plays in network B. If you crossover their weight vectors gene-by-gene at fixed positions, you are just as likely to combine A's "unit 3" weights with B's "unit 3" weights even though those units learned unrelated features, and the offspring inherits a mismatched, usually badly broken combination. This is called the competing conventions problem, and it is the reason naively averaging or splicing two independently-trained networks rarely gives you something as good as either parent, even when both parents solve the task well. Position in the gene vector, by itself, carries no information about which structural role a gene plays.
NEAT: Evolving Topology and Weights Together
Stanley and Miikkulainen's NeuroEvolution of Augmenting Topologies (NEAT, published in Evolutionary Computation, 2002) solves the alignment problem by tagging every gene with history rather than position. Each connection gene records an innovation number, a globally increasing counter assigned the first time that specific structural mutation (a new connection between two particular nodes, or a new node splitting an existing connection) ever occurs anywhere in the run. Two genomes descended from a common ancestor that both carry a gene with innovation number 5 are guaranteed to be recording the same historical event, even if their current weight on that connection has since drifted apart through independent mutation. Crossover then aligns genomes by innovation number instead of by list position: genes whose innovation number is present in both parents ("matching" genes) are inherited randomly from either parent, gene by gene, while genes present in only one parent (called disjoint if they fall within the numeric range the other genome also covers, or excess if they fall beyond it) are inherited from whichever parent has the higher fitness. NEAT also starts every run from the minimal possible topology, no hidden nodes at all, and grows structure only through mutation, and it protects brand-new structural mutations from being immediately outcompeted by grouping genomes into species based on genetic similarity and applying fitness sharing within each species, so a genome's effective fitness is reduced according to how many similar individuals crowd its own niche rather than by how different it is from the rest of the population, letting a new, different topology compete mainly against its own small, similar niche instead of the entire population, so a promising new connection gets a few generations to have its weights tuned before it has to compete head-on with the rest of the population.
Work through one concrete crossover. Parent A (the fitter of the two) has five connection genes: innovation 1, input-1 to output, weight 0.7, enabled; innovation 2, input-2 to output, weight -0.5, enabled; innovation 3, input-1 to a hidden node, weight 0.2, enabled; innovation 4, hidden to output, weight 0.4, enabled; innovation 5, input-2 to hidden, weight -0.1, disabled. Parent B, less fit, shares the same ancestry up through innovation 3 but then diverged onto a different structural mutation: innovation 1, input-1 to output, weight 0.9, enabled; innovation 2, input-2 to output, weight -0.5, disabled; innovation 3, input-1 to hidden, weight 0.6, enabled; innovation 6, a second hidden node to output, weight -0.3, enabled; innovation 7, input-1 to that second hidden node, weight 0.9, enabled. Genes 1, 2, and 3 are matching (present in both, inherited randomly gene by gene); genes 4 and 5 are disjoint and present only in fitter Parent A, so the child inherits them; genes 6 and 7 are excess and present only in less-fit Parent B (they fall beyond innovation number 5, the highest innovation number in Parent A), so the child does not inherit them at all. In this particular draw the child ends up with gene 1 from A, gene 2 from B (so that connection is disabled in the child even though it was enabled in the fitter parent), gene 3 from A, plus genes 4 and 5 automatically from A. The resulting child has exactly Parent A's topology (no second hidden node) with one connection's weight and enabled-status pulled from Parent B, which is precisely what innovation-number alignment buys you: a structurally sensible recombination instead of the scrambled mess a positional crossover would produce.
Scaling Neuroevolution to Deep Networks
Everything above evolved a handful of parameters. Modern reinforcement-learning policies have millions. Two results, both from 2017, showed evolutionary search is not confined to toy genomes. Such, Madhavan, Conti, Lehman, Stanley, and Clune (Uber AI Labs, "Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning," 2017) trained deep convolutional policies with millions of weights to play Atari games using a plain genetic algorithm, competitive with contemporary deep RL methods on several games. The trick that makes this tractable is an indirect, compact encoding: instead of storing every weight of every generation's network, each genome is stored as a short list of random seeds, one seed per mutation event in that individual's lineage. To reconstruct the actual weight vector, you replay the seeds through a pseudorandom number generator in order, regenerating each generation's perturbation on demand and adding it to the previous weights. A genome that decodes into millions of floating-point weights can be transmitted and stored as a few integers, which is what makes population-scale evolution of deep networks computationally feasible at all. In the same year, Salimans, Ho, Chen, Sidor, and Sutskever (OpenAI, "Evolution Strategies as a Scalable Alternative to Reinforcement Learning," 2017) took a related but distinct approach: perturb the current weight vector with many random Gaussian directions, weight each direction by the fitness it achieved, and average them into a step. This is technically closer to a finite-difference estimate of a gradient than to biological evolution (there is no selection of individuals, no crossover, just noise and a weighted average), but it inherited the name "evolution strategies," a term coined in the 1960s by Ingo Rechenberg and Hans-Paul Schwefel for exactly this style of noise-based parameter search. Its practical appeal is the same as the genetic algorithm's: workers only need to exchange scalars (a random seed and the fitness it produced), never gradients or full weight vectors, so it scales to thousands of parallel machines with almost no communication overhead. Neuroevolution's biggest downstream legacy, though, is architecture search rather than weight search: Real, Aggarwal, Huang, and Le ("Regularized Evolution for Image Classifier Architecture Search," AAAI 2019) evolved convolutional architectures with a genetic algorithm to produce AmoebaNet, extending NEAT's core idea, evolve the topology, not just the weights, to the scale of production image classifiers.
Where Neuroevolution Wins and Where Gradients Still Win
The honest comparison against gradient-based RL (policy gradient methods, PPO, and the like) cuts both ways. Neuroevolution's advantages are structural: it needs no backward pass and no differentiable simulator, it is embarrassingly parallel because workers only need to report a fitness scalar back to a central process, it has no credit-assignment problem to solve (no need to figure out which action in a long episode caused the eventual reward, since fitness only ever looks at the final outcome), and a diverse population explores the parameter space more broadly than a single point sliding downhill, which helps against deceptive or sparse rewards that mislead a gradient estimate. Its disadvantage is sample efficiency: dense gradient information, when it is available, tells you the locally best direction to move in one shot, while a population using only fitness scalars has to implicitly discover that direction through many noisy evaluations, and that gap widens sharply as the number of parameters grows into the hundreds of millions. This is exactly why neuroevolution is not how large language models or large vision backbones are trained; wherever you can get a clean, dense, differentiable gradient signal, backpropagation extracts far more learning per sample. Neuroevolution earns its place precisely where that signal is unavailable, noisy, or actively misleading, and where the thing you need to search over includes the architecture itself, not only its weights.
Active Recall
Attempt each question before reading its answer.
1. A robotics team has a reward function that returns +1 only if a legged robot walks ten metres without falling, and 0 otherwise, computed by a physics simulator with no differentiable path from motor-torque weights to that outcome. Why does this favor neuroevolution over standard backpropagation-based RL?
2. In the worked example, suppose the crossover point were 2 instead of 1 (splitting each three-gene genome after the second gene instead of the first). Recompute child1 and child2's genomes and fitness values, and state whether elitism still preserves G4 as the population's best individual.
3. Why does naively averaging the weights of two independently-trained networks with the same architecture usually produce a worse network than either parent, even though both parents solve the task well?
4. In the NEAT crossover trace, gene 2 (input-2 to output) was inherited from Parent B, the less-fit parent, and came out disabled even though the child's overall topology matches the fitter Parent A. Explain how NEAT's rule for matching genes made this possible.
5. Why does OpenAI's evolution-strategies paper describe its method as "not really" a genetic algorithm, despite the name?
6. Why does Uber AI Labs' seed-based genome encoding matter specifically for scaling neuroevolution to deep networks, rather than for small genomes like the three-gene example?
Answers.
1. The reward is a sparse, all-or-nothing signal produced by a black-box simulator; there is no differentiable chain connecting a specific weight to the +1/0 outcome, so backpropagation has nothing to propagate through. Neuroevolution only needs the scalar reward itself as a fitness value, so it can optimize directly against this kind of black-box, non-differentiable, sparse objective without needing any gradient at all.
2. With crossover point 2, child1 = parent_a[:2] + parent_b[2:] = [1.5, 1.0] + [2.0] = [1.5, 1.0, 2.0], and child2 = parent_b[:2] + parent_a[2:] = [0.5, 0.5] + [1.5] = [0.5, 0.5, 1.5]. Evaluating child1: at x=1, hidden = relu(2.5) = 2.5, prediction = 5.0, error = -2; at x=2, hidden = relu(4.0) = 4.0, prediction = 8.0, error = -3. SSE = 4 + 9 = 13, fitness = -13.0. Evaluating child2: at x=1, hidden = relu(1.0) = 1.0, prediction = 1.5, error = 1.5; at x=2, hidden = relu(1.5) = 1.5, prediction = 2.25, error = 2.75. SSE = 2.25 + 7.5625 = 9.8125, fitness = -9.8125. Both children are worse than with crossover point 1 (-5.0 and -4.5625 before), so this particular crossover point is a worse split for this pair of parents. But elitism still preserves G4 unchanged at -1.5625, because elitism copies the best-scoring genome from the already-evaluated previous population forward regardless of what crossover produces afterward; changing the crossover point changes only the offspring's genomes and fitness, never which individual was already the elite going into that generation.
3. Two independently-trained networks of the same architecture can compute the same function while assigning different structural roles to their hidden units in different positions (the competing conventions problem). Averaging weight-by-weight by position mixes a unit that learned one feature in network A with a unit at the same position that learned a different, unrelated feature in network B, producing a hybrid that computes neither original function correctly.
4. NEAT's rule for matching genes (genes whose innovation number appears in both parents) is to inherit each one randomly and independently from either parent, regardless of which parent is fitter overall. Gene 2 has innovation number 2 and appears in both Parent A and Parent B, so it was eligible to come from either side; in this draw it happened to come from Parent B, disabled, even though the rest of the child's structure (genes 4 and 5, which are disjoint and only exist in Parent A) came from the fitter parent.
5. The method perturbs the current parameter vector with many random directions and combines them into a single weighted-average update, which is structurally a finite-difference estimate of a gradient. It has no population of surviving individuals selected against each other and no crossover recombining genetic material between individuals, the two operations that define a genetic algorithm; it only borrows the historical name "evolution strategies" from Rechenberg and Schwefel's 1960s noise-based search methods.
6. A deep network's weight vector can have millions of entries, so storing and transmitting the full vector for every individual in every generation is expensive in memory and communication bandwidth. Encoding a genome as a short list of random seeds lets each generation's perturbation be regenerated on demand from a pseudorandom number generator instead of stored explicitly, which only pays off once the weight vector is large enough that a handful of integers is meaningfully cheaper than the full vector; for a three-gene genome like the worked example, storing the three numbers directly is already as compact as it gets, so the seed trick buys nothing.
Think About It
Think about this: How would you explain neuroevolutionary approaches: evolving neural networks to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Practice Exercises
Now it is time to practice! Complete these challenges to solidify your understanding:
- Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
- Exercise 2: Find a real-world example where neuroevolutionary approaches: evolving neural networks is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
- Exercise 3: Create a mind-map connecting neuroevolutionary approaches: evolving neural networks to at least 3 other topics you have studied.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind neuroevolutionary approaches: evolving neural networks, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.