Model Merging: Combining Capabilities from Multiple Models

Two Fine-Tunes, One Deployment Slot

A Bengaluru edtech team starts from the same open-weight 7-billion-parameter checkpoint and fine-tunes it twice. One run trains on NCERT question-answer pairs until the model tutors cleanly in formal English. A separate run, on a different dataset owned by a different vendor, trains the same starting checkpoint on Hindi-English code-switched customer-support transcripts until the model handles Hinglish conversation fluently. Both fine-tunes work well individually. The product spec wants one model that tutors in Hinglish, and the serving budget has room for exactly one 7B model in GPU memory.

Three routes exist. Retrain from scratch on the union of both datasets — expensive, and the two datasets may not even be poolable, since the support transcripts carry a data-processing agreement that forbids exporting them into a third party's training pipeline. Run both models and blend their output token probabilities at inference time — this is ensembling, and it needs two full models resident in memory plus two forward passes per token, doubling both cost and latency. Or take the two already-trained weight files, combine them directly with arithmetic on a laptop with no GPU and no training data at all, and get a single checkpoint with both behaviours folded in. That third route is model merging, and it is the subject of this chapter: not multi-task training, not ensembling, but combining independently fine-tuned parameter sets into one functioning model through operations on the weights themselves.

Why Averaging Weights Even Makes Sense

The obvious objection: neural network weights are not interchangeable numbers you can average like exam scores. A single hidden unit's weight vector only means something in the context of every other weight around it — swap two hidden units' worth of parameters and, so long as you swap their downstream connections too, the network computes exactly the same function. This is permutation symmetry: a trained network's loss-minimising solution is really an equivalence class of solutions related by relabelling neurons. If you take two networks trained independently from two different random initialisations and average their weights coordinate-by-coordinate, you are almost certainly averaging a weight that plays "role 47" in network A with a weight that plays some unrelated role in network B. The result usually sits in a high-loss region between two unrelated basins and performs badly on everything — this is the well-documented loss barrier between independently trained solutions.

Fine-tunes of the same pretrained checkpoint are a different situation. Both the toxicity-detection fine-tune and the Hinglish fine-tune in the earlier example start from the identical base weights θ₀ and take relatively small gradient steps away from it. Empirically, and now with theoretical backing, models that stay in the same low-loss "basin" as their shared starting point remain linearly mode connected: the straight-line path in weight space between θ₀, the first fine-tune, and the second fine-tune stays in low-loss territory the whole way, because no neuron needed to be relabelled — coordinate 47 in the base model, in fine-tune A, and in fine-tune B all still refer to the same functional role, just nudged slightly. This is the empirical basis behind "model soups" (Wortsman et al., 2022), where simply averaging the weights of several independently fine-tuned copies of the same base model — no retraining, no data — reliably beats any single one of them on the held-out task. It is also why merging methods insist on a shared base checkpoint (or, when that is unavailable, first solve the permutation-alignment problem explicitly, an approach called Git Re-Basin) before doing anything as naive as coordinate-wise arithmetic.

Task Vectors: Arithmetic in Weight Space

Ilharco et al. (2022) formalised the useful unit of merging as a task vector. If θ₀ is the base checkpoint and θ_A is what you get after fine-tuning on task A, define

τ_A = θ_A − θ_0

τ_A is a vector living in the same space as the model's parameters, and it points in the direction that "adding task-A behaviour" moves the weights. Because that direction was computed relative to a shared starting point, and because of the linear-connectivity property above, these task vectors compose: you can add several of them onto the base model and get a single model exhibiting several fine-tuned behaviours at once, without ever training on combined data.

θ_merged = θ_0 + Σ λ_i · τ_i

where each λ_i is a scaling coefficient (often just 1, sometimes tuned down when merging many task vectors to avoid any one of them dominating). Negating a task vector instead of adding it — using θ_0 − τ_i — pushes the model away from that behaviour, which is used for targeted forgetting: for instance, subtracting a task vector built from toxic-completion fine-tuning data to suppress that behaviour in a general-purpose model, without touching anything else the model knows.

Worked Example: Merging Two Specialists

To see the arithmetic land exactly, shrink the whole picture to a two-parameter linear scorer, y = w · x, so every number can be checked by hand. Treat w as a stand-in for "the model's weights" and x as a two-feature input: x₁ measures how strongly a message contains toxic keywords, x₂ measures how strongly it code-mixes Hindi and English. A message is flagged when w · x exceeds a threshold of 4.

The base checkpoint has never seen either fine-tuning task: w₀ = (1, 1). Fine-tuning on toxicity data pulls the weight toward w_A = (3, 1); fine-tuning on code-mixing data pulls it toward w_B = (1, 3). The two task vectors:

import numpy as np

theta0 = np.array([1.0, 1.0])
thetaA = np.array([3.0, 1.0])   # fine-tuned on toxicity
thetaB = np.array([1.0, 3.0])   # fine-tuned on code-mixing

tauA = thetaA - theta0   # [2.0, 0.0]
tauB = thetaB - theta0   # [0.0, 2.0]

theta_merged = theta0 + tauA + tauB   # [3.0, 3.0]
print(theta_merged)

Trace it: tauA = (3−1, 1−1) = (2, 0); tauB = (1−1, 3−1) = (0, 2); theta_merged = (1+2+0, 1+0+2) = (3, 3). Printed output: [3. 3.].

Now check whether the merged model actually kept both capabilities, using a clean toxicity test point x_A = (2, 0) — strong toxic signal, zero code-mixing — and a clean code-mixing test point x_B = (0, 2):

x_taskA = np.array([2.0, 0.0])
x_taskB = np.array([0.0, 2.0])

print(theta0 @ x_taskA)        # 1*2 + 1*0 = 2.0   -> below threshold 4, base model misses it
print(theta_merged @ x_taskA)  # 3*2 + 3*0 = 6.0   -> above threshold, flagged correctly
print(theta_merged @ x_taskB)  # 3*0 + 3*2 = 6.0   -> above threshold, flagged correctly

Every line checks out: the base model scores 2.0 on the toxicity probe, below the threshold of 4, so it would have missed it before any fine-tuning — exactly what "the base model doesn't have either capability yet" should mean. The merged model scores 6.0 on both probes, matching what the individually fine-tuned specialists would have scored on their own task, because the two features are orthogonal here: τ_B's contribution to x_A is 3×0=0, so it never dilutes τ_A's effect, and vice versa. A single forward pass through one 2-parameter model now does what two separate specialists used to do — the merging story in miniature.

When Merging Fails: Interference

Orthogonal task vectors are the easy case. Real fine-tunes touch overlapping parameters, and when two task vectors disagree about which direction a shared parameter should move, naive summation cancels the useful signal instead of combining it. Extend the toy model with a third fine-tune: task C retrains the same base checkpoint to stop over-relying on x₁, because in its training data, keyword-stuffing was causing false positives that needed suppressing. That pulls the weight the other way: w_C = (−0.5, 1.0).

thetaC = np.array([-0.5, 1.0])
tauC = thetaC - theta0          # [-1.5, 0.0]

theta_AC = theta0 + tauA + tauC # [1+2-1.5, 1+0+0] = [1.5, 1.0]
print(theta_AC @ x_taskA)       # 1.5*2 + 1.0*0 = 3.0  -> BELOW threshold 4

τ_A pushes dimension 1 by +2.0; τ_C pushes the same dimension by −1.5. Summed, the net shift on that parameter is only +0.5, and the merged score on the toxicity probe drops from 6.0 (task A alone) to 3.0 — below the flagging threshold. Merging A and C by plain addition has quietly deleted the toxicity-detection capability that A was supposed to contribute, even though nothing about A's own weights changed. This is interference, and it is the central practical problem in model merging once you go beyond two conveniently orthogonal fine-tunes.

TIES-Merging: Trim, Elect Sign, Disjoint Merge

Yadav et al. (2023) address interference with TIES-merging, a three-step fix applied to the collection of task vectors before they are summed. First, trim: in a real task vector with millions of parameters, most coordinates are small, noisy shifts that contribute nothing useful; keep only the top-k% by magnitude per task vector and zero out the rest, which removes noise without touching the parameters that actually encode the fine-tuned behaviour. Second, elect sign: for each parameter, compare the total positive magnitude contributed by all task vectors against the total negative magnitude, and pick whichever side is larger as the "elected" sign for that coordinate. Third, disjoint merge: for each parameter, average together only the task vectors whose sign agrees with the election, discarding the ones that disagree entirely rather than letting them cancel the agreeing ones.

Apply that to the A-and-C interference case, dimension 1 only: τ_A contributes +2.0, τ_C contributes −1.5. Positive-side total magnitude is 2.0, negative-side is 1.5, so the elected sign is positive. Under disjoint merge, τ_C's entry is dropped entirely for this parameter because its sign lost the election; only τ_A's +2.0 survives, and since it is the sole agreeing entry, the merged value is its average, 2.0 — unchanged from τ_A alone.

theta_TIES = theta0 + np.array([2.0, 0.0])  # tauC's dim-1 entry dropped, tauA's kept
print(theta_TIES)              # [3.0, 1.0]
print(theta_TIES @ x_taskA)    # 3*2 + 1*0 = 6.0  -> above threshold, capability restored

The TIES-merged model scores 6.0 on the toxicity probe again, matching task A's own performance, while task C's genuine, non-conflicting adjustments elsewhere in the parameter space are still folded in undisturbed. TIES does not recover information that was never there; it simply refuses to let two fine-tunes cancel each other out on the parameters where they actively disagree, trading "average everything" for "average only what agrees, and settle disagreements by a magnitude vote."

DARE and SLERP: Two More Tools

DARE (Yu et al., 2023) attacks interference from a different angle: redundancy. Most entries in a task vector, it turns out, can be zeroed out entirely without hurting the fine-tuned behaviour much, because fine-tuning tends to nudge many more parameters than are strictly necessary to encode the new skill. DARE randomly drops a large fraction p of each task vector's entries (p = 0.9 is typical — drop nine parameters out of every ten) and rescales the survivors by 1⁄(1−p) so the vector's expected value is unchanged: if a coordinate survives with probability (1−p), multiplying it by 1⁄(1−p) keeps E[dropped-and-rescaled value] equal to the original value, an unbiased sparsification. Sparser, less redundant task vectors collide with each other far less often when several are summed, so DARE is frequently layered underneath TIES ("DARE-TIES") before the sign-election step even runs.

SLERP (spherical linear interpolation) solves a narrower but common case: merging exactly two full fine-tunes of the same architecture directly, without going through a task-vector subtraction at all. Linearly interpolating two high-dimensional weight vectors, θ_new = (1−t)θ_A + tθ_B, tends to shrink the resulting vector's norm whenever θ_A and θ_B point in different directions — the straight-line midpoint of two vectors on a sphere dips inside the sphere. SLERP instead interpolates along the great-circle arc connecting them, θ_new = [sin((1−t)Ω)/sinΩ]·θ_A + [sin(tΩ)/sinΩ]·θ_B, where Ω is the angle between θ_A and θ_B; this preserves the weight vector's norm throughout the interpolation and is the default merge operator in community tools like mergekit for blending two open-weight checkpoints layer by layer.

Common Misconception: Merging Is Not Ensembling

Students who have just met ensembling in a machine-learning context often assume "combining models" always means "run all of them and average their predictions." That is ensembling, and it is a fundamentally different operation from merging. Ensembling combines outputs: every model stays intact, every model runs its own full forward pass on every input, and the results are averaged (or voted) at the end — cost and memory scale with the number of models, N times the parameters resident in memory and N times the inference compute per query. Merging combines weights, once, offline, before any inference happens: the output is a single checkpoint with the same parameter count and the same inference cost as any one of the original models. The edtech example from the opening only fits inside the serving budget because merging produces one 7B model, not two — that constraint is precisely what ensembling cannot satisfy and merging can. The trade-off runs the other way too: ensembling never suffers from the sign-conflict interference this chapter spent two sections on, because the models never touch each other's weights; merging can suffer interference, which is exactly why TIES, DARE, and careful scaling coefficients exist as a body of technique in the first place.

Active Recall

Attempt each question before reading its answer.

1. Base weights θ₀ = (2, 4). Two fine-tunes: θ_A = (5, 4), θ_B = (2, 7). Compute the merged weights via full task arithmetic (λ = 1 for each).
2. Why must two models share the same base checkpoint (or at least be brought into alignment first) before coordinate-wise weight averaging can work?
3. Three task vectors carry the values 4, −1, and −1 on some shared parameter. Under TIES-merging, which sign gets elected, and which vectors' contributions survive into the merge for that parameter?
4. A team has enough VRAM for exactly one model's worth of weights but wants both coding ability and Hindi-translation ability from two separately fine-tuned checkpoints. Should they ensemble or merge, and why?
5. DARE rescales surviving parameters by 1⁄(1−p) after randomly dropping a fraction p of a task vector's entries. Why rescale at all, rather than just leaving the survivors as they are?
6. In the worked interference example, merging τ_A and τ_C by plain summation dropped the toxicity-probe score from 6.0 to 3.0, below the threshold of 4. In one sentence, why did that happen, and which technique from this chapter is built specifically to prevent it?

Answers.

1. τ_A = θ_A − θ_0 = (3, 0). τ_B = θ_B − θ_0 = (0, 3). θ_merged = θ_0 + τ_A + τ_B = (2+3+0, 4+0+3) = (5, 7).
2. Because a trained network's parameters only carry meaning relative to the roles of the surrounding neurons, and independently trained networks are related by permutation symmetry — coordinate 47 in one network has no particular reason to play the same role as coordinate 47 in another. Fine-tunes of a shared base checkpoint stay in the same loss basin (linear mode connectivity), so their coordinates keep referring to the same functional roles, which is what makes coordinate-wise combination meaningful instead of arbitrary.
3. Positive-side magnitude is 4, negative-side magnitude is 1+1=2, so the elected sign is positive. Only the vector contributing +4 agrees with the elected sign and survives into the merge for that parameter (merged value 4); both −1 vectors are dropped under the disjoint-merge step.
4. Merge, not ensemble. Merging produces one checkpoint with the same parameter count and VRAM footprint as either original model on its own, whereas ensembling needs both full models resident simultaneously plus a separate forward pass through each — double the memory the team has available.
5. To keep the task vector's expected value unbiased. If a coordinate survives with probability (1−p), multiplying the survivors by 1⁄(1−p) makes the expectation of the sparsified vector equal to the original vector, so the pruned, rescaled task vector still points the same direction on average even though most of its entries are now exactly zero.
6. τ_A and τ_C pushed the same parameter in opposite directions (+2.0 versus −1.5), so plain addition partially cancelled the two shifts instead of combining two distinct capabilities — this destructive interference is exactly what TIES-merging's sign-election and disjoint-merge steps are designed to detect and resolve.

Think About It

Think about this: How would you explain model merging: combining capabilities from multiple models to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where model merging: combining capabilities from multiple models is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting model merging: combining capabilities from multiple models to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind model merging: combining capabilities from multiple models, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.