Multi-Task Learning: Sharing Knowledge Across Tasks

A ride-hailing platform operating across Indian metros needs three predictions from every trip request: how many minutes until pickup and drop (ETA), what the fare should be, and how likely the rider is to cancel before the driver arrives. The naive engineering approach is to build three separate neural networks — one per prediction — each trained on its own labelled data, each learning its own notion of how distance, time-of-day, traffic density, and driver rating combine into a number. That approach works, but it throws away something obvious: all three tasks are computed from the *same* underlying situation. A trip that looks unusually slow for its distance (bad traffic) is also a trip more likely to get cancelled and one whose fare should include a surge component. Three independent networks each have to rediscover "traffic is bad right now" from scratch, using only their own task's labels, before they can use that fact.

Multi-task learning (MTL) is the family of techniques that trains one model to solve several related tasks at once, deliberately sharing internal representations between them so that what the model learns for one task can help the others. This chapter builds the formal setup, works through the exact arithmetic of how sharing changes a gradient update — including the case where two tasks actively disagree — and shows where that disagreement comes from and how it is diagnosed and reduced.

From single-task learning to a shared objective

In ordinary supervised learning you pick one task, one loss function L(y, f(x;θ)), and you minimize it over a single parameter vector θ. If you have T related tasks and train them independently, you get T separate parameter vectors θ₁, …, θ_T, each fit only to its own labels.

MTL restructures the parameters into two groups. A set of shared parameters θ_shared defines a shared encoder g(x; θ_shared) that maps the raw input into an internal representation h = g(x; θ_shared). Each task then has its own small task-specific head f_i(h; θ_i) that maps that shared representation to its own output. Training minimizes a single combined objective:

L_total(θ_shared, θ_1, …, θ_T) = Σ_i w_i · L_i(y_i, f_i(g(x; θ_shared); θ_i))

where w_i is a weight controlling how much task i influences the shared parameters. Every gradient step now updates θ_shared using information pooled from every task simultaneously — that pooling is the entire mechanism of multi-task learning, and it is also the source of both its benefit and its main failure mode, as the worked example below shows precisely.

Hard sharing versus soft sharing

There are two structurally different ways to implement "sharing," and the distinction matters for system design, not just theory.

Hard parameter sharing is what the equation above describes: one literal set of shared weights, physically reused by every task, with only the small output heads kept separate. This is the dominant pattern in production systems because it is cheap — one forward pass through the trunk serves every task — and because Caruana's original analysis of multi-task neural networks (Caruana, 1997, Machine Learning) argued that forcing tasks through a shared bottleneck acts as a strong regularizer: a set of shared weights that has to simultaneously explain ETA, fare, and cancellation is much less free to overfit noise specific to any one of them than a set of weights that only ever sees one task's labels.

Soft parameter sharing keeps a separate parameter set per task but adds a penalty that pulls corresponding parameters toward each other, or learns an explicit linear combination of per-task feature maps — the cross-stitch network of Misra et al. (CVPR, 2016) learns, at every layer, how much of each task's representation to mix into every other task's. Soft sharing is more flexible (a task that is genuinely different can drift away from the others) but roughly doubles or triples the parameter count and the inference cost compared to a hard-shared trunk, so production systems reach for it only when hard sharing measurably hurts one of the tasks.

Why sharing actually helps: four concrete mechanisms

"More data" is not, by itself, an explanation — the cancellation-prediction task doesn't get any new cancellation labels by being trained alongside the ETA task. Caruana identified several more specific mechanisms, each visible in the ride-hailing example:

Statistical data amplification. If both traffic-driven slowdowns and traffic-driven cancellations are present in the training data, but noisily, the shared representation sees the traffic signal reinforced from two independent label sources instead of one, making it easier to isolate from noise than either task's data alone would allow.

Attention focusing. A feature that is only weakly informative for one task but strongly informative for another (say, "driver rating" barely moves ETA but strongly predicts cancellations) gets pulled into the shared representation because at least one task needs it — the ETA head then gets to use it too, for free.

Eavesdropping. A task with very little labelled data (cancellations might be rare relative to completed trips) can piggyback on a feature that a data-rich task (ETA, computed from every single trip) already learned to extract, rather than having to learn it from its own sparse signal.

Representation bias / regularization. The shared trunk is constrained to a representation that is useful across tasks simultaneously, which is a narrower, better-generalizing hypothesis space than "whatever minimizes one task's training loss," directly reducing overfitting risk on the shared parameters.

The architecture, and where the tasks disagree

The diagram below shows the hard-parameter-sharing architecture for the ride-hailing example, and — in the lower panel — the mechanism that the rest of this chapter derives numerically: at every training step, each task head sends a gradient back into the shared weight, and those gradients do not have to agree.

A fully worked example: computing the shared gradient by hand

Take one training example from the ride-hailing model above, with a single (deliberately simplified, scalar) shared weight w so every step can be checked by hand. Let the normalized trip-distance feature be x = 2, and let the current parameters be shared weight w = 0.5, ETA-head weight a = 0.8, cancellation-head weight b = -1.2.

Forward pass. The shared representation is h = w·x = 0.5 × 2 = 1.0.

ETA head (regression): ŷ_ETA = a·h = 0.8 × 1.0 = 0.8. Suppose the true (scaled) ETA is y_ETA = 1.2, giving squared-error loss L_ETA = (1.2 − 0.8)² = 0.16.

Cancellation head (classifier): logit z = b·h = −1.2 × 1.0 = −1.2, so ŷ_cancel = σ(−1.2) = 1 / (1 + e^{1.2}) = 1 / 4.3201 ≈ 0.2315. Suppose this trip actually was cancelled, y_cancel = 1, giving binary cross-entropy loss L_cancel = −ln(0.2315) ≈ 1.4633.

Backward pass into the shared weight. For the ETA head, ∂L_ETA/∂ŷ_ETA = −2(1.2 − 0.8) = −0.8, and since ŷ_ETA = a·h, ∂ŷ_ETA/∂h = a = 0.8, so ∂L_ETA/∂h = −0.8 × 0.8 = −0.64. Since h = w·x, ∂h/∂w = x = 2, giving ∂L_ETA/∂w = −0.64 × 2 = −1.28.

For the cancellation head, the standard sigmoid-plus-cross-entropy gradient collapses to ∂L_cancel/∂z = ŷ_cancel − y_cancel = 0.2315 − 1 = −0.7685. Since z = b·h, ∂z/∂h = b = −1.2, so ∂L_cancel/∂h = −0.7685 × (−1.2) ≈ 0.9222, and ∂L_cancel/∂w = 0.9222 × 2 ≈ 1.8444.

Here is the entire point of multi-task learning made concrete: the ETA task's gradient on the shared weight is negative (−1.28), meaning gradient descent will increase w; the cancellation task's gradient is positive (≈1.8444), meaning gradient descent will decrease w. In one dimension, "cosine similarity between task gradients" is just the sign of their product, and (−1.28)(1.8444) < 0 — the two tasks are pulling the shared representation in opposite directions. This is exactly the phenomenon Yu et al. (NeurIPS, 2020) target with their PCGrad method: when two task gradients have negative cosine similarity, the conflicting component of one gradient is projected out before the update is applied, rather than letting the tasks fight to a compromise no task actually wants.

With equal task weights w_ETA = w_cancel = 0.5, the actual step taken is 0.5(−1.28) + 0.5(1.8444) ≈ 0.2822 — small and positive, meaning the shared weight barely moves, and moves in the cancellation task's preferred direction despite the ETA task wanting the opposite. Loss-balancing methods like GradNorm (Chen et al., ICML, 2018) exist precisely because fixed, equal weights like 0.5/0.5 are usually wrong: they let whichever task happens to produce the larger-magnitude gradient dominate the shared parameters, which has nothing to do with which task is actually more important.

Confirming the arithmetic in code

The same computation, run as an actual hard-parameter-sharing network, should reproduce every number derived above:

import torch
import torch.nn as nn

class RideMTLNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.shared = nn.Linear(1, 1, bias=False)      # w
        self.eta_head = nn.Linear(1, 1, bias=False)     # a
        self.cancel_head = nn.Linear(1, 1, bias=False)  # b

    def forward(self, x):
        h = self.shared(x)
        eta_pred = self.eta_head(h)
        cancel_logit = self.cancel_head(h)
        return eta_pred, cancel_logit

model = RideMTLNet()
with torch.no_grad():
    model.shared.weight.fill_(0.5)
    model.eta_head.weight.fill_(0.8)
    model.cancel_head.weight.fill_(-1.2)

x = torch.tensor([[2.0]])
y_eta = torch.tensor([[1.2]])
y_cancel = torch.tensor([[1.0]])

eta_pred, cancel_logit = model(x)
loss_eta = (y_eta - eta_pred).pow(2).mean()
loss_cancel = nn.functional.binary_cross_entropy_with_logits(cancel_logit, y_cancel)

total_loss = 0.5 * loss_eta + 0.5 * loss_cancel
total_loss.backward()

print(round(eta_pred.item(), 4), round(loss_eta.item(), 4))
print(round(torch.sigmoid(cancel_logit).item(), 4), round(loss_cancel.item(), 4))
print(round(model.shared.weight.grad.item(), 3))

Tracing this by hand: model.shared(x) computes 0.5 × 2 = 1.0, so eta_pred = 0.8 × 1.0 = 0.8 and cancel_logit = −1.2 × 1.0 = −1.2 — matching the hand derivation exactly. The three printed lines are therefore 0.8 0.16, then 0.2315 1.4633, then 0.282 — the gradient stored in model.shared.weight.grad after backward() is the same combined value computed by hand above, because PyTorch's autograd is doing precisely the chain-rule steps worked through in the previous section, summed across both loss terms.

What sharing costs — and saves — in parameters

Hard sharing is attractive operationally as well as statistically. Suppose the shared encoder has 4,000,000 parameters and each task head has 120,000 parameters, across 4 related tasks (ETA, fare, cancellation, and driver-assignment priority). Training four fully separate networks costs 4 × (4,000,000 + 120,000) = 16,480,000 parameters in total. The hard-shared network costs 4,000,000 + 4 × 120,000 = 4,480,000 parameters — a saving of 12,000,000 parameters, a 72.8% reduction. That reduction is not just a training-time convenience: at inference time, a ride-hailing backend serving millions of requests per day runs one shared forward pass instead of four, cutting both latency and the memory footprint of models held in production.

Common misconception

Students who have just learned about ensembling often assume multi-task learning is the same idea: train several models, then combine their outputs. It is not. An ensemble trains separate models — possibly on the same data, possibly even the same task — and combines their predictions after training is finished; the models never share parameters and never influence each other's gradients. Multi-task learning trains one model whose shared parameters are updated by a combined loss during training itself, exactly as worked through above, where the ETA and cancellation gradients physically add together inside a single backward pass before w is ever updated. Ensembling reduces prediction variance after the fact; multi-task learning changes what the shared representation learns to encode in the first place. A model can be both — an ensemble of multi-task networks — but collapsing the two ideas into one is a category error that will lead you to mis-implement both.

Active recall

Attempt each question before reading its answer.

Q1. In the worked example, ∂L_ETA/∂w = −1.28 and ∂L_cancel/∂w ≈ 1.8444. What does the opposite sign indicate, and what is the one-dimensional equivalent of "negative cosine similarity between task gradients"?

A1. Opposite signs mean the two tasks want the shared weight to move in opposite directions — a gradient descent step that helps one task's loss necessarily hurts the other's at this point. In one dimension, cosine similarity between two scalars is just the sign of their product; here (−1.28)(1.8444) < 0, the exact signature of gradient conflict that methods like PCGrad detect and correct.

Q2. Recompute the combined gradient on w if the task weights are changed from (0.5, 0.5) to (0.8, 0.2), favoring the ETA task. What direction does the update now move w in?

A2. 0.8(−1.28) + 0.2(1.8444) = −1.024 + 0.3689 ≈ −0.655. The combined gradient is now negative, so the SGD step (w ← w − lr·grad) increases w — the update now follows the ETA task's preference, effectively overriding the cancellation task's pull because its loss weight was cut to 0.2.

Q3. Suppose the trip-distance feature changes from x = 2 to x = 3 (a longer trip), with w = 0.5, a = 0.8, b = −1.2, and the same targets y_ETA = 1.2, y_cancel = 1 unchanged. Recompute the shared representation, both predictions, both losses, and both gradients on w — trace the full ripple, not just the most obvious quantity.

A3. The shared representation changes first, since it is the single quantity both heads depend on: h = 0.5 × 3 = 1.5 (up from 1.0). This ripples into both heads. ETA: ŷ_ETA = 0.8 × 1.5 = 1.2, which now exactly equals the true target, so L_ETA = 0 and every downstream ETA gradient collapses to zero: ∂L_ETA/∂w = 0. Cancellation: z = −1.2 × 1.5 = −1.8, ŷ_cancel = σ(−1.8) ≈ 0.1419, L_cancel = −ln(0.1419) ≈ 1.9530 (up from 1.4633, since the prediction moved further from the true label of 1), ∂L_cancel/∂z = 0.1419 − 1 = −0.8581, ∂L_cancel/∂h = −0.8581 × (−1.2) ≈ 1.0298, ∂L_cancel/∂w = 1.0298 × 3 ≈ 3.0893. The combined equal-weight gradient is now 0.5(0) + 0.5(3.0893) ≈ 1.5446 — entirely determined by the cancellation task, since the ETA task's gradient vanished at this exact point. The lesson: because w is shared, changing one input feature never affects "just one task's gradient" — it changes h, which changes every head's prediction, loss, and gradient simultaneously, even when one of those changes happens to be zero.

Q4. A colleague suggests replacing hard parameter sharing with soft parameter sharing for the ride-hailing model so that the fare task, which behaves quite differently from the other two, isn't forced through the same trunk. What is the tradeoff?

A4. Soft sharing (e.g., cross-stitch networks) keeps a separate parameter set per task and only couples them through a learned mixing penalty or combination, so a genuinely different task can partially diverge from the others rather than being squeezed into one bottleneck representation. The cost is that you no longer get hard sharing's parameter savings or its regularization strength — you are back to close to T full parameter sets, so both memory and inference cost rise roughly toward the single-task baseline, and the shared-parameter regularization effect (Caruana, 1997) that constrained overfitting is weakened.

Q5. A shared encoder has 4,000,000 parameters; each of 4 task heads has 120,000 parameters. Compute the total parameter count for (a) four independent single-task networks and (b) one hard-shared multi-task network, and the percentage reduction.

A5. Independent: 4 × (4,000,000 + 120,000) = 16,480,000. Shared: 4,000,000 + 4 × 120,000 = 4,480,000. Reduction: (16,480,000 − 4,480,000) / 16,480,000 ≈ 0.7282, a 72.8% reduction in total parameters.

Q6. True or false: training three separate networks on ETA, fare, and cancellation, then averaging their three sets of predictions on a held-out trip, is an example of multi-task learning.

A6. False — that describes ensembling. Multi-task learning requires the parameters to be shared and jointly updated by a combined loss during training, as in the shared weight w whose gradient in Q1–Q3 was the sum of contributions from every task's loss. Three networks trained in isolation, however their outputs are later combined, never let one task's gradient influence another task's parameters, so no cross-task knowledge transfer of the kind MTL is built for ever occurs.

Think About It

Think about this: How would you explain multi-task learning: sharing knowledge across tasks to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where multi-task learning: sharing knowledge across tasks is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting multi-task learning: sharing knowledge across tasks to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind multi-task learning: sharing knowledge across tasks, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.