Few-Shot Learning: Learning from Limited Examples

The problem: enrolling a face with three selfies

Open any UPI app for the first time and it asks for a short face scan — three or four frames, a blink, a head turn. From that point on, the app must recognise your face at login, reject an impostor holding your phone, and do this correctly for every one of the tens of millions of users who have ever enrolled, most of whom it has never seen a single training image of before you registered. The bank did not retrain its face-recognition model when you signed up. It could not have: a production face-verification CNN can carry 20-50 million parameters, and fine-tuning that network to convergence takes thousands of gradient steps and thousands of labelled images per class. A bank onboarding forty lakh new UPI users a month cannot run that pipeline forty lakh times. Yet the system works, correctly, from three images, the moment you finish enrolling.

This is few-shot learning: building a system that can correctly classify a brand-new category after seeing only a handful of labelled examples of it — often as few as one (one-shot learning) — with no retraining of the underlying network. It is the chapter's actual subject, and it belongs to a broader field called meta-learning, "learning to learn." Few-shot learning is the problem — classify well from K examples of a new class. Meta-learning is the method most modern few-shot systems use to solve it: instead of training a model to solve one fixed task, you train it on thousands of small, related tasks so that it becomes good at rapidly solving a new one. Every algorithm in this chapter is a meta-learning algorithm applied to the few-shot problem.

Why ordinary supervised learning cannot do this

A standard classifier — the kind you built in earlier chapters, softmax over a fixed set of output classes trained by minimising cross-entropy on thousands of labelled images per class — fails a few-shot task in two distinct ways. First, with only 1-5 images per class, gradient descent on a 25-million-parameter network overfits catastrophically; the network can memorise three photographs perfectly and still learn nothing about the person's face that generalises to a fourth photograph taken in different light. Second, and more fundamentally, the set of classes itself is not fixed. Every new UPI user is a new class the network was never trained to output. A softmax layer has a fixed number of output neurons decided at training time — you cannot add a neuron for user forty-lakh-and-one without re-architecting and retraining the network.

Few-shot learning sidesteps both problems by changing what the network is trained to output. Instead of training it to name a fixed set of classes, you train it to produce a good embedding — a vector representation of an input image such that images of the same underlying identity land close together in that vector space, and images of different identities land far apart. Classifying a new class then becomes a geometry problem, not a retraining problem: place the K support examples of the new class in that embedding space, summarise where they cluster, and classify anything new by which cluster it is closest to. The embedding network itself never needs to change when a new class arrives.

Formalising the task: N-way K-shot episodes

Few-shot classification tasks are described by two numbers. An N-way K-shot task presents N classes, each represented by K labelled support examples (together, the support set S), and asks the model to classify one or more unlabelled query examples (the query set Q) into one of the N classes. Your bank's face check is a 1-way verification problem in the limit (is this you, yes or no) built on top of a general N-way K-shot embedding; the Aadhaar-style "match this face against your enrolled record" setting is closer to 1-way K=3-5-shot. A more classical benchmark task — used in the actual research papers this chapter is built on — is 5-way 1-shot classification of handwritten characters from the Omniglot dataset (1,623 characters across 50 alphabets, 20 examples each) or 5-way 5-shot classification of natural images from miniImageNet. The number of classes seen at deployment (N) and examples per class (K) are small by design — that is the entire point of the benchmark — but do not confuse this with how much data the overall system trains on, a point the next section returns to directly because it is the single most common place students go wrong.

The meta-learning trick: an embedding trained across thousands of episodes

Here is the mechanism, using the specific algorithm this chapter works through in full: Prototypical Networks (Snell, Swersky & Zemel, 2017). Training happens on a large base dataset with many classes and plenty of examples per class — for the bank, this is a huge pre-existing face dataset with thousands of identities, none of which are the actual UPI customers who will enrol later. Training proceeds in episodes, and each episode is constructed to look exactly like the few-shot task the network will face at deployment:

1. Sample N classes at random from the base dataset.
2. Sample K support images and Q query images per sampled class.
3. Run every support and query image through the embedding network f_θ to get vectors in ℝ^d.
4. For each class, compute its prototype — the mean of that class's K support embeddings.
5. For each query embedding, compute the (squared Euclidean) distance to every prototype, turn the negative distances into a probability distribution via softmax, and compute the cross-entropy loss against the query's true class.
6. Backpropagate that loss into θ and update the embedding network — then throw the episode away and sample a completely new set of N classes for the next episode.

Because a fresh, random set of N classes is sampled every episode, the network is never allowed to memorise a fixed class list — the only way to get the loss down across thousands of different, randomly composed episodes is to learn an embedding function that puts any two images of the same underlying identity close together and any two different identities far apart, as a general property of the function, not as a property of any particular class. That generality is exactly what transfers when the network meets your three enrolment selfies, a set of classes (a single new "class": you) it never saw during training. At deployment, θ is frozen. No gradient step happens. The bank's server computes your prototype from three embeddings — one forward pass each — and every future login is a nearest-prototype lookup.

Worked example: classifying a query image with Prototypical Networks

Take a toy 2-dimensional embedding space (real systems use 128-512 dimensions; 2D keeps the arithmetic checkable by hand) and a 3-way, 2-shot episode with three digit classes — "3", "7", "9" — already passed through f_θ:

Class "3" (A): support embeddings (1,1) and (1,3)
Class "7" (B): support embeddings (4,1) and (6,1)
Class "9" (C): support embeddings (1,5) and (3,5)
Query image q: embedding (2,2), true label "3"

Step 1 — compute prototypes (mean of each class's support embeddings):

proto_A = ((1+1)/2, (1+3)/2) = (1, 2)
proto_B = ((4+6)/2, (1+1)/2) = (5, 1)
proto_C = ((1+3)/2, (5+5)/2) = (2, 5)

Step 2 — squared Euclidean distance from q=(2,2) to each prototype:

d2(q,A) = (2-1)^2 + (2-2)^2 = 1 + 0 = 1
d2(q,B) = (2-5)^2 + (2-1)^2 = 9 + 1 = 10
d2(q,C) = (2-2)^2 + (2-5)^2 = 0 + 9 = 9

Step 3 — softmax over the negative distances to get class probabilities. This is the entire trick of Prototypical Networks: treat -distance as a logit, exactly the way a softmax classifier treats a linear layer's output as a logit.

logits: (-1, -10, -9)
exp(-1)  = 0.367879
exp(-10) = 0.0000454
exp(-9)  = 0.0001234
Z = 0.367879 + 0.0000454 + 0.0001234 = 0.368048

P(A) = 0.367879 / 0.368048 = 0.9995
P(B) = 0.0000454 / 0.368048 = 0.0001
P(C) = 0.0001234 / 0.368048 = 0.0003
(sum = 1.0000, within rounding)

The network predicts class A ("3") with 99.95% confidence — correctly, since q sits almost exactly on top of proto_A on the y-axis and far from the other two prototypes. Here is the same computation traced through code, so you can check it independently:

import numpy as np

support = {
    "A": np.array([[1, 1], [1, 3]]),
    "B": np.array([[4, 1], [6, 1]]),
    "C": np.array([[1, 5], [3, 5]]),
}
prototypes = {k: v.mean(axis=0) for k, v in support.items()}
# {'A': [1. 2.], 'B': [5. 1.], 'C': [2. 5.]}

query = np.array([2, 2])
sq_dist = {k: np.sum((query - p) ** 2) for k, p in prototypes.items()}
# {'A': 1.0, 'B': 10.0, 'C': 9.0}

logits = {k: -d for k, d in sq_dist.items()}
exp_l = {k: np.exp(v) for k, v in logits.items()}
Z = sum(exp_l.values())
probs = {k: v / Z for k, v in exp_l.items()}
# probs approximately {'A': 0.9995, 'B': 0.0001, 'C': 0.0003}

One more fact worth deriving, because it explains why this simple recipe works so well: expand the squared distance, ||q - c||² = ||q||² - 2 q·c + ||c||². The term ||q||² is identical for every class k, so it shifts every logit by the same constant — and softmax is invariant to a constant shift added to all logits (softmax(x + c) = softmax(x), since the constant factors out of both numerator and denominator). That means classification by nearest-prototype-under-softmax is exactly equivalent to a linear classifier with weights w_k = 2·c_k and bias b_k = -||c_k||², applied directly to the query embedding. Check it: for class A, w_A = (2,4), b_A = -(1²+2²) = -5, so w_A·q + b_A = 2(2)+4(2)-5 = 7. For class B, w_B=(10,2), b_B=-26, giving 10(2)+2(2)-26 = -2. For class C, w_C=(4,10), b_C=-29, giving 4(2)+10(2)-29 = -1. Each of these is exactly 8 more than the corresponding -distance value (-1, -10, -9) — and 8 = ||q||² = 2²+2², the constant that cancels out of softmax. Prototypical Networks with squared Euclidean distance are, underneath the geometric picture, a linear classifier whose weight vector for each class is fixed at inference time to twice that class's prototype. This is why they generalise from so few support examples: there are no weights left to overfit.

A second path: gradient-based meta-learning (MAML)

Prototypical Networks are a metric-based approach — the model architecture itself is a fixed similarity computation, and meta-training only ever adjusts the embedding function. Model-Agnostic Meta-Learning (MAML), introduced by Finn, Abbeel and Levine in 2017, takes a different route: instead of learning a metric, it learns a starting point. MAML meta-trains an initial parameter vector θ such that, for a new task's support set, a handful of ordinary gradient descent steps starting from θ produces a model that performs well on that task's query set. Concretely, for each sampled episode MAML computes an "inner loop" update θ' = θ - α∇L_support(θ), then measures the loss of θ' on the query set, and the "outer loop" backpropagates that query loss through the inner update all the way back into θ. After meta-training, adapting to a genuinely new class still means taking a few real gradient steps on its K support examples — unlike Prototypical Networks, which take zero gradient steps at deployment and instead just do a forward pass and a distance computation.

The trade-off is real and worth naming precisely: MAML is architecture-agnostic — the same recipe works for classification, regression, or reinforcement-learning policies, because "a few gradient steps produce a good model" makes no assumption about what kind of model it is. Prototypical Networks are specialised to classification and assume that a simple distance in embedding space is a sufficient decision rule, which is a strong assumption but one that, when it holds, is far cheaper at test time (one forward pass, no gradients, no second derivatives to compute or approximate) and far more stable to train with very small K. A bank's face-verification system is a textbook case for the metric-based route: the task is always "which enrolled identity is this," the model architecture never changes, and inference-time speed and stability matter enormously at production scale — which is why systems in this space are built on embedding networks and nearest-neighbour or nearest-prototype matching rather than on per-user gradient adaptation.

How the two phases fit together

Common misconception: "few-shot" means the model barely trained on any data

The name invites exactly the wrong intuition. Students consistently assume that if a model can classify from three examples, it must be a small, lightly-trained model — the opposite of "big data." The truth is the reverse: the embedding network in the worked example above was meta-trained on a base dataset with many classes and abundant examples per class, across thousands of sampled episodes, precisely so that it would generalise well to a class it has never seen with only K examples. The "few" in few-shot learning describes the support set size for the novel class at deployment time — it says nothing about the total training data the system consumed to become capable of that trick. A production few-shot face-verification network is typically pretrained on datasets with hundreds of thousands of identities and millions of images before it ever meets your three enrolment selfies. Confusing "few labelled examples of the new class" with "a lightly-trained model" is the single most common way to misread this topic, and it inverts the actual engineering effort: meta-training an embedding good enough to generalise from three shots is usually a larger data and compute investment than training an ordinary fixed-class classifier, not a smaller one.

Active recall

Attempt these before reading the worked answers below.

1. A photo-storage app lets you tag a person's name after providing just two example photos, and it then finds that person in your whole gallery. Explain, in terms of episodes and prototypes, why an ordinary fixed-output-layer classifier could not be retrained fast enough to do this per user, and what the app is doing instead.
2. Define "5-way 1-shot" precisely: what does the 5 count, what does the 1 count, and what is the query set for?
3. A 3-way, 1-shot episode has support embeddings A=(0,0), B=(4,0), C=(2,4), and a query q=(1,1). Compute the squared distance from q to each prototype, then the softmax probabilities. Which class does the model predict, and with roughly what confidence?
4. True or false, with justification: "Few-shot learning models need very little data overall, which is why they're useful when data is scarce."
5. What is the one sentence that distinguishes MAML from Prototypical Networks in terms of what happens computationally at test time on the new class's support set?
6. In the derivation connecting Prototypical Networks to a linear classifier, which term of the expanded squared distance is responsible for the equivalence holding, and why does it not affect the predicted class?

Worked answers

1. A fixed-output classifier has one softmax neuron per class, decided at training time; adding a new person means adding a neuron and retraining the network on all classes, which cannot happen per user in real time. The app instead runs a face embedding network — meta-trained once, offline, across many episodes of many people's photos — and freezes it. When you tag a person from two photos, the app computes those two photos' embeddings, averages them into a prototype for that name, and thereafter classifies any new photo by nearest prototype. No retraining occurs at tagging time.

2. "5-way" means 5 classes appear in the episode (the support set spans 5 categories). "1-shot" means each of those 5 classes is represented by exactly 1 labelled support example (K=1), so the support set has 5 images total. The query set is a separate batch of unlabelled examples from those same 5 classes that the model must classify — it is what the episode's loss (during training) or the deployment prediction (during testing) is measured against; it is never included in the support set used to build prototypes.

3. Prototypes equal the single support example per class here (K=1): proto_A=(0,0), proto_B=(4,0), proto_C=(2,4). Squared distances from q=(1,1): d²(q,A)=(1-0)²+(1-0)²=2; d²(q,B)=(1-4)²+(1-0)²=9+1=10; d²(q,C)=(1-2)²+(1-4)²=1+9=10. Logits: (-2,-10,-10). exp(-2)=0.13534, exp(-10)=0.0000454 (for both B and C). Z=0.13534+0.0000454+0.0000454=0.13543. P(A)=0.13534/0.13543≈0.9993, P(B)=P(C)≈0.0000454/0.13543≈0.0003 each. Predicted class: A, with about 99.9% confidence — B and C are equidistant from q and share the remaining probability equally.

4. False. Few-shot learning needs very little labelled data for the new class at deployment, but the embedding or initialisation that makes this possible is itself built from a large base dataset with many classes, trained across thousands of episodes. It is useful when data is scarce for the specific new category you care about, not when data is scarce overall — the meta-training phase typically requires more total data than training a single ordinary classifier would.

5. Prototypical Networks take zero gradient steps at test time — the new class is handled by one forward pass through the frozen embedding network plus a distance computation; MAML takes a few real gradient-descent steps on the new class's support set, starting from the meta-learned initial parameters, before it can classify the query set.

6. The term ||q||² (the squared norm of the query alone, with no dependence on the class k) is what makes the equivalence work: expanding ||q-c_k||² produces ||q||² - 2q·c_k + ||c_k||², and since ||q||² is identical across every class k in a given episode, it acts as one constant added to all logits. Softmax is invariant to adding the same constant to every logit, so dropping that term changes the absolute logit values but leaves every predicted probability, and therefore the predicted class, unchanged.

Think About It

Think about this: How would you explain few-shot learning: learning from limited examples to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where few-shot learning: learning from limited examples is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting few-shot learning: learning from limited examples to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind few-shot learning: learning from limited examples, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.