Semantic Similarity: Understanding Meaning

Riya and Aman are both submitting a Class 10 Social Science project on water conservation. Their school, like a growing number of schools and colleges in India, runs written submissions through an originality-checking tool before a teacher even opens them. Riya writes: "We must save water to protect our environment for future generations." Aman is short on time. He does not copy Riya's sentence outright — that would be caught instantly — so he rewrites it in his own words: "We ought to conserve water in order to protect our planet for the generations to come." Every important idea in Aman's sentence is identical to Riya's. Every important word has simply been swapped for a synonym: save became conserve, environment became planet. A teacher reading both sentences would spot the resemblance in ten seconds. But if the originality checker's method is simply "count how many words the two sentences share," it will report something far less alarming — and, as you are about to compute yourself, dangerously misleading. The gap between what a machine sees (a bag of tokens) and what a human sees (a meaning) is exactly the subject of this chapter: how do we get a computer to measure semantic similarity — how close two pieces of text are in meaning — instead of merely how many words they happen to share?

Lexical Overlap: The First, Flawed Idea

Before reaching for anything sophisticated, the obvious first move is to compare the literal words. This is called lexical similarity — similarity measured by shared tokens, with no notion of meaning built in at all. If you have studied set theory, the natural way to measure it will look immediately familiar: treat each sentence as a set of words and measure how much the two sets overlap. This measure has a name, Jaccard similarity, defined for two sets A and B as:

J(A, B) = |A ∩ B| / |A ∪ B|

In words: divide the size of the intersection (words appearing in both) by the size of the union (every distinct word appearing in either). A score of 1 means the sets are identical; a score of 0 means they share nothing.

Let's apply this properly to Riya's and Aman's sentences. The first real step any text-comparison tool takes is stopword removal — stripping out very common connector words ("we," "to," "the," "in order to," helper words like "must" and "ought") that carry grammatical scaffolding but almost no topical content, so the comparison isn't swamped by words that nearly every sentence contains anyway. After stopword removal:

Riya's content words = {save, water, protect, environment, future, generations}
Aman's content words = {conserve, water, protect, planet, generations, come}

Now find the intersection by checking each word: save vs conserve — different tokens, no match. water — appears in both. protect — appears in both. environment vs planet — different tokens, no match. future — only in Riya's set. generations — appears in both. come — only in Aman's set. That gives an intersection of exactly three words: {water, protect, generations}. The union collects every distinct word from both sets: Riya's six words, plus the three of Aman's that weren't already counted (conserve, planet, come), for nine words total. So:

J(Riya, Aman) = 3 / 9 = 0.333

Only a third of the words overlap — comfortably below any reasonable "flag this as copied" threshold a school might set. Meanwhile, a third student, Kabir, submits a project on an unrelated topic: "UPI has transformed digital payments across India," with content words {upi, transformed, digital, payments, india}. Comparing this against Riya's set gives zero shared words — intersection size 0, union size 11 — so J(Riya, Kabir) = 0.0, correctly identifying two genuinely unrelated pieces of writing.

Here is the same computation in Python, matching every number above exactly:

def jaccard_similarity(set_a, set_b):
    intersection = set_a & set_b
    union = set_a | set_b
    return len(intersection) / len(union)

riya = {"save", "water", "protect", "environment", "future", "generations"}
aman = {"conserve", "water", "protect", "planet", "generations", "come"}
kabir = {"upi", "transformed", "digital", "payments", "india"}

print(round(jaccard_similarity(riya, aman), 3))    # 0.333
print(round(jaccard_similarity(riya, kabir), 3))   # 0.0

Jaccard similarity did exactly what it was designed to do: count shared tokens. The trouble is that "designed to do" and "what we actually needed" are two different things. Kabir's genuinely unrelated essay and Aman's barely-disguised copy score 0.0 and 0.333 respectively — both low, with Aman's not dramatically higher than pure unrelatedness, even though Aman's essay says exactly the same thing as Riya's. Lexical similarity has no concept of "conserve means the same as save." To fix that, we need to go back to representing words by meaning, not by spelling.

From Words to Meaning: Bringing Back Vectors

Recall the core idea of word embeddings: every word is assigned a vector — a list of numbers, a single point in a multi-dimensional space — learned so that words appearing in similar contexts across huge amounts of text end up with similar vectors. "Save" and "conserve" show up in near-identical contexts across millions of sentences ("___ money," "___ energy," "please ___ water"), so a well-trained embedding model places their vectors close together, pointing in nearly the same direction. "Waste" — used in the opposite kind of context — gets pushed toward a very different direction. This single fact is exactly the fix Jaccard similarity was missing: two words can be totally different tokens while sitting right next to each other in meaning-space.

But "close together" and "pointing in nearly the same direction" are two different geometric ideas, and it turns out the choice between them matters enormously for text. The rest of this chapter is about making that choice precisely.

Two Ways to Measure "Close": Distance and Direction

Given two vectors, there are two natural ways to ask how similar they are.

The first is plain Euclidean distance — the straight-line distance between the two points. You have already met this exact idea in coordinate geometry, usually in two dimensions, as the distance formula: for points (x₁, y₁) and (x₂, y₂), the distance is √((x₂−x₁)² + (y₂−y₁)²). Word vectors simply extend this to many more dimensions, but the formula's shape is identical — square each coordinate difference, add them up, take the square root. A smaller distance is supposed to mean "more similar."

The second is cosine similarity — instead of measuring the gap between the tips of two vectors, it measures the angle between them, ignoring how long each vector is:

cos(theta) = (a . b) / (|a| |b|)

Here a · b is the dot product (multiply each pair of matching coordinates and add the results: a·b = a₁b₁ + a₂b₂ + ... + aₙbₙ), and |a| is the vector's norm, or magnitude — its own length, computed the same way as Euclidean distance but measured from the origin: |a| = √(a₁² + a₂² + ... + aₙ²). The dot product alone already carries a strong signal: it comes out large and positive when two vectors point in nearly the same direction, close to zero when they are perpendicular (unrelated), and negative when they point in roughly opposite directions. Dividing by both magnitudes strips away the effect of each vector's length entirely, leaving a pure measure of direction agreement that always falls between −1 (exactly opposite) and +1 (exactly aligned).

Both formulas are legitimate, well-defined ways to compare two vectors. The natural question — the one every real NLP system had to answer — is which one actually matches how humans judge whether two pieces of text mean the same thing. That question has a definite, provable answer, and the next section derives it.

A Worked Example: Why Cosine Similarity Wins

Set up three small, hand-computable word vectors in two dimensions (real embeddings use hundreds of dimensions; two is just enough to compute — and later draw — by hand):

save     = (3, 4)
conserve = (4, 3)
waste    = (4, -3)

First, compute Euclidean distance for both pairs. For save and conserve:

d = sqrt((4-3)^2 + (3-4)^2) = sqrt(1^2 + (-1)^2) = sqrt(1 + 1) = sqrt(2) ≈ 1.414

For save and waste:

d = sqrt((4-3)^2 + (-3-4)^2) = sqrt(1^2 + (-7)^2) = sqrt(1 + 49) = sqrt(50) ≈ 7.071

So far, so sensible: conserve (distance 1.414) sits much closer to save than waste does (distance 7.071). Now compute cosine similarity for the same pairs. For save and conserve, first the dot product, then each magnitude, then the ratio:

dot product = (3)(4) + (4)(3) = 12 + 12 = 24
|save|      = sqrt(3^2 + 4^2) = sqrt(9 + 16) = sqrt(25) = 5
|conserve|  = sqrt(4^2 + 3^2) = sqrt(16 + 9) = sqrt(25) = 5
cos(theta)  = 24 / (5 * 5) = 24 / 25 = 0.96

For save and waste:

dot product = (3)(4) + (4)(-3) = 12 - 12 = 0
|waste|     = sqrt(4^2 + (-3)^2) = sqrt(16 + 9) = sqrt(25) = 5
cos(theta)  = 0 / (5 * 5) = 0

Both metrics currently tell the same story: conserve is highly similar to save (cosine 0.96, close to the maximum of 1), while waste is unrelated to it — a cosine of exactly 0 means the two vectors are perpendicular, carrying no directional agreement at all. So why prefer one metric over the other? Because they stop agreeing the moment vector length enters the picture — and in real text, length varies constantly. A single word is short; a whole paragraph that keeps circling back to the same idea produces a "longer," more emphatic vector pointing the same way. Model exactly that: suppose instead of the single word conserve, we compare save against an entire paragraph that repeats the conservation theme four times over. Its vector points in exactly the same direction as conserve, just scaled up:

paragraph = 4 x conserve = (16, 12)

Euclidean distance between save and this paragraph:

d = sqrt((16-3)^2 + (12-4)^2) = sqrt(13^2 + 8^2) = sqrt(169 + 64) = sqrt(233) ≈ 15.264

Euclidean distance has ballooned to 15.264 — more than ten times larger than the distance to the single word conserve (1.414) — suggesting the paragraph is wildly dissimilar to "save." But the paragraph isn't about anything different; it is about the exact same thing, said more thoroughly. Now compute cosine similarity for the same pair:

dot product = (3)(16) + (4)(12) = 48 + 48 = 96
|paragraph| = sqrt(16^2 + 12^2) = sqrt(256 + 144) = sqrt(400) = 20
cos(theta)  = 96 / (5 * 20) = 96 / 100 = 0.96

Exactly 0.96 — identical to cos(save, conserve), down to the last digit. This is not a coincidence, and it is worth proving rather than just observing. For any positive number k, scaling a vector b to k·b multiplies both the dot product and the magnitude by k, and the two k's cancel exactly:

cos(a, k*b) = (a . k*b) / (|a| * |k*b|) = k*(a . b) / (|a| * k*|b|) = (a . b) / (|a| * |b|) = cos(a, b)

Scaling a vector changes its length but never its direction, and cosine similarity is built, by construction, to measure direction alone. This is why NLP systems overwhelmingly reach for cosine similarity rather than Euclidean distance when comparing text: real documents vary enormously in length, and a trustworthy similarity measure should not penalise a paragraph for restating its point more thoroughly, only for being about something genuinely different.

Here is the full computation traced in code, reproducing every number above:

import numpy as np

def euclidean_distance(a, b):
    return np.sqrt(np.sum((a - b) ** 2))

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

save = np.array([3, 4])
conserve = np.array([4, 3])
waste = np.array([4, -3])
paragraph = 4 * conserve          # same direction as "conserve", 4x the length

print(round(euclidean_distance(save, conserve), 3))   # 1.414
print(round(euclidean_distance(save, paragraph), 3))  # 15.264
print(round(cosine_similarity(save, conserve), 3))    # 0.96
print(round(cosine_similarity(save, paragraph), 3))   # 0.96  -- unchanged
print(round(cosine_similarity(save, waste), 3))       # 0.0

The diagram below shows why: save, conserve, and the scaled-up paragraph vector all sit on rays close together, differing only in how far out along that ray each one reaches, while waste points off at a sharp angle.

Meaning from Dictionaries: The Symbolic Alternative

Embeddings are not the only way to measure semantic similarity, and it's worth knowing the alternative because it solves a different problem. WordNet, developed at Princeton University beginning in the 1980s under psychologist George Miller, is a large hand-built lexical database of English. Instead of learning vectors from text, WordNet groups words into synsets — sets of words that are interchangeable in some context because they express the same underlying concept — and links those synsets into a hierarchy through relations like hypernymy ("is a kind of"). "Car" and "automobile" sit in the very same synset. "Car" and "truck" are both a short hop below the more general concept "motor vehicle." "Car" and "banana" share no useful path at all except by climbing all the way up to something extremely general, like "physical entity." A word-similarity score can be built directly from this structure: the fewer hops needed to connect two words through the hierarchy, the more similar they are judged to be.

This approach has a genuinely different profile of strengths and weaknesses from embeddings. It needs no training corpus at all — the hierarchy was built once, by hand, by human lexicographers — so it works even for rare words that little text has ever been written about, and its reasoning is fully explainable: you can literally point to the chain of hypernyms connecting two words. But it only knows what humans explicitly wrote into it. Brand names, slang, and newly coined technical terms are often missing entirely, and a resource like this has to be built separately for every language — an English WordNet cannot tell you anything about two Hindi or Tamil words. Embeddings, trained automatically from huge amounts of real text, cover almost any word that appears often enough in that text and extend more readily to other languages, given enough data in each — at the cost of needing that data in the first place, and of being only as unbiased as the text they were trained on.

Choosing the Right Tool

Four tools, four different jobs:

• Jaccard similarity is fast and needs no trained model at all — a good fit for catching near-verbatim duplicates, comparing tag sets, or a quick first pass before something more expensive runs. It is blind to synonyms and word order.
• Cosine similarity on word or sentence embeddings is the default choice for meaning-aware comparison in modern NLP — semantic search, paraphrase and plagiarism detection, recommendation — precisely because it ignores document length and focuses purely on direction, as proven above.
• Euclidean distance remains the right tool when comparing fixed-length numeric feature vectors that are already on comparable scales, for instance in k-nearest-neighbours classification over measurements like height and weight, but is risky for text, where vector length often reflects how much was written rather than what it means.
• WordNet-style symbolic similarity is useful when no training corpus is available, when the vocabulary is small and well-established, or when an explainable, rule-based answer matters more than a learned one.

Where Semantic Similarity Runs Every Day

An originality checker that embeds sentences and compares them by cosine similarity, rather than counting shared words, is exactly the fix Riya and Aman's story needed — and it is only one of many places this same machinery runs quietly in the background. Search on an e-commerce site has to match a query such as "wireless earbuds" against product listings titled "Bluetooth in-ear headphones" — different words, same product category, solvable only by meaning-based comparison, not spelling. Job portals match a candidate's résumé, phrased in their own words, against job descriptions that a different recruiter wrote in completely different wording, using the same underlying idea. Search engines route queries like "best budget phone" and "affordable smartphone recommendations" to overlapping sets of results because their embeddings land close together, not because the words match.

Check Your Understanding

1. Two more snippets: "reduce plastic waste" has content words {reduce, plastic, waste}; "cut plastic pollution" has content words {cut, plastic, pollution}. Compute the Jaccard similarity.
2. Compute the cosine similarity between a = (2, 2) and b = (5, 5) by hand, showing the dot product and both magnitudes.
3. Why does Jaccard similarity fail on Riya's and Aman's essays even though a teacher instantly sees they say the same thing?
4. Why is cosine similarity, not Euclidean distance, the standard choice for comparing text embeddings?
5. Two ordinary English words never once appear near each other in the huge corpus a word-embedding model was trained on, so the model has learned little about how they relate. Which approach from this chapter could still relate them, and how?

Answers. (1) Intersection = {plastic}, size 1; union = {reduce, plastic, waste, cut, pollution}, size 5; J = 1/5 = 0.2. (2) Dot product = (2)(5) + (2)(5) = 20; |a| = √(4+4) = √8 = 2√2 ≈ 2.828; |b| = √(25+25) = √50 = 5√2 ≈ 7.071; cos θ = 20 / (2.828 × 7.071) = 20/20 = 1.0 — exactly aligned, because b is simply 2.5 × a, and cosine similarity is unaffected by scaling. (3) Jaccard only counts identical tokens; "save"/"conserve" and "environment"/"planet" are synonym pairs that occupy different token slots, so despite meaning the same thing they contribute nothing to the intersection, badly undercounting the true similarity. (4) Real text varies hugely in length, and Euclidean distance is sensitive to vector magnitude, which for text often reflects length or repetition rather than meaning; cosine similarity is scale-invariant (proven above), so it judges a short paraphrase and a longer restatement of the same idea as equally similar, matching how a human would judge them. (5) WordNet-style symbolic similarity, since it does not depend on how often two words co-occur in any corpus — it only requires that both words already exist somewhere in the hand-built hierarchy, related through shared hypernyms, however rarely they happen to appear together in real text.

Summary

Semantic similarity is the problem of measuring how close two pieces of text are in meaning, and getting it right matters precisely because meaning survives rewording while literal words do not. Lexical methods like Jaccard similarity — intersection over union of the word sets — are fast and need no training, but, as Riya and Aman's essays showed with hard numbers, they badly undercount true similarity whenever a paraphrase swaps in synonyms: a score of just 0.333 for two sentences that say the exact same thing. Representing words as embeddings and comparing them geometrically fixes this, but only if the right geometry is chosen: Euclidean distance is sensitive to vector length, while cosine similarity — the dot product divided by both magnitudes — depends on direction alone, provably unchanged when a vector is scaled up, which is exactly the property text similarity needs given how wildly document length varies. WordNet-style symbolic similarity offers a third, training-free path built on hand-curated dictionaries rather than learned vectors. Run Aman's actual sentence through a properly built semantic-similarity pipeline — sentence vectors compared by cosine similarity, not literal word overlap — and it would score close to 1, correctly flagging what a naive checker, counting only shared words, was always going to miss.

Think About It

Think about this: How would you explain semantic similarity: understanding meaning to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where semantic similarity: understanding meaning is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting semantic similarity: understanding meaning to at least 3 other topics you have studied.