Open the Google News app or Inshorts on your phone and search for a major event — say, an India–Pakistan T20 match, or a Reserve Bank of India interest rate announcement. You will notice something that feels obvious once you see it, but is technically remarkable: dozens of articles from completely different publishers, from The Hindu to NDTV to ESPNcricinfo, get automatically bundled under a single story card. No human editor at the app sat down and manually tagged each one. Nobody typed a rule saying "these seventeen articles are all about the same match." The app worked that out on its own, purely from the words in the text.
That is document clustering at work: an algorithm that is handed a pile of text documents with no labels attached, and groups the similar ones together based on what they actually say. This chapter builds that capability from first principles — starting with how a computer can tell two pieces of text are "similar" at all, moving through the most widely used clustering algorithm, K-Means, and finishing with a working Python implementation you can run on your own headlines.
Unsupervised Learning: Finding Structure Without Labels
Earlier in this course you likely built a text classifier — a model trained on examples that were already labeled, such as emails marked "spam" or "not spam." That is supervised learning: the model learns a mapping from input to a known, correct output, because a human supplied the correct answers during training.
Document clustering belongs to a different family entirely: unsupervised learning. There are no labels anywhere in the process. Nobody has gone through the pile of news articles marking which ones are about cricket and which are about the stock market. The algorithm is simply handed a collection of documents and asked to find whatever natural structure exists — groups of items that are more similar to each other than they are to everything else. The output is not a prediction of a known category; it is a discovery of categories that were never explicitly defined anywhere. This is exactly why it suits a news aggregator so well: nobody can pre-write a label for every story that might ever be published, so the system has to organize incoming articles purely by how similar they are to one another, as they arrive.
Turning Headlines into Vectors
Before any clustering can happen, text has to become numbers. A clustering algorithm has no built-in notion of "cricket" or "finance" — it only knows how to compare numeric vectors. The standard approach, which you have likely already met as TF-IDF (Term Frequency–Inverse Document Frequency) or its simpler cousin bag-of-words, converts each document into a vector where every dimension corresponds to one word in the overall vocabulary. If the vocabulary across all documents contains 500 unique words, every document becomes a 500-dimensional vector, with each entry recording how important that word is to that particular document — a plain count for bag-of-words, or a count that is boosted for rare, distinctive words and suppressed for common ones, for TF-IDF.
Once every document is a vector, "similar documents" gets a precise geometric meaning: similar documents are vectors that sit close together in that high-dimensional space. Two headlines about the same cricket match will both have large entries for words like "wicket," "over," and "innings," and near-zero entries for words like "rupee" or "inflation," so their vectors point in roughly the same direction and land near each other. A finance headline does the opposite. Clustering, at its core, is the task of finding groups of nearby vectors.
Measuring How Close Two Documents Are
To group nearby vectors we first need a precise, computable definition of "near." The most familiar option is Euclidean distance — ordinary straight-line distance, the same idea used to find the distance between two points on a graph: square the difference along each dimension, add the squares up, and take the square root. In Python, for vectors of any length:
import math
def euclidean_distance(a, b):
return math.sqrt(sum((a[i] - b[i]) ** 2 for i in range(len(a))))
def cosine_similarity(a, b):
dot = sum(a[i] * b[i] for i in range(len(a)))
mag_a = math.sqrt(sum(x ** 2 for x in a))
mag_b = math.sqrt(sum(x ** 2 for x in b))
return dot / (mag_a * mag_b)
P, Q = (1, 1), (4, 5)
U, V = (2, 2), (8, 8)
print(round(euclidean_distance(P, Q), 2))
print(round(euclidean_distance(U, V), 2))
print(round(cosine_similarity(U, V), 2))This prints:
5.0
8.49
1.0euclidean_distance(P, Q) is the easy one to check by hand: P and Q differ by 3 along one axis and 4 along the other, and sqrt(3^2 + 4^2) = sqrt(25) = 5. But look at the second and third lines together, because they reveal exactly why Euclidean distance is a poor fit for text. U = (2, 2) and V = (8, 8) can be thought of as two documents that mention the same two topics in exactly the same proportion — V simply repeats the same pattern four times as much, the way a long article covers the same story as a short one, just at greater length. Topically, U and V are identical; V is literally U scaled up. Yet their Euclidean distance is 8.49, large purely because one document happens to be longer.
Cosine similarity fixes this by ignoring magnitude entirely and measuring only the angle between two vectors — the dot product of the two vectors, divided by the product of their lengths. For U and V: the dot product is 2×8 + 2×8 = 32; the length of U is sqrt(2^2+2^2) = sqrt(8) ≈ 2.83; the length of V is sqrt(8^2+8^2) = sqrt(128) ≈ 11.31; and 32 / (2.83 × 11.31) ≈ 1.0, matching the code's output exactly. A cosine similarity of 1 means two vectors point in exactly the same direction — maximally similar — while 0 means they are perpendicular, sharing no common pattern at all. Cosine similarity correctly reports that U and V are perfectly aligned in topic, regardless of the fact that Euclidean distance saw them as far apart. This is precisely why real text-clustering systems, including the one built later in this chapter, favour cosine similarity over raw Euclidean distance.
The K-Means Algorithm
K-Means is the most widely used clustering algorithm, and it works directly with the vectors and distances described above. "K" is simply the number of clusters you want, chosen up front, and "means" refers to the fact that each cluster is represented by the average — the centroid — of the points currently assigned to it. The algorithm runs in four steps:
- 1. Choose K. Decide how many clusters to look for.
- 2. Initialize centroids. Pick K starting points in the vector space — often by choosing K of the actual documents at random — as a first guess at where each cluster's centre is.
- 3. Assign. For every document, measure its distance to each of the K centroids, and assign it to whichever cluster's centroid is nearest.
- 4. Update. Recompute each centroid as the mean position of all the documents now assigned to it.
Steps 3 and 4 then repeat: reassign every point to its nearest newly updated centroid, recompute the centroids again, and so on. The algorithm has converged once a full assignment step changes nobody's cluster — every point is already closest to the centroid it is already assigned to, so nothing moves on the next round. In practice K-Means converges quickly, usually within a handful of iterations, because each round can only decrease, never increase, the total distance between points and their assigned centroid.
Worked Example: Clustering Six Headlines by Hand
Consider six real-style headlines, three about cricket and three about markets, deliberately unlabeled so the algorithm has to discover the grouping on its own:
- H1: "India post a dominant total as the top order fires"
- H2: "Bowling change turns the match in the final overs"
- H3: "Century stand puts the chase firmly out of reach"
- H4: "Benchmark indices close higher on strong earnings"
- H5: "Central bank holds rates steady amid inflation concerns"
- H6: "Rupee slips as crude oil prices climb higher"
A real vectorizer would turn these into vectors with one dimension per vocabulary word — dozens of dimensions, too many to trace by hand. So, purely to make the arithmetic manageable, we compress each headline down to two representative numbers on a simple 0–6 scale: how strongly it reads as cricket coverage, and how strongly it reads as market coverage. This is the same idea as bag-of-words counting, just restricted to two dimensions so the whole computation fits on paper:
H1 = (5, 0) H4 = (0, 5)
H2 = (4, 1) H5 = (1, 4)
H3 = (6, 1) H6 = (1, 6)We want K = 2 clusters. Initialize the two centroids at two of the actual points — a common, simple starting choice — say C1 = H1 = (5, 0) and C2 = H4 = (0, 5).
Iteration 1 — assign. Compute each headline's Euclidean distance to both centroids:
Point Dist to C1=(5,0) Dist to C2=(0,5) Assigned to
H1 (5,0) 0.00 7.07 C1
H2 (4,1) 1.41 5.66 C1
H3 (6,1) 1.41 7.21 C1
H4 (0,5) 7.07 0.00 C2
H5 (1,4) 5.66 1.41 C2
H6 (1,6) 7.21 1.41 C2Every cricket headline lands closer to C1, and every market headline lands closer to C2, giving clusters {H1, H2, H3} and {H4, H5, H6}.
Iteration 1 — update. Recompute each centroid as the mean of its assigned points:
New C1 = mean of (5,0), (4,1), (6,1) = (15/3, 2/3) = (5.00, 0.67)
New C2 = mean of (0,5), (1,4), (1,6) = (2/3, 15/3) = (0.67, 5.00)Iteration 2 — check for changes. Reassign every point using the updated centroids:
Point Dist to C1=(5.00,0.67) Dist to C2=(0.67,5.00) Assigned to
H1 (5,0) 0.67 6.62 C1
H2 (4,1) 1.05 5.21 C1
H3 (6,1) 1.05 6.67 C1
H4 (0,5) 6.62 0.67 C2
H5 (1,4) 5.21 1.05 C2
H6 (1,6) 6.67 1.05 C2Nobody switched clusters. The assignment produced in iteration 2 is identical to iteration 1, which means the next update step would not move the centroids either — the algorithm has converged, and it found exactly the split a human would expect: cricket headlines in one cluster, market headlines in the other, without ever being told what "cricket" or "market" meant.
Choosing K: The Elbow Method
The worked example above had an advantage: we already knew there were two topics, so we set K = 2 in advance. In practice, nobody hands you the correct number of clusters — that is exactly what you are trying to discover. One standard way to choose K is the elbow method, built on a quantity called WCSS (Within-Cluster Sum of Squares), also called inertia: for a given clustering, square the distance from every point to its own cluster's centroid, and add all of those squared distances up. WCSS measures how tightly packed the clusters are — lower is better — and it is guaranteed to keep falling as K increases, since more clusters always give every point more, and closer, centroids to be near.
Running K-Means on the same six headlines for every value of K from 1 to 6, and keeping the best possible WCSS at each K, gives:
K = 1: WCSS = 61.67
K = 2: WCSS = 5.33
K = 3: WCSS = 3.67
K = 4: WCSS = 2.00
K = 5: WCSS = 1.00
K = 6: WCSS = 0.00The jump from K = 1 to K = 2 collapses the WCSS by more than 90%, from 61.67 down to 5.33 — splitting the data into cricket and market clusters removes almost all of the internal spread in a single move. Every step after that buys much less: K = 3 only shaves off another 1.67, and by K = 6 every point is simply its own cluster, forcing WCSS to exactly zero, since a cluster containing a single point has zero distance from that point to its own centroid — trivially true, and useless for finding real structure. Plotted as a graph of WCSS against K, this pattern, a steep drop followed by a long, flat tail, looks like a bent arm, and the "elbow" — the point where the curve stops falling steeply — is the recommended value of K. Here the elbow sits unmistakably at K = 2, confirming numerically what the headlines already suggested by eye.
Clustering Real Headlines with scikit-learn
Hand-tracing two-dimensional vectors builds intuition, but real documents produce vectors with dozens or hundreds of dimensions, far too many to compute by hand. In Python, scikit-learn provides both the TF-IDF vectorizer and the K-Means implementation, so the entire pipeline, raw text in and cluster labels out, takes only a few lines:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
headlines = [
"India wins the cricket match after a thrilling final over",
"The batsman scored a century in today's cricket match",
"Bowlers took five wickets to win the cricket series",
"Sensex and Nifty surge as the stock market rallies today",
"RBI cuts interest rates to support the stock market",
"Rupee strengthens against the dollar in the currency market",
]
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(headlines)
kmeans = KMeans(n_clusters=2, random_state=42, n_init=10)
kmeans.fit(X)
for headline, label in zip(headlines, kmeans.labels_):
print(f"Cluster {label}: {headline}")Running this prints:
Cluster 0: India wins the cricket match after a thrilling final over
Cluster 0: The batsman scored a century in today's cricket match
Cluster 0: Bowlers took five wickets to win the cricket series
Cluster 1: Sensex and Nifty surge as the stock market rallies today
Cluster 1: RBI cuts interest rates to support the stock market
Cluster 1: Rupee strengthens against the dollar in the currency marketA clean split, computed automatically from full sentences rather than hand-picked two-number summaries. A few details are worth unpacking. stop_words='english' tells the vectorizer to discard common filler words, such as "a," "the," "in," "after," and "to," before building the vocabulary, since these appear in almost every document regardless of topic and would only add noise to the distance calculations. TfidfVectorizer automatically builds one dimension per remaining vocabulary word and weights each entry by how distinctive that word is, exactly as described earlier in this chapter. random_state=42 simply fixes the random number generator so the centroid initialization is reproducible: rerun the code without it, and the contents of each cluster stay the same, but which cluster gets called 0 and which gets called 1 can flip, since that numbering is arbitrary and depends on where the centroids happened to start.
Beyond K-Means: A Quick Look at Other Methods
K-Means is not the only clustering algorithm, and it is worth knowing what else exists even without working through the mechanics of each one. Hierarchical clustering does not require choosing K up front at all: it starts with every document as its own cluster, then repeatedly merges the two closest clusters until only one giant cluster remains, producing a tree called a dendrogram that can be cut at any level to get however many clusters are wanted. DBSCAN (Density-Based Spatial Clustering of Applications with Noise) groups points that are densely packed together and explicitly labels sparse, isolated points as noise rather than forcing them into the nearest cluster, which is useful when a dataset genuinely contains documents that do not belong to any clean topic. Both trade away some of K-Means's simplicity and speed for flexibility K-Means does not have; which one to reach for depends on the shape of the data and on whether the number of clusters is even known in advance.
What Clustering Cannot Do
Clustering is powerful, but it is worth being precise about its limits.
It cannot tell you what a cluster means. K-Means found that {H1, H2, H3} belong together and {H4, H5, H6} belong together, but it has no concept of "cricket" or "finance" anywhere in its logic — it only knows that certain vectors sit close together in space. A human still has to look inside each cluster and attach a label. This is a fundamental property of unsupervised learning, not a flaw to be fixed: the algorithm discovers structure, not meaning.
It is sensitive to where the centroids start. K-Means's assign-then-update procedure always settles into a stable arrangement, but not necessarily the best one — a poor set of starting centroids can trap the algorithm in a mediocre split, sometimes cutting one real topic into two clusters or merging two different topics into one. It is a common misconception that scikit-learn protects against this automatically by default. It does not: since scikit-learn version 1.4, released in 2024, the KMeans parameter n_init defaults to 'auto', which resolves to running the algorithm only a single time whenever the default k-means++ initialization method is used. That is precisely why the code example above explicitly passed n_init=10: that setting overrides the default and forces scikit-learn to repeat the entire clustering process ten times, each from a different random starting point, keeping whichever run produced the lowest WCSS. Leaving n_init at its default is a reasonable choice when experimenting quickly, but explicitly raising it is good practice whenever the quality of the final clustering genuinely matters.
It has to be told K. Unlike alternatives such as DBSCAN, K-Means cannot report that a dataset does not actually contain clean groups; it will happily force whatever K it is given, even a bad one. The elbow method helps, but it is a heuristic, not a guarantee, and on messier real-world data the "elbow" is often a gentle curve rather than a sharp bend, leaving real room for judgment.
It assumes clusters are roughly round and similar in size. Because K-Means measures distance to a single central point, it struggles with clusters that are long, thin, oddly shaped, or very different in size from one another, a limitation that algorithms like DBSCAN were specifically designed to address.
From Toy Headlines to Real Products
The same mechanics scale far beyond six headlines. News aggregators cluster thousands of incoming articles to build the story cards seen on Google News and Inshorts. E-commerce platforms cluster incoming customer support messages so similar complaints get routed to the right team without a human reading every ticket first. Plagiarism and duplicate-content detectors use the same distance calculations to flag documents that sit suspiciously close together in vector space. Academic search tools cluster research papers by topic to power "related work" recommendations. In every case, the underlying question is identical to the one this chapter worked through by hand: convert text into vectors, define a notion of distance, and group whatever sits close together.
The next time a news app quietly bundles a dozen articles from a dozen different newsrooms into a single story, without a single human editor deciding that they belong together, that is not magic. It is TF-IDF vectors, a distance metric, and an algorithm not so different from the six-headline trace worked through above, running at a scale of millions of articles instead of six.
Think About It
Think about this: How would you explain document clustering: grouping similar texts to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind document clustering: grouping similar texts, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.