It is 11 p.m. and a first-year associate at a Mumbai law firm is forty contracts into a due-diligence pile of three hundred vendor agreements. The client is acquiring a logistics company, and the acquirer's counsel needs a single answer before the deal can close: which of these three hundred contracts contain a change-of-control clause, one that lets the counterparty terminate or renegotiate the moment ownership of the logistics company changes hands. Missing even one such clause buried on page 34 of a fifty-page agreement can blow up deal economics after signing. This is not a hypothetical. It is the exact workflow that legal-tech tools built on natural language processing were designed to compress from weeks to hours, and it is the workflow the Indian Supreme Court had in mind when it deployed SUPACE (Supreme Court Portal for Assistance in Court's Efficiency) in 2021, an AI research-assistance tool explicitly scoped to surface relevant facts and precedent for judges, never to decide cases. The same underlying problem, reliably reading a long, dense, adversarially-drafted document and extracting exactly the right span of text, sits underneath e-discovery software, RBI/SEBI compliance screening at Indian banks and NBFCs, and the contract-review products (Kira Systems, Harvey, and a growing set of Indian entrants like Leegality and SpotDraft) that law firms now license instead of assigning to associates. This chapter builds the NLP machinery that makes this possible, from why generic language models struggle with contracts, through the extractive-QA formulation that the field converged on, to the chunking and evaluation mechanics that make it work on real, 50-page documents.
Why contracts break the NLP pipelines you already know
You have spent Grade 11 building token classifiers and sequence-to-sequence models on relatively short, self-contained text: news headlines, product reviews, single-paragraph passages. Contracts violate almost every assumption those pipelines rely on, in four specific ways.
Length. A standard BERT-family encoder from Grade 11's NLP unit caps out at 512 subword tokens, roughly 350-400 English words. A commercial lease or an NDA easily runs to 3,000-8,000 tokens; an M&A agreement can exceed 50,000. You cannot feed the whole document through a vanilla transformer encoder in one pass.
Self-reference. Contracts define their own vocabulary. A clause reading "Recipient shall not disclose Confidential Information except as permitted under Section 6" is uninterpretable without resolving "Confidential Information" back to its definition in Section 1.3 and "Section 6" back to the actual carve-out clause. A model that treats each sentence independently, as a bag-of-sentences classifier would, loses this binding entirely.
Boilerplate versus negotiated deviation. Most of a contract's text is standard-form language reused across thousands of agreements. The legally material content is often a single deviation from that standard, a "shall" changed to "may," a liability cap raised from one month's fees to twelve. A model trained to spot topically distinctive language will systematically miss the deviations that matter most, because they look almost identical to the boilerplate around them.
Domain vocabulary shift. Words like "indemnify," "novation," "force majeure," "in perpetuity," and "notwithstanding" are rare in the web text and Wikipedia that general-purpose language models are pretrained on, yet they carry precise, high-stakes legal meaning in contracts. A model's subword embeddings for these terms are undertrained relative to how much weight a lawyer places on them.
Three ways to formulate the task, and why the field picked the hardest one
Given a contract and a clause type of interest (say, "limitation of liability"), there are three natural ways to frame what the model should output.
| Formulation | Model output | Where it fails |
|---|---|---|
| Document classification | "contains limitation-of-liability clause: yes/no" | Useless to a lawyer, who needs to read the actual clause text, not a binary flag |
| Abstractive summarization | A generated paraphrase of the clause | Paraphrase can silently drop or invert a legal condition; not auditable against source text |
| Extractive question answering | Exact start/end character span copied from the contract | Fails only when no clause of that type exists, or the clause is split across a chunk boundary |
The Contract Understanding Atticus Dataset (CUAD), released by Hendrycks, Burns, Chen, and Ball at NeurIPS 2021, made the deliberate design choice to frame all 41 of its clause categories, across 510 real commercial contracts with more than 13,000 expert-labeled annotations, as extractive question answering rather than 41-way classification. Each clause type becomes a natural-language question ("Highlight the parts related to the termination of this contract"), and the model must return the literal span of contract text that answers it, or abstain if the clause is absent. This mirrors the SQuAD task format from Rajpurkar, Zhang, Lopyrev, and Liang (EMNLP 2016), reused here because it forces the model to ground every answer in text a lawyer can independently verify against the source document. That auditability, not just accuracy, is why extractive QA remains the production standard even now that general-purpose LLMs can accept much longer prompts.
Domain-adapted transformers: teaching the model the vocabulary of contracts
Feeding contract text into a BERT model pretrained on Wikipedia and BooksCorpus wastes the pretraining signal on the words that matter least (common English) and undertrains the words that matter most (legal terms of art). Chalkidis, Fergadiotis, Malakasiotis, Aletras, and Androutsopoulos addressed this directly in "LEGAL-BERT: The Muppets straight out of Law School" (Findings of EMNLP 2020), pretraining BERT-style encoders from scratch on a large corpus of legislation, court judgments, and contracts drawn from EU and US legal sources. The resulting subword vocabulary and embeddings encode "indemnification" and "force majeure" the way general BERT encodes "the" and "contract," as high-frequency, well-calibrated tokens rather than rare, noisy ones. Downstream clause classification and span-extraction F1 improve measurably over generic BERT initialized with the same architecture, purely from this domain-matched pretraining corpus.
The length problem needs a separate fix. Standard self-attention computes a score between every pair of tokens, which costs O(n²) time and memory in sequence length n; doubling document length quadruples the compute. Beltagy, Peters, and Cohan's "Longformer: The Long-Document Transformer" (2020) replaces full attention with a combination of local sliding-window attention (each token attends only to its w nearest neighbors) plus a small number of global tokens that attend to, and are attended to by, the entire sequence. This drops complexity to roughly linear in n, making it practical to run a single forward pass over an 8,000-token contract instead of chopping it into pieces. Longformer and chunked Legal-BERT are complementary, not competing, choices: Longformer trades some accuracy for handling more context at once; chunking keeps the smaller, cheaper, better-studied 512-token encoder and pays the cost in engineering complexity instead of GPU memory.
Worked example: fitting a 3,000-token NDA through a 512-token encoder
Suppose your pipeline uses Legal-BERT with its native 512-token limit, and the NDA you need to process tokenizes to 3,000 tokens. You cannot truncate, since the clause a lawyer needs might be at token 2,700. The standard fix is a sliding window: split the document into overlapping chunks, run the model on each chunk independently, and merge the results. The overlap exists so that a clause spanning a chunk boundary is not silently split in two, since each chunk shares its edge tokens with its neighbor.
def chunk_tokens(total_tokens, window=512, stride=128):
step = window - stride # non-overlapping advance per chunk
chunks = []
start = 0
while True:
end = min(start + window, total_tokens)
chunks.append((start, end))
if end == total_tokens:
break
start += step
return chunks
chunks = chunk_tokens(3000)
print(len(chunks))
for c in chunks:
print(c)Trace it by hand: step = 512 - 128 = 384. Starting at 0, each successive window begins 384 tokens after the last and covers 512 tokens, so consecutive windows share a 128-token overlap. The loop stops the moment a window's end reaches the document length. Running through the arithmetic: starts at 0, 384, 768, 1152, 1536, 1920, 2304, and 2688; the window starting at 2688 covers tokens 2688 to min(2688+512, 3000) = 3000, which equals the total, so the loop terminates there. That is 8 chunks. The printed output is exactly:
8
(0, 512)
(384, 896)
(768, 1280)
(1152, 1664)
(1536, 2048)
(1920, 2432)
(2304, 2816)
(2688, 3000)Notice the last chunk is only 312 tokens wide (3000 - 2688), not the full 512, because the document ended before the window filled. Each of these 8 chunks is run through the extractive-QA model independently with the same clause question, producing up to 8 candidate spans; a simple post-processing step keeps the highest-confidence span and discards duplicates found in the overlap region by more than one chunk.
Worked example: scoring whether the model found the right span
Once the model returns a predicted span, you need a metric that rewards near-misses more informatively than plain exact-match accuracy would. CUAD and SQuAD both use token-level precision, recall, and F1 computed over the bag of tokens in the predicted span versus the bag of tokens in the human-labeled gold span, alongside a stricter exact-match (EM) score.
Take a termination clause. The gold span, labeled by a lawyer, is: "the licensee may terminate this agreement upon thirty days written notice", 11 tokens. Suppose the model's predicted span stops one token late at the start and two tokens early at the end: "licensee may terminate this agreement upon thirty days", 8 tokens, missing "the" at the front and "written notice" at the back.
from collections import Counter
def span_f1(gold_tokens, pred_tokens):
gold_counts = Counter(gold_tokens)
pred_counts = Counter(pred_tokens)
common = gold_counts & pred_counts # per-token min count
num_common = sum(common.values())
if num_common == 0:
return 0.0, 0.0, 0.0
precision = num_common / len(pred_tokens)
recall = num_common / len(gold_tokens)
f1 = 2 * precision * recall / (precision + recall)
return precision, recall, f1
gold = "the licensee may terminate this agreement upon thirty days written notice".split()
pred = "licensee may terminate this agreement upon thirty days".split()
p, r, f1 = span_f1(gold, pred)
em = int(pred == gold)
print(round(p, 3), round(r, 3), round(f1, 3), em)Trace it: gold has 11 tokens, pred has 8. Every one of the 8 predicted tokens appears in gold exactly once, and Counter.__and__ takes the per-token minimum count, so num_common = 8. Precision is 8/8 = 1.0 (nothing predicted was wrong), recall is 8/11 ≈ 0.727 (three gold tokens were missed), and F1 is 2 × 1.0 × 0.727 / (1.0 + 0.727) ≈ 0.842. Since the token lists are not identical, exact match is 0. The printed line is 1.0 0.727 0.842 0. This is the standard shape of a legal-NLP error: the model is never wrong about what it includes (precision stays perfect), but it under-extracts the boundary, which is exactly the failure mode the diagram below visualizes.
The pipeline, and the chunking and span-matching mechanics that make it work
The misconception: "an LLM reading the whole contract is the same as extraction"
A common assumption once students see that modern LLMs accept prompts of 100,000+ tokens is that chunking and extractive QA are now obsolete plumbing, since you can simply paste the whole contract into the context window and ask, in plain English, "does this NDA have a non-compete clause, and what are its terms?" This is wrong on two independent grounds, and both matter for why production legal-tech still builds on the extractive pipeline above rather than replacing it with a single long-context prompt.
First, a fluent generated answer is not grounded. When an LLM is asked to summarize or paraphrase what a clause says, nothing forces the output tokens to be a verbatim copy of the source text; the model can smooth over a conditional ("terminate only with 90 days notice and cause") into an unconditional claim ("may terminate at will"), and a reader has no cheap way to tell the paraphrase apart from the original meaning without going back to the contract itself, which defeats the purpose of automating the read. Extractive QA structurally prevents this failure mode, since the only thing the model is allowed to output is a literal span of the input, so it is either the right text or a verifiably wrong span, never a plausible-sounding invention.
Second, a large context window does not guarantee even attention across that window. Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni, and Liang's "Lost in the Middle: How Language Models Use Long Contexts" (2023) found that model performance on retrieval-style tasks degrades measurably when the relevant fact sits in the middle of a long context, compared to when it sits near the beginning or end, even though the model technically has access to every token. A change-of-control clause buried in the middle of a 50-page acquisition agreement is exactly the geometry this failure mode targets. Retrieval, indexing the chunked contract by embedding similarity and pulling only the most relevant chunks into the model's working context before extraction, exists precisely to route around this weakness rather than trust the model to scan uniformly across tens of thousands of tokens on its own.
From extracted spans to a decision a lawyer can act on
Extraction is the input to a risk-scoring layer, not the end product. A production system runs each of the 40-odd clause questions against every chunk, keeps the highest-confidence span per clause type, and then applies rules or a lightweight classifier over the extracted set: a missing limitation-of-liability clause might be flagged high risk, a non-standard indemnification clause flagged for negotiation, a governing-law clause naming an unfamiliar jurisdiction flagged for local counsel review. Every flag routes back to a human reviewer with the exact source span highlighted, never a bare "risk: high" label, because the entire point of the extractive design is that the lawyer of record remains able to verify the system's claim in seconds rather than trust it blind. This matters doubly in India, where a cloud-hosted contract-review tool processing a client's confidential agreements must also satisfy the Digital Personal Data Protection Act, 2023, on purpose limitation and data handling, an operational constraint layered on top of the NLP accuracy problem, not a substitute for it.
Active recall
Attempt each question before reading its answer.
- Why did CUAD frame all 41 clause categories as extractive question answering instead of 41-way document classification?
- In the chunking worked example (3,000 tokens, window 512, stride/overlap 128), suppose the window is widened to 640 tokens while the overlap stays at 128 tokens. How many chunks result, and does anything besides the chunk count change?
- In the span-F1 worked example, suppose the predicted span shrinks further to just "licensee may terminate this agreement upon" (6 tokens). Recompute precision, recall, F1, and exact match.
- A classmate argues that once GPT-class models accept 100,000-token prompts, chunking and extractive spans are unnecessary for contract review. What two independent facts does this argument ignore?
- A contract has 8,000 tokens. Full self-attention needs on the order of n² = 64,000,000 pairwise score computations per layer. Longformer uses a local window of w = 512 tokens per token plus g = 64 global tokens. Roughly estimate Longformer's score computations per layer (local term ≈ n×w, global term ≈ 2×n×g) and the approximate reduction factor versus full attention.
- Why does pretraining Legal-BERT from scratch on legal text outperform fine-tuning generic BERT on the same downstream clause-classification data?
Answer 1. A binary "clause present: yes/no" label is not the deliverable a lawyer needs; they need to read the actual negotiated language to judge whether it is favorable, standard, or a deal-breaker. Extractive QA forces the model to return the literal source span, which is both more useful (the text itself) and independently auditable (a wrong span is verifiably wrong against the source), unlike a generated summary that could misstate the clause's substance while still reading fluently.
Answer 2. New step = 640 − 128 = 512. Starting points: 0, 512, 1024, 1536, 2048, 2560. The window starting at 2560 covers min(2560+640, 3000) = 3000, so the loop stops there: 6 chunks total, down from 8. But the chunk count is not the only thing that changes: the final (partial) chunk now spans 2560 to 3000, which is 440 tokens wide, versus 312 tokens wide in the original window=512 case. Both the number of chunks and the size of the trailing partial chunk shift when the window size changes, since the window size determines both the step (fewer, wider strides) and how much of the tail is left over when the document length is not an exact multiple of the step.
Answer 3. Predicted tokens: licensee, may, terminate, this, agreement, upon = 6 tokens, all of which appear in the 11-token gold span. num_common = 6. Precision = 6/6 = 1.0. Recall = 6/11 ≈ 0.545. F1 = 2×1.0×0.545/(1.0+0.545) ≈ 1.091/1.545 ≈ 0.706. Exact match remains 0, since the token lists still differ. Precision stays perfect because the model still predicts nothing wrong, but recall and F1 both drop further as the missed tail of the clause ("thirty days written notice") grows.
Answer 4. First, a generated paraphrase is not grounded: nothing in the generation process guarantees the output text is a verbatim, verifiable copy of the source clause, so a fluent-sounding answer can misstate a conditional or a cap without being obviously wrong. Second, a large context window does not guarantee uniform attention across it; the "Lost in the Middle" finding (Liu et al., 2023) shows retrieval accuracy degrading for facts placed mid-context even when the model technically has access to the full document, which is exactly where a buried clause in a long contract tends to sit.
Answer 5. Local term ≈ n×w = 8,000 × 512 = 4,096,000. Global term ≈ 2×n×g = 2 × 8,000 × 64 = 1,024,000. Total ≈ 5,120,000 score computations per layer, versus 64,000,000 for full attention, an approximate reduction factor of 64,000,000 / 5,120,000 ≈ 12.5×. This is an order-of-magnitude estimate (it ignores some double-counting between the local and global terms), but it captures why Longformer's complexity grows roughly linearly in document length while standard attention's grows quadratically, which is the entire point of using it on long contracts.
Answer 6. Fine-tuning generic BERT only adjusts the model's final layers (or lightly adjusts all layers) starting from subword embeddings and attention patterns learned on Wikipedia and books, where terms like "indemnify," "novation," and "force majeure" are rare and their embeddings are correspondingly undertrained. Pretraining from scratch on legislation, case law, and contracts gives the model's core representations, not just its output head, extensive exposure to legal syntax and vocabulary before any downstream task-specific fine-tuning begins, so the fine-tuning stage has a much better-informed starting point and needs far less labeled data to reach the same clause-classification accuracy.
Practice Exercises
Now it is time to practice! Complete these challenges to solidify your understanding:
- Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
- Exercise 2: Find a real-world example where ai for legal applications: nlp and contract analysis is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
- Exercise 3: Create a mind-map connecting ai for legal applications: nlp and contract analysis to at least 3 other topics you have studied.