Picture a Class 12 student building a small billing module for a mock e-commerce seller dashboard — the kind of project many students prototype in the style of a Flipkart or Meesho seller panel. She writes a comment specifying exactly how GST should be charged on a cart total, then asks an AI code assistant to write the function underneath it:
# Slabs for this seller dashboard:
# cart_total <= 1000 -> 0% GST (exempt category)
# 1000 < cart_total <= 5000 -> 12%
# 5000 < cart_total <= 20000 -> 18%
# cart_total > 20000 -> 28%
def compute_gst(cart_total):
...The assistant fills in a function that looks completely professional:
def compute_gst(cart_total):
if cart_total < 1000:
rate = 0.0
elif cart_total < 5000:
rate = 0.12
elif cart_total < 20000:
rate = 0.18
else:
rate = 0.28
return cart_total * rateIt compiles. It runs. It even passes a quick eyeball check for a cart total of ₹2,500 or ₹15,000. But trace compute_gst(1000) line by line: the first condition asks whether 1000 < 1000, which is False, so control falls to elif cart_total < 5000, which is True, setting rate = 0.12. The function returns 1000 * 0.12 = 120.0. But the comment's spec says a cart total of exactly ₹1,000 is exempt — it should return 0.0. Every boundary comparison the model wrote uses strict <, while the student's own specification used <= at every tier. The assistant did not misread the comment out of carelessness in any human sense; it produced the single most statistically probable continuation given everything it has seen in billions of lines of training code, and "amount < threshold" is an overwhelmingly more common pattern across public repositories than the precise inequality this particular spec demanded. This chapter is about why that happens, how these systems actually generate code token by token, how to measure whether they got it right, and what changes when you let them run their own code and check the answer.
What a Code-Generation Model Is Actually Doing
An AI programming assistant like GitHub Copilot, Amazon Q Developer, Cursor, or Claude Code is, at its core, the same transformer decoder architecture your Grade 11 and Grade 12 mathematics units built up from scratch — stacks of masked self-attention and feed-forward layers, trained with backpropagation to minimize cross-entropy loss on a next-token prediction task. What differs from a general chat model is almost entirely the training corpus and the input format. Instead of (or in addition to) natural-language text, the model is trained on hundreds of billions of tokens of source code scraped from public repositories. OpenAI's Codex (Chen et al., "Evaluating Large Language Models Trained on Code," arXiv:2107.03374, 2021) was the first widely documented large-scale demonstration of this: fine-tuning a GPT-family model on GitHub Python code produced a system that solved 28.8% of a new benchmark of hand-written programming problems on its first attempt, versus 0% for the base GPT-3 model evaluated the same way. The BigCode project's "The Stack" dataset (Kocetkov et al., 2022) — roughly 3 terabytes of permissively licensed source code — later became the standard open corpus behind models like StarCoder and Meta's Code Llama (Rozière et al., 2023).
Before any of that training happens, source code has to become a sequence of integers. A code-aware tokenizer, typically a byte-pair-encoding (BPE) vocabulary learned from the training corpus, breaks text into subword pieces based on which character sequences co-occur most frequently. A long identifier like cart_total is rarely its own vocabulary entry; it is far more likely represented as a small handful of merged pieces — something in the spirit of cart, _, total — because the vocabulary was built to compress the statistics of the whole corpus, not to respect any single variable name. This matters pedagogically for one reason: whitespace and indentation, which are semantically load-bearing in Python, must be tokenized carefully (leading spaces, tabs, and newlines are typically kept as explicit tokens) or the model loses the information that separates one code block from another. A code tokenizer that collapsed indentation the way a natural-language tokenizer collapses extra spaces would make Python's block structure invisible to the model.
Decoding: Turning a Probability Distribution into a Program
Training produces a model that, given everything generated so far, outputs a probability distribution over every token in the vocabulary for what comes next. Generating actual code means repeatedly sampling from that distribution, appending the chosen token to the context, and repeating — this is autoregressive decoding, and it is the literal mechanism diagrammed below. The strategy used to pick a token from the distribution matters enormously in practice:
Greedy decoding always takes the single highest-probability token (the argmax). It is deterministic and fast but can lock the model into a locally plausible but globally wrong path, exactly as in the compute_gst example. Beam search keeps several candidate sequences alive in parallel and prunes low-probability branches, useful in machine translation but rarely used for code because it tends to produce bland, repetitive completions. Sampling with temperature divides the logits by a temperature T before the softmax — T < 1 sharpens the distribution toward the most likely tokens (more deterministic, safer for code), T > 1 flattens it (more diverse, riskier). Top-p (nucleus) sampling restricts sampling to the smallest set of tokens whose cumulative probability exceeds a threshold p, discarding the long unlikely tail entirely. A newer, code-specific technique is grammar-constrained decoding: at every step, tokens that would make the partial program syntactically invalid (an unmatched bracket, a keyword in the wrong position) are masked out of the distribution before sampling, guaranteeing the output parses. This is a genuinely useful production technique — but it only guarantees syntax, not semantics. A grammar-constrained decoder would happily still emit compute_gst's exact off-by-one bug, because "cart_total < 5000" is perfectly valid Python; constrained decoding closes the syntax gap, never the logic gap.
The diagram below traces one full decoding step over a toy example that continues the GST scenario, using the probabilities a next-token softmax might plausibly assign after seeing if cart_total < 1000::
Fill-in-the-Middle and Repository-Level Context
Real programming assistants rarely generate a file from an empty prompt — a developer is usually editing inside an existing function, with code both before and after the cursor. A purely causal, left-to-right model has no native way to "see" the suffix, since it was only ever trained to predict left to right. The fix, introduced as Fill-in-the-Middle training (Bavarian et al., "Efficient Training of Language Models to Fill in the Middle," arXiv:2207.14255, 2022), is elegant: during training, documents are randomly cut into a prefix, a middle, and a suffix, then reassembled in a new order — prefix, then suffix, then middle — with special separator tokens marking each boundary, before being fed to the model as an ordinary left-to-right sequence:
<PRE> def compute_gst(cart_total):
if cart_total < 1000: <SUF>
else:
rate = 0.28
return cart_total * rate <MID>The causal objective is unchanged — the model still just predicts the next token given everything to its left — but because the suffix now appears earlier in the rearranged sequence, the model has effectively "seen" both sides of the gap by the time it starts generating the middle. At inference time the editor performs the same PSM (prefix-suffix-middle) rearrangement automatically, and the model's output is spliced back into its original position once it emits a designated end-of-middle token.
The other major context problem is scale: a real codebase spans thousands of files, and the function a developer is editing may depend on a class defined three directories away. Modern assistants address this with retrieval-augmented generation for code — embedding the repository's files (or chunks of them) into a vector space, retrieving the chunks most relevant to the current cursor position or open files, and prepending them to the prompt before generation — combined, increasingly, with simply widening the context window itself into the hundreds of thousands of tokens so more of the repository can be included directly.
Worked Example: Measuring Correctness with pass@k
If a model can generate many candidate solutions to a problem, how do you turn that into a single reliability number? The Codex paper's answer, now the standard metric for code-generation benchmarks like HumanEval, is pass@k: the probability that at least one of k randomly chosen candidates (drawn from a larger pool the model generated) passes all the unit tests for the problem. You might think to estimate this by literally drawing k samples and checking — but that estimator has high variance for small k, especially near 0 or 1. Chen et al. (2021) instead generate a larger pool of n samples per problem (on the order of 100–200), count how many of them, c, actually pass, and compute the unbiased combinatorial estimator:
pass@k = 1 - C(n - c, k) / C(n, k)where C(a, b) is "a choose b." The intuition: C(n-c, k) counts the ways to pick k samples that are all wrong (drawn only from the n-c failing ones); dividing by C(n, k), the total ways to pick any k of the n samples, gives the probability that a random draw of k is entirely wrong; one minus that is the probability at least one is right.
Worked example. Suppose a model generates n = 5 candidate solutions to a problem, and c = 2 of them pass all unit tests.
pass@1: C(n-c, 1) / C(n, 1) = C(3,1) / C(5,1) = 3/5. So pass@1 = 1 − 3/5 = 2/5 = 0.4. Sanity check: for k=1 this estimator must always equal the plain success rate c/n, since picking exactly one sample at random succeeds with probability c/n directly — 2/5 = 0.4 confirms the formula reduces correctly at k=1.
pass@3: C(n-c, 3) / C(n, 3) = C(3,3) / C(5,3) = 1/10 (there is exactly one way to choose all 3 wrong samples out of the 3 that exist, against 10 total ways to choose any 3 of the 5). So pass@3 = 1 − 1/10 = 0.9. This also matches direct reasoning: with only 3 failing samples in the pool, drawing 3 of the 5 samples fails only in the single case where you happen to pick exactly those 3 failing ones.
Reported on this metric, Codex's 12-billion-parameter model solved 28.8% of HumanEval problems at pass@1 — meaning a single greedy sample was correct on fewer than 3 in 10 problems, even though the same model's pool of many samples contained a correct solution far more often. That gap between "a single generation is right" and "some generation in a large pool is right" is exactly why production coding assistants increasingly wrap generation in a loop that tests and repairs its own output rather than trusting the first draft — the subject of the next section.
The Misconception: "It Compiled and Ran, So It Must Be Right"
The single most common and most dangerous misconception students form about AI programming assistants is that a syntactically clean, well-formatted, successfully-executing suggestion has been checked for correctness the way a careful human reviewer would check it. It has not. A code-generation model, absent any tool-use loop, never executes the code it writes, never runs it against the specification, and has no internal notion of "correct" beyond "statistically consistent with what code tends to look like in similar contexts." The compute_gst example is not a contrived edge case — off-by-one boundary errors are precisely where this failure mode concentrates, because boundary conditions are the place where the literal spec and the statistically dominant pattern from training data are most likely to diverge, and nothing in ordinary next-token generation forces the model to re-derive the boundary from the spec rather than reproduce the far more common pattern it has seen thousands of times before. Fluent, confident-looking output and correct output are orthogonal properties; the first is what a language model is trained to produce, the second requires independent verification — reading the code against the spec, writing and running unit tests, or handing the loop to an agentic system that does exactly that automatically.
Agentic Programming Assistants: Closing the Verification Gap
The natural fix to "the model never checks its own work" is to give it the ability to check its own work. An agentic coding assistant — the pattern underlying tools like Claude Code, Cursor's agent mode, and Cognition Labs' Devin — wraps the same underlying language model in a loop with real tool access: it can read files, write files, run the test suite, read compiler and runtime errors, and search the repository, then feed every observation back into its own context before deciding what to do next. This plan-act-observe-revise pattern traces back to the ReAct framework (Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models," 2022), which interleaves the model's reasoning steps with concrete tool calls and their results rather than asking for one unverified answer in a single shot.
Crucially, this loop directly attacks the misconception above: an agentic assistant that writes compute_gst, then runs the test assert compute_gst(1000) == 0.0 and sees it fail, receives that failure as new context and can revise the comparison operators — a correction a pure autocomplete model has no mechanism to make, because it never sees a "failed" signal at all. This is precisely what the SWE-bench benchmark (Jimenez et al., "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?," ICLR 2024) measures: given a real, historical GitHub issue and the corresponding repository, can an agent produce a patch that makes the project's actual hidden test suite pass? Performance on SWE-bench climbed substantially over 2024 and 2025 as agentic scaffolding — better tool use, longer verify-and-repair loops, more careful retrieval of relevant files — improved, even holding the underlying model family roughly fixed. The lesson generalizes: for code, unlike open-ended prose, there frequently exists an objective, checkable ground truth (does the test pass, does the type-checker accept it, does the program terminate correctly on the given inputs), and systems that exploit that checkability by actually running the check outperform systems that only pattern-match toward what correct code tends to look like.
What the Evidence Says About Using These Tools
None of this is purely theoretical. A controlled experiment at GitHub (Peng et al., "The Impact of AI on Developer Productivity: Evidence from GitHub Copilot," arXiv:2302.06590, 2023) randomly assigned developers to complete an HTTP server implementation task with or without Copilot access and found the assisted group finished roughly 56% faster on average — a genuinely large, measured productivity effect, not a marketing claim. At the same time, security researchers (Pearce et al., "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions," IEEE Symposium on Security and Privacy, 2022) systematically prompted Copilot with scenarios drawn from known vulnerability classes (the MITRE "Top 25" CWE list) and found that a substantial fraction of the resulting completions — historically cited around 40% across their vulnerability-relevant scenarios — contained an exploitable weakness, even though the code compiled and superficially "worked." A separate, more recent failure mode is package hallucination, sometimes called slopsquatting: researchers probing multiple code-generating models (Spracklen et al., 2024) found that a nontrivial share of suggested import or pip install statements referenced packages that do not actually exist on PyPI or npm — plausible-sounding names the model generated by statistical analogy to real package-naming conventions. Because these hallucinated names are predictable and repeatable, attackers can pre-register exactly those names as real, malicious packages, so that a developer who trusts the assistant's import line without checking installs malware. Speed, security, and hallucination are not competing hypotheses about whether these tools are "good" — they are three independent, separately measured properties of the same underlying mechanism, and a systems engineer working with these assistants has to hold all three at once: faster median output, a meaningfully elevated tail risk of subtle bugs and vulnerabilities, and an entirely new supply-chain attack surface that did not exist before generative import statements did.
Active Recall
Attempt each question before reading its answer.
Q1. A model generates n = 8 candidate solutions to a problem; c = 5 pass all unit tests. Compute pass@1 and pass@2 using the unbiased estimator.
Q2. Why does the Codex evaluation methodology compute pass@k from a combinatorial formula over a larger pool of n samples, rather than simply generating k samples directly and checking how often at least one passes?
Q3. Suppose the seller-dashboard team changes the GST specification so that every slab boundary becomes lower-inclusive to match what the assistant originally wrote — that is: cart_total < 1000 is exempt; 1000 ≤ cart_total < 5000 is 12%; 5000 ≤ cart_total < 20000 is 18%; cart_total ≥ 20000 is 28%. Using the exact same, unmodified compute_gst function from earlier, trace all three boundary values — 1000, 5000, and 20000 — against this new spec. Does the code now match?
Q4. In Fill-in-the-Middle training, why is the document rearranged into prefix-suffix-middle (PSM) order rather than simply feeding the model the prefix and suffix in their natural left-to-right order with a gap?
Q5. Given the toy next-token distribution from the diagram — slab 0.52, rate 0.21, amount 0.14, gst_rate 0.08, other 0.05 — which token does greedy decoding select? Under top-p (nucleus) sampling with p = 0.9, which tokens fall inside the nucleus?
Q6. Why can an agentic coding assistant that executes the test suite resolve a real GitHub issue on SWE-bench that a pure autocomplete model, using the identical underlying language model, cannot?
Answers
A1. pass@1 = c/n = 5/8 = 0.625. For pass@2: C(n-c, 2)/C(n, 2) = C(3,2)/C(8,2) = 3/28. pass@2 = 1 − 3/28 = 25/28 ≈ 0.893.
A2. Directly sampling only k completions and checking has high variance, especially for small k or when the true success probability is near 0 or 1 — a single unlucky draw can swing the estimate to 0% or 100% even when the model's real underlying success rate is moderate. Generating a much larger pool of n samples first and applying the combinatorial formula uses all the information in that larger pool to compute an exact, unbiased estimate of what k-sample performance would be, without the noise of actually drawing k samples over and over.
A3. Trace each boundary against the unchanged code (if cart_total < 1000: rate=0.0; elif cart_total < 5000: rate=0.12; elif cart_total < 20000: rate=0.18; else: rate=0.28). At 1000: first condition (1000<1000) is False, second (1000<5000) is True, so rate=0.12 — the new spec says 1000≤x<5000 gives 12%, so this matches. At 5000: first two conditions are False, third (5000<20000) is True, so rate=0.18 — the new spec says 5000≤x<20000 gives 18%, so this matches. At 20000: first three conditions are all False (20000<20000 is False), so it falls to else, rate=0.28 — the new spec says x≥20000 gives 28%, so this matches. All three boundaries now agree: the exact same code is fully correct under this revised spec. This confirms the code was never intrinsically "buggy" in isolation — it was a mismatch between one particular specification and the statistically dominant pattern the model reproduced; correctness is only ever defined relative to a spec, which is why verification against the actual requirement, not just re-reading the code, is the only reliable check.
A4. A causal transformer decoder is only ever trained to predict the next token from everything to its left — it has no architectural mechanism for conditioning on text that comes later in the natural reading order. By physically rearranging the document during training so the suffix appears (with a marker) before the middle, the "future" content the model needs is repositioned into its left context, and the ordinary causal, left-to-right training objective is left completely unchanged. The trick is entirely in the data rearrangement, not the architecture.
A5. Greedy decoding always picks the single highest-probability token, which is "slab" (0.52). For nucleus sampling with p = 0.9, sort tokens by descending probability and accumulate: slab (0.52, cumulative 0.52), rate (0.21, cumulative 0.73), amount (0.14, cumulative 0.87), gst_rate (0.08, cumulative 0.95). The cumulative probability crosses 0.9 as soon as gst_rate is included, so the nucleus is {slab, rate, amount, gst_rate}; "other" (0.05) is excluded and can never be sampled at this p.
A6. A pure autocomplete model produces one (or a few) forward passes and stops — it has no way to learn that its patch is wrong unless the wrongness happens to be visible purely from the surrounding text pattern. An agentic assistant with tool access runs the project's actual test suite after generating a candidate patch, and a failing test result is fed back into the model's context as new evidence, allowing it to generate a revised patch that specifically addresses the observed failure. The identical underlying model produces very different real-world reliability depending on whether it gets this execute-and-observe feedback loop, because SWE-bench success requires closing the exact verification gap this chapter has been building toward: fluent code generation plus an external, objective correctness check, not fluent code generation alone.
Think About It
Think about this: How would you explain code generation and ai programming assistants to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind code generation and ai programming assistants, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.