Diffusion Models: Mathematics of Generative Models

A seller on Meesho photographs a saree on a cheap tripod, uploads it to an AI product-photo tool, and types "studio lighting, white background, festive drape." Ten seconds later the tool returns a crisp, professional-looking image of the same saree in a setting that was never photographed. If you watch the generation happen frame by frame, something strange is visible: the process does not start from a blank canvas. It starts from what looks like television static (a field of pure random pixel noise), and that static slowly, over dozens of steps, resolves into cloth, folds, and light. This is a diffusion model, the same family of architecture behind Stable Diffusion, DALL-E, and Sora. The question this chapter answers is precise and mathematical: how do you train a network whose entire job is to turn noise into structure, one small step at a time, and why does that turn out to be easier than generating the image in one shot?

The physical idea: run corruption backward

The name "diffusion" is borrowed deliberately from physics. Drop ink into a glass of water and it disperses: a structured, low-entropy state (ink concentrated at one point) evolves into a diffuse, high-entropy state (ink spread uniformly). That forward process is easy to describe mathematically: at every instant, each ink particle takes a small random step. Sohl-Dickstein et al. (2015) asked a sharper question: if the forward process is a simple, well-understood random walk, can a neural network learn to run that random walk in reverse, taking a diffuse cloud and reconstructing where the ink started?

Applied to images, the "ink drop" is a real photograph and the "water" is Gaussian noise. The forward process destroys a photograph by repeatedly adding small amounts of Gaussian noise until nothing recognizable remains. This destruction process needs no learning at all: it is fixed, mechanical, and mathematically simple, a Markov chain of Gaussian steps. The generative model is the network trained to reverse it: given a noisy image at step t, predict what step t−1 looked like. Chain that reversal from pure noise back to step 0, and you have generated a new photograph that never existed.

The forward process, precisely

Let x_0 be a real data point (an image, or for our worked example, a single scalar pixel value). Fix a noise schedule β_1, β_2, …, β_T with each β_t ∈ (0,1), typically small and increasing with t. The forward process is the Markov chain:

q(x_t | x_{t-1}) = N( x_t ; sqrt(1 - β_t) · x_{t-1} ,  β_t · I )

Read this as: to get x_t, shrink x_{t-1} slightly (multiply by sqrt(1-β_t), a number just under 1) and add fresh Gaussian noise with variance β_t. The shrink factor is not cosmetic: it is what keeps the process variance-preserving. If you only added noise without shrinking the signal, variance would explode step after step. With the shrink factor, if x_{t-1} has unit variance, x_t also has unit variance: Var(x_t) = (1-β_t)·1 + β_t = 1. Every step of the chain lives on the same scale.

Define α_t = 1 - β_t and the cumulative product ᾱ_t = α_1·α_2·⋯·α_t. Using the reparameterization trick (writing a Gaussian sample as mean plus standard deviation × ε, with ε ~ N(0,I)), each step becomes:

x_t = sqrt(α_t) · x_{t-1} + sqrt(β_t) · z_t,   z_t ~ N(0,1) i.i.d.

Unrolling this recursion (substitute x_{t-1}'s own definition in terms of x_{t-2}, and so on, using the fact that a sum of independent Gaussians is itself Gaussian with variances adding) collapses the entire t-step chain into a single equation: the single most useful identity in the whole subject, because it means you never need to simulate t sequential noising steps during training; you can jump straight to any t:

q(x_t | x_0) = N( x_t ; sqrt(ᾱ_t) · x_0 ,  (1 - ᾱ_t) · I )
⇒  x_t = sqrt(ᾱ_t) · x_0 + sqrt(1 - ᾱ_t) · ε,   ε ~ N(0,1)

As t grows, ᾱ_t shrinks toward 0 (it is a product of numbers below 1), so the coefficient on the original signal x_0 vanishes and the coefficient on pure noise approaches 1. At t = T with a well-designed schedule, x_T is indistinguishable from a draw from N(0, I), which is exactly what makes sampling possible: you can start generation by literally drawing noise from a standard normal, with no knowledge of any real image.

Worked example: hand-tracing four noising steps

Real diffusion models use T ≈ 1000 steps with tiny betas (around 0.0001 to 0.02). To trace every arithmetic step by hand, use an exaggerated toy schedule with T = 4: β_1=0.1, β_2=0.2, β_3=0.3, β_4=0.4, starting from a single scalar data point x_0 = 2.0 (think of it as one normalized pixel intensity), with fixed noise draws z_1=0.50, z_2=−0.30, z_3=0.80, z_4=−0.10 so the trajectory is reproducible.

β  = [0.1, 0.2, 0.3, 0.4]
α  = [0.9, 0.8, 0.7, 0.6]                      # α_t = 1 - β_t
ᾱ  = [0.9, 0.72, 0.504, 0.3024]                # cumulative product

x0 = 2.0
z  = [0.50, -0.30, 0.80, -0.10]

x = [x0]
for t in range(4):
    xt = (α[t] ** 0.5) * x[-1] + (β[t] ** 0.5) * z[t]
    x.append(xt)

# x = [2.000000, 2.055480, 1.704314, 1.864109, 1.380687]

Tracing the loop by hand: x_1 = sqrt(0.9)·2.0 + sqrt(0.1)·0.50 = 0.948683·2.0 + 0.316228·0.50 = 1.897367 + 0.158114 = 2.055480. Then x_2 = sqrt(0.8)·2.055480 + sqrt(0.2)·(−0.30) = 0.894427·2.055480 − 0.447214·0.30 = 1.838477 − 0.134164 = 1.704314, and continuing the same pattern gives x_3 = 1.864109 and x_4 = 1.380687. Notice the value does not monotonically drift away from 2.0: noise is random, not a consistent push in one direction, but its variance around the shrinking signal grows steadily. By t=4, ᾱ_4 = 0.3024, meaning only 30.24% of the variance in x_4 traces back to the original signal and 69.76% is injected noise.

The closed-form shortcut can be checked against this same trajectory. Define the "equivalent noise" implied by each x_t: ε̄_t = (x_t − sqrt(ᾱ_t)·x_0) / sqrt(1−ᾱ_t). For t=4: ε̄_4 = (1.380687 − sqrt(0.3024)·2.0) / sqrt(0.6976) = (1.380687 − 1.099817) / 0.835225 = 0.280870 / 0.835225 = 0.336280. Plugging ε̄_4 = 0.336280 back into x_4 = sqrt(ᾱ_4)·x_0 + sqrt(1−ᾱ_4)·ε̄_4 reproduces 1.380687 exactly, confirming the four sequential steps and the single closed-form jump describe the same distribution.

The reverse process and what the network actually learns

Generation requires inverting the chain: given x_t, sample x_{t-1}. The true reverse conditional q(x_{t-1} | x_t) has no closed form on its own, but q(x_{t-1} | x_t, x_0) (the reverse step given the original clean image too) is Gaussian, with a mean that Bayes' rule gives exactly:

μ̃_t(x_t, x_0) = ( sqrt(ᾱ_{t-1})·β_t / (1-ᾱ_t) ) · x_0
              + ( sqrt(α_t)·(1-ᾱ_{t-1}) / (1-ᾱ_t) ) · x_t

The catch: at generation time you do not have x_0: the whole point is to produce it. So the network ε_θ(x_t, t) is trained to predict the noise ε that was mixed into x_t (exactly the ε̄_t from the closed-form equation above), which lets you estimate x_0 via x̂_0 = (x_t − sqrt(1−ᾱ_t)·ε_θ) / sqrt(ᾱ_t) and substitute it into μ̃_t. Ho et al. (2020, DDPM) show this substitution simplifies to a clean, x_0-free formula for the mean the reverse network should target:

μ_θ(x_t, t) = (1/sqrt(α_t)) · ( x_t − (β_t / sqrt(1-ᾱ_t)) · ε_θ(x_t, t) )

with the reverse step sampled as x_{t-1} = μ_θ(x_t,t) + σ_t·z, fresh z ~ N(0,1), and posterior variance σ_t² = β_t·(1-ᾱ_{t-1})/(1-ᾱ_t). Because μ_θ and μ̃_t are algebraically the same quantity, training reduces to a supervised noise-prediction problem: sample a random t, sample ε, build x_t via the closed form, and minimize how far ε_θ(x_t,t) is from the true ε. This is the DDPM simplified loss:

L_simple(θ) = E_{t, x_0, ε} [ || ε − ε_θ(x_t, t) ||² ]

No adversarial discriminator, no likelihood computed over the full data density: just a mean-squared-error regression on noise, at a randomly chosen corruption level, repeated across a billion (image, noise, timestep) triples.

Worked example continued: one reverse step, and why μ_θ is not x_{t-1}

Take the toy trajectory's last point, x_4 = 1.380687, and suppose the network has been trained perfectly, so ε_θ(x_4, 4) = ε̄_4 = 0.336280 exactly. Compute the reverse mean:

α_4 = 0.6, β_4 = 0.4, ᾱ_4 = 0.3024, ᾱ_3 = 0.504

μ_θ(x_4,4) = (1/sqrt(0.6)) · ( 1.380687 − (0.4/sqrt(0.6976))·0.336280 )
           = 1.290994 · ( 1.380687 − 0.478907·0.336280 )
           = 1.290994 · ( 1.380687 − 0.161042 )
           = 1.290994 · 1.219645
           = 1.574546

Cross-checking with the alternative x_0-based formula μ̃_4, using the true x_0 = 2.0 directly, gives 1.574546 as well: the two routes agree exactly, confirming the algebra. Also compute the posterior spread: σ_4² = 0.4·(1−0.504)/(1−0.3024) = 0.4·0.496/0.6976 = 0.284404, so σ_4 = 0.533295.

Now compare μ_θ(x_4,4) = 1.574546 to the actual value from the forward trajectory, x_3 = 1.864109. They differ by 0.290, nearly half a standard deviation (σ_4 = 0.533). This is not an error in the network or the arithmetic. It is the central fact about what "reversing" diffusion means.

The misconception this exposes

A natural assumption, especially once you've seen the forward chain written as "add noise z_t," is that the reverse network's job is to subtract that exact same z_t back out: that denoising is literal, deterministic inversion, like undoing a specific arithmetic operation. The numbers above show this is false. Given only x_4 (and an estimate of x_0), there are infinitely many trajectories x_0 → x_1 → x_2 → x_3 → x_4 consistent with landing on that exact x_4, each having used different random draws along the way. The reverse network cannot know which specific z_3 was actually drawn in the one trajectory that happened to occur: that information was destroyed the moment the noise was added, the same way you cannot recover which two numbers were added just by looking at their sum. What μ_θ computes is the mean of the posterior distribution over all such trajectories: the best average guess of x_3 given x_4, not a replay of history. That is precisely why the sampling step adds fresh noise, x_{t-1} = μ_θ + σ_t·z, rather than returning μ_θ alone: the reverse process is a genuinely stochastic Markov chain, sampling one plausible ancestor out of many, which is also exactly why running the same trained model twice from the same x_T produces two different, both valid, images. Denoising diffusion is closer to "hallucinate a statistically consistent history" than "invert a known operation."

A brief note on score matching

There is a second, equivalent lens worth knowing at this level because it connects diffusion models to a distinct research lineage (Song and Ermon, 2019). The score function of a distribution is ∇_x log p(x), the direction in data space that most increases likelihood. It can be shown that the noise-prediction target ε_θ(x_t,t) is proportional to −sqrt(1-ᾱ_t) · ∇_{x_t} log q(x_t): predicting the noise and estimating the score of the noised distribution are the same regression problem, up to a scale factor. This is why you will see diffusion models described interchangeably as "denoising models" and "score-based generative models" in papers: they are the same mathematics derived from two different starting stories, one from noise-prediction (Ho et al.), one from Langevin-dynamics sampling along an estimated score field (Song and Ermon). Both converge on the identical training loss.

Forward and reverse chains, side by side

Training and sampling as algorithms

Putting the objective into pseudocode makes the "regression, not generation" character of training explicit. unet_predict_noise below stands for the actual noise-prediction network (a U-Net with timestep conditioning); its internals are outside this chapter's scope, so it is used here only as a named, assumed helper, not something meant to run standalone:

# Training: repeat until convergence
def train_step(x0_batch, alpha_bar, T, unet_predict_noise, optimizer):
    batch_size = x0_batch.shape[0]
    t = random_integers(low=1, high=T, size=batch_size)      # random timestep per sample
    eps = sample_standard_normal(shape=x0_batch.shape)         # true noise
    ab_t = alpha_bar[t]                                        # gather ᾱ_t per sample
    x_t = sqrt(ab_t) * x0_batch + sqrt(1 - ab_t) * eps          # closed-form forward jump
    eps_pred = unet_predict_noise(x_t, t)                       # (assumed helper, not shown)
    loss = mean((eps - eps_pred) ** 2)                          # L_simple
    loss.backward()
    optimizer.step()
    return loss

# Sampling: draw a new image from pure noise
def sample(alpha, alpha_bar, beta, T, unet_predict_noise, shape):
    x_t = sample_standard_normal(shape)                         # x_T ~ N(0, I)
    for t in range(T, 0, -1):
        eps_pred = unet_predict_noise(x_t, t)                   # (assumed helper, not shown)
        mu = (x_t - (beta[t] / sqrt(1 - alpha_bar[t])) * eps_pred) / sqrt(alpha[t])
        if t > 1:
            sigma = sqrt(beta[t] * (1 - alpha_bar[t-1]) / (1 - alpha_bar[t]))
            x_t = mu + sigma * sample_standard_normal(shape)
        else:
            x_t = mu                                            # no noise on the final step
    return x_t  # this is x_0, the generated sample

Two design choices in this loop directly reflect the math derived above. First, t is sampled uniformly at random per training example rather than looping through all T steps sequentially: the closed-form jump is exactly what makes this possible, since x_t can be constructed directly from x_0 without simulating the intermediate steps. Second, sampling omits the noise term on the very last step (t=1 → t=0): the posterior variance formula σ_t² = β_t(1-ᾱ_{t-1})/(1-ᾱ_t) is only meaningful for t ≥ 2, since it references ᾱ_{t-1}, and the final output should be the clean estimate, not one more noisy draw.

Active recall

Attempt these before reading the answers.

1. Why does the forward step multiply x_{t-1} by sqrt(1-β_t) instead of leaving it unscaled and just adding noise?
2. Using the toy schedule β=[0.1,0.2,0.3,0.4], what is ᾱ_2, and what fraction of the variance in x_2 comes from the original signal versus injected noise?
3. What quantity does the network ε_θ(x_t, t) actually predict, and how is that turned into an estimate of x_0?
4. Why does the sampling algorithm add fresh random noise σ_t·z at each reverse step instead of just outputting μ_θ directly?
5. In the worked reverse-step example, μ_θ(x_4,4) = 1.574546 while the actual x_3 from the forward pass was 1.864109. Is this a sign the network failed? Justify your answer using the posterior variance σ_4.
6. What single mathematical fact makes it possible to train on a randomly chosen timestep t instead of simulating the full T-step chain for every example?

Answers.

1. Without the shrink factor, variance would grow without bound across steps: Var(x_t) = Var(x_{t-1}) + β_t keeps increasing. With the factor, Var(x_t) = (1-β_t)·Var(x_{t-1}) + β_t, which stays fixed at 1 if Var(x_{t-1})=1. This is what makes the process converge to a clean, fixed N(0,I) at t=T rather than diverging.

2. ᾱ_2 = α_1·α_2 = 0.9 × 0.8 = 0.72. So 72% of the variance in x_2 traces to the signal (coefficient sqrt(0.72)=0.8485 on x_0) and 28% is injected noise (coefficient sqrt(0.28)=0.5292 on ε).

3. It predicts ε, the standard-normal noise mixed into x_t via x_t = sqrt(ᾱ_t)x_0 + sqrt(1-ᾱ_t)ε. Rearranging that same equation for x_0 gives the estimate x̂_0 = (x_t - sqrt(1-ᾱ_t)·ε_θ) / sqrt(ᾱ_t).

4. Because μ_θ is only the mean of the posterior distribution over possible x_{t-1} values consistent with x_t: many different noise histories could have produced the same x_t. Sampling from the full distribution (mean plus scaled fresh noise) rather than always returning the mean is what lets the same starting noise x_T, or nearby ones, generate diverse outputs, and it also matches the true generative process the model was trained to approximate.

5. Not a failure. The gap, 0.290, is well within one posterior standard deviation σ_4 = 0.533. μ_θ is not supposed to equal the one particular x_3 that happened to occur; it is supposed to be the mean of the distribution x_3 was drawn from. A perfectly trained network reproduces that posterior mean exactly (verified algebraically in the example, where the ε-based formula and the x₀-based formula agreed to full precision), not any single sampled point.

6. The closed-form identity q(x_t|x_0) = N(x_t; sqrt(ᾱ_t)x_0, (1-ᾱ_t)I), obtained by collapsing the Markov chain algebraically. Because it expresses x_t directly in terms of x_0 and a single Gaussian draw, any timestep can be constructed in one line without stepping through t-1 intermediate noisy versions.

Think About It

Think about this: How would you explain diffusion models: mathematics of generative models to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where diffusion models: mathematics of generative models is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting diffusion models: mathematics of generative models to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind diffusion models: mathematics of generative models, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.