---
title: "DDIM from Scratch, Part 2: Removing the Markov Assumption"
date: 2026-06-29
tags: ["Diffusion Models", "Generative Models", "Deep Learning"]
author: "Yiyang Yu"
---

> Part 2 of the DDIM series. In [Part 1](ddim-01-principles-and-ddpm-review.md) we diagnosed the
> problem: DDPM's reverse process is shackled to a Markov chain, forcing 1000 one-step-at-a-time
> denoising steps. Now we **break the shackle** — and derive DDIM's reverse distribution and its
> famous sampling formula by the method of **undetermined coefficients**, all while keeping the
> ability to **free-load (白嫖)** a pretrained DDPM model.

## Where exactly was Markov used?

Recall the DDPM reverse derivation. Bayes gave us (Formula 4):

$$q(x_{t-1}\mid x_t, x_0) = \frac{q(x_t \mid x_{t-1}, x_0)\,q(x_{t-1}\mid x_0)}{q(x_t \mid x_0)}.$$

The first numerator term $q(x_t \mid x_{t-1}, x_0)$ is unknown — until we invoke the **Markov
property**: any $x_t$ depends only on its immediate predecessor, so the distant $x_0$ drops out,
$q(x_t\mid x_{t-1}, x_0) = q(x_t\mid x_{t-1})$. That single move turned the unknown into the known
and let us simplify Formula 4 into the solvable Formula 5.

**DDIM refuses to use the Markov property.** So $q(x_t\mid x_{t-1}, x_0)$ stays unknown, and **Formula
4 can no longer be simplified to Formula 5**. We've lost our derivation path. We need a different
strategy.

---

## Two requirements for free-loading

DDIM wants the acceleration "for free" — reuse a DDPM-trained model with minimal cost. Beyond
dropping Markov, that imposes **two requirements**:

### Requirement 1 — don't change $q(x_t \mid x_0)$

Look back at the DDPM loss (Formula 8): the only distribution it actually uses is the **marginal**
$q(x_t\mid x_0)$ (Formula 2) — *not* the joint $q(x_{1:T}\mid x_0)$. To free-load DDPM's trained
model, the loss must obey a **"gradient-equivalence everywhere"** principle:

> **The one distribution DDPM's loss depends on — $q(x_t\mid x_0)$ — must not change at all because
> of DDIM's step-skipping.**

"Equivalence" here is in the *broad* sense: a leading **constant** coefficient is fine. If DDIM's
loss $L_{\sigma}$ differs from DDPM's $L_\gamma$ only by a constant multiplier $a$ (i.e. $L_\sigma = a\,L_\gamma + b$), the gradient direction is unchanged — you just rescale the learning rate by
$1/a$ and get identical optimization. A constant out front is harmless; a *different functional
form* is fatal. So **the distribution $q(x_t\mid x_0)$ is locked.**

### Requirement 2 — the reverse distribution must be simple (Gaussian)

Our entire goal is to go from $x_t$ to $x_{t-1}$. For that we must be able to **sample**, so the
reverse distribution $q(x_{t-1}\mid x_t, x_0)$ has to be a **simple distribution — ideally a
Gaussian** — which we can draw from via reparameterization.

> **Study tip:** when learning diffusion models, *think simple*. A lot of things are simply
> **assumed** by the authors. Don't agonize over whether the Gaussian is "optimal" or whether some
> fancier form would be better. The goal is to generate images — the simplest, most direct choice is
> often a perfectly good one. Since dropping Markov left $q(x_t\mid x_{t-1}, x_0)$ unknown, we can't
> *derive* the reverse distribution anymore, so we **assume** it's Gaussian and move on.

---

## Setting up the ansatz (undetermined coefficients)

By analogy with DDPM, assume DDIM's reverse denoising distribution is also a DDPM-like Gaussian:

$$
q_{\sigma_t}(x_{t-1}\mid x_t, x_0) \sim \mathcal{N}\big(\mu(x_0, x_t),\ \sigma_t^2 \mathbf{I}\big). \tag{9}
$$

- **Mean** $\mu(x_0, x_t) = k\,x_0 + m\,x_t$ — a weighted combination of $x_0$ and $x_t$ (the
  conditions), with $k, m$ as **undetermined coefficients** to solve for. (DDPM's mean was likewise
  a weighted sum of $x_0$ and $x_t$.)
- **Variance** $\sigma_t^2 \ge 0$ is a **human-specified, known constant** — exactly as DDPM's
  reverse variance $\tilde\beta_t$ was a constant. We're free-loading all the way: copy the mean
  *form* and assume the variance is a known constant too.

Reparameterized sampling then gives:

$$
x_{t-1} = \mu(x_0, x_t) + \sigma_t\,\epsilon_t = k\,x_0 + m\,x_t + \sigma_t\,\epsilon_t. \tag{10}
$$

One equation, two unknowns ($k, m$) — not solvable alone. We need another equation, then solve the
system.

---

## The second equation: Requirement 1 in action

By Formula 2 **and** Requirement 1, the marginal of $x_{t-1}$ is locked to the DDPM form:

$$
x_{t-1} \sim \mathcal{N}\big(\sqrt{\alpha_{t-1}}\,x_0,\ (1-\alpha_{t-1})\mathbf{I}\big),
\quad\text{i.e.}\quad
x_{t-1} = \sqrt{\alpha_{t-1}}\,x_0 + \sqrt{1-\alpha_{t-1}}\,\epsilon_{t-1}. \tag{11}
$$

The two expressions (10) and (11) for $x_{t-1}$ **must be equal** — otherwise Requirement 1 fails:

$$
k\,x_0 + m\,x_t + \sigma_t\,\epsilon_t = \sqrt{\alpha_{t-1}}\,x_0 + \sqrt{1-\alpha_{t-1}}\,\epsilon'_{t-1}. \tag{12}
$$

**Snag:** the left side has $x_t$, the right side doesn't — we can't pin down $m$. Fix it by
expanding $x_t$ with Formula 2 as well, $x_t = \sqrt{\alpha_t}\,x_0 + \sqrt{1-\alpha_t}\,\epsilon''_t$:

$$
k\,x_0 + m\big(\sqrt{\alpha_t}\,x_0 + \sqrt{1-\alpha_t}\,\epsilon''_t\big) + \sigma_t\,\epsilon_t
= \sqrt{\alpha_{t-1}}\,x_0 + \sqrt{1-\alpha_{t-1}}\,\epsilon'_{t-1}. \tag{13}
$$

Now both sides are organized in terms of $x_0$ and noise.

> ⚠️ **Why the primes ($\epsilon_t$, $\epsilon'_{t-1}$, $\epsilon''_t$)?** Each $\epsilon$ is an
> **independent** standard normal. You **cannot** cancel them across the equals sign or treat them
> with ordinary algebra. The primes are a deliberate reminder.
>
> **Memorize this:** *random variables can only be handled by analyzing their distributions — never
> by elementary add/subtract/multiply/divide.* (We hammered this in the SDE walkthrough.) Two
> independent $\epsilon$'s don't "cancel"; if anything they *combine* into a new variance.

---

## Solving by matching distributions

Since random-variable equations are solved via distributions, match the **distribution** on each
side of (13). Prerequisite — the Gaussian variance-addition property:

$$a\,\epsilon' + b\,\epsilon'' \sim \mathcal{N}\big(\mathbf{0},\ (a^2 + b^2)\mathbf{I}\big).$$

**Left side of (13)** is Gaussian with

$$
\text{mean} = (k + m\sqrt{\alpha_t})\,x_0, \qquad
\text{variance} = \big(m^2(1-\alpha_t) + \sigma_t^2\big)\mathbf{I}.
$$

**Right side of (13)** is $\mathcal{N}\big(\sqrt{\alpha_{t-1}}\,x_0,\ (1-\alpha_{t-1})\mathbf{I}\big)$.

Equating the two Gaussians gives **two equations in two unknowns**:

$$\textbf{(mean)}\quad k + m\sqrt{\alpha_t} = \sqrt{\alpha_{t-1}},$$
$$\textbf{(variance)}\quad m^2(1-\alpha_t) + \sigma_t^2 = 1-\alpha_{t-1}.$$

Solve the variance equation for $m$, then back-substitute for $k$:

$$
m = \sqrt{\frac{1-\alpha_{t-1}-\sigma_t^2}{1-\alpha_t}},
\qquad
k = \sqrt{\alpha_{t-1}} - \sqrt{\alpha_t}\cdot\sqrt{\frac{1-\alpha_{t-1}-\sigma_t^2}{1-\alpha_t}}. \tag{14}
$$

Plugging $k, m$ back into the assumed mean $\mu = kx_0 + mx_t$ and collecting terms yields the
paper's reverse distribution:

$$
\boxed{\,q_{\sigma_t}(x_{t-1}\mid x_t, x_0) = \mathcal{N}\!\left(
\sqrt{\alpha_{t-1}}\,x_0 + \sqrt{1-\alpha_{t-1}-\sigma_t^2}\cdot\frac{x_t - \sqrt{\alpha_t}\,x_0}{\sqrt{1-\alpha_t}},\
\ \sigma_t^2\mathbf{I}\right).\,} \tag{15}
$$

> *A small rant:* the DDIM paper just **states** this result — it shows almost no derivation, very
> "look how clever I am." The undetermined-coefficients method above is the honest way to get $k$
> and $m$.

---

## Where's the step-skipping?

The distribution (15) still looks like it relates $t$ and $t-1$. So where's the acceleration?

The key: **we never imposed the Markov property**, so $t-1$ and $t$ denote the $(t{-}1)$-th and
$t$-th **sampling** steps — which **need not be consecutive in DDPM time**.

> Example: the $t$-th DDIM step can correspond to DDPM's **900th** timepoint, and the $(t{-}1)$-th
> DDIM step to DDPM's **800th** — a **100-step jump** in one move. Allowed, precisely because no
> Markov assumption was used.

We never eliminated $x_0$ via Markov; instead we *posited* the whole distribution directly (in order
to free-load). Both requirements are met, so **the Markov shackle is now completely gone — skip away.**

---

## The DDIM sampling formula (Eq 16)

Reparameterize-sample from (15). At inference we don't have the true $x_0$, so we substitute the
**predicted** $x_0$. From $x_t = \sqrt{\alpha_t}\,x_0 + \sqrt{1-\alpha_t}\,\epsilon$, solving for
$x_0$ with the model's noise estimate $\epsilon_\theta(x_t)$ gives
$\hat{x}_0 = \big(x_t - \sqrt{1-\alpha_t}\,\epsilon_\theta(x_t)\big)/\sqrt{\alpha_t}$. The result:

$$
\boxed{\,x_{t-1} = \sqrt{\alpha_{t-1}}\underbrace{\left(\frac{x_t - \sqrt{1-\alpha_t}\,\epsilon_\theta(x_t)}{\sqrt{\alpha_t}}\right)}_{\text{``predicted }x_0\text{''}}
+ \underbrace{\sqrt{1-\alpha_{t-1}-\sigma_t^2}\cdot\epsilon_\theta(x_t)}_{\text{``direction pointing to }x_t\text{''}}
+ \underbrace{\sigma_t\,\epsilon_t}_{\text{random noise}}.\,} \tag{16}
$$

### Reading the three terms (a geometric picture)

Take $x_t$ as the origin (think simple!). Then $\hat x_0 \approx$ the estimated clean image, sitting
some distance away.

- **"predicted $x_0$"** — the model's estimate of the clean image, obtained by inverting Formula 2.
  Straightforward.
- **"direction pointing to $x_t$"** — this term carries the **opposite sign** from the noise inside
  the predicted-$x_0$ term. The predicted-$x_0$ piece moves *away from* $x_t$ (along $-\epsilon_\theta$);
  this $+\epsilon_\theta$ piece pushes *back toward* $x_t$. Net effect: it nudges the trajectory back
  a bit in the $x_t$ direction.
- **random noise** $\sigma_t\epsilon_t$ — a stochastic perturbation that scatters the result.

(This also connects to the earlier intuition for *why DDIM can skip steps but not too many* — worth
revisiting alongside.)

---

## The "I" in DDIM: Implicit

DDIM's signature move: **set $\sigma_t = 0$.** Then the random-noise term in (16) **vanishes** — the
sampling process becomes fully **deterministic**.

But a true diffusion process is defined by *randomness* — particles diffuse via Brownian motion. Kill
the randomness and, strictly, it's **no longer a diffusion model**. Yet its underlying network is
still a DDPM-trained model. So we have something that is "diffusion and not-diffusion" at once. The
author's resolution:

> Call it an **Implicit Diffusion Model** — a *hidden* diffusion model. Its essence comes from
> diffusion, but it's not literally a diffusion process, because with $\sigma_t = 0$ there's no
> $dW$ term, no Wiener process — it's an **ODE**.

That deterministic, randomness-free sampler is exactly the **"I" — Implicit** — in DDIM. The model
is rooted in diffusion, but the sampling no longer fits the strict definition of one.

---

## Summary

- Markov was used only to drop $x_0$ from $q(x_t\mid x_{t-1}, x_0)$ (Formula 4 → 5). DDIM forgoes
  Markov, so it must **posit** the reverse distribution instead of deriving it.
- **Two free-loading requirements:** (1) keep the marginal $q(x_t\mid x_0)$ unchanged
  (gradient-equivalence), (2) make the reverse distribution a simple **Gaussian**.
- Assume $\mu = kx_0 + mx_t$ with constant variance $\sigma_t^2$; solve $k, m$ by **matching
  distributions** (never canceling the independent $\epsilon$'s) ⇒ Eq (15).
- Because no Markov was used, $t$ and $t-1$ can be **far-apart** DDPM timepoints ⇒ **step-skipping**.
- Sampling formula (16) = predicted-$x_0$ + direction-toward-$x_t$ + random-noise.
- Setting $\sigma_t = 0$ removes randomness ⇒ deterministic **ODE** sampler ⇒ the **Implicit** in
  DDIM.

---

*Notes taken from [a video tutorial](https://www.bilibili.com/video/BV1v2pUeLE2v/) on DDIM (Part 2 of the series).*


---

## References

- Song, J., Meng, C., & Ermon, S. (2021). *Denoising Diffusion Implicit Models.* ICLR 2021. [arXiv:2010.02502](https://arxiv.org/abs/2010.02502)
- Ho, J., Jain, A., & Abbeel, P. (2020). *Denoising Diffusion Probabilistic Models.* NeurIPS 2020. [arXiv:2006.11239](https://arxiv.org/abs/2006.11239)