A hand-holding deep dive into DDIM (Denoising Diffusion Implicit Models). This first part lays out the big picture — what problem DDIM solves and why — then quickly recaps the DDPM machinery we'll build on. Prerequisite: you must already know DDPM. If you don't, read the zero-barrier DDPM series first; otherwise this will go over your head.

The whole thesis in one letter

DDIM and DDPM differ by exactly one letter: the I, for Implicit. You read that right — one letter. But that letter is the core of DDIM. If you can't pin down what "implicit" means here, you haven't really grasped DDIM.

Here's the headline:

  • DDPM is slow. The number of sampling (denoising) steps equals the number of noising steps — typically 1000. Generating one image means running 1000 reverse steps. Painful.
  • Why so slow? Because of the Markov property. DDPM's reverse process is shackled to a Markov chain, which forces one-step-at-a-time denoising.
  • DDIM's move: de-Markovianize. By removing the Markov shackle, DDIM slashes the number of sampling steps while keeping image quality essentially unchanged.

The mathematical essence

When you study a diffusion model, always nail down its mathematical nature. Here it is:

DDIM turns DDPM's reverse-time stochastic differential equation (SDE) into an ordinary differential equation (ODE). In the "experienced-driver" picture, it upgrades the car from an SDE-car to an ODE-car.

Why is an SDE worse than an ODE for fast sampling? Intuitively: the moment you see the word stochastic, your heart should sink a little. Recall from the SDE walkthroughs that the reverse SDE carries a term g(t) dWg(t)\,dW, where dWdW is a Wiener process — a random perturbation. With randomness baked into every step, the numerical solution is hard to take in big jumps. An ODE has no such randomness: over a stretch of time it marches in a single, definite direction, so you can confidently skip steps. (Flow matching, an ODE method, is fast for exactly this reason.)

The killer feature: free-loading (白嫖)

If DDIM required retraining a model, it'd be a non-starter. Its strongest selling point is that its sampling algorithm can free-load directly off a model trained with DDPM:

Take any pretrained DDPM model, apply DDIM's sampling procedure, and get high-quality images — no retraining needed.

That constraint — "must reuse the DDPM model as-is" — will turn out to drive the entire derivation.

DDPM recap (the loot we carry in)

Since DDIM free-loads off DDPM, let's quickly reassemble what we already know. (All of this is derived in detail in the DDPM series — we'll move fast.)

Notation convention: every ϵ\epsilon in this writeup — regardless of sub/superscripts — denotes a standard normal ϵ∼N(0,I)\epsilon \sim \mathcal{N}(\mathbf{0}, \mathbf{I}) unless explicitly stated otherwise.

Formula 1 — one-step noising

The forward process is a Markov chain with Gaussian transitions:

q(xt∣xt−1)=1−βt xt−1+βt ϵt ∼ N(1−βt xt−1, βtI).(1)q(x_t \mid x_{t-1}) = \sqrt{1-\beta_t}\,x_{t-1} + \sqrt{\beta_t}\,\epsilon_t \ \sim\ \mathcal{N}\big(\sqrt{1-\beta_t}\,x_{t-1},\ \beta_t \mathbf{I}\big). \tag{1}

Formula 2 — the closed-form "skip" (老司机 jump)

Via the Markov property plus the nesting-dolls substitution (xt−1→xt−2→⋯→x0x_{t-1} \to x_{t-2} \to \cdots \to x_0), we get the experienced-driver formula that stands at x0x_0 and reaches any xtx_t in one shot:

q(xt∣x0)=αt x0+1−αt ϵt ∼ N(αt x0, (1−αt)I).(2)q(x_t \mid x_0) = \sqrt{\alpha_t}\,x_0 + \sqrt{1-\alpha_t}\,\epsilon_t \ \sim\ \mathcal{N}\big(\sqrt{\alpha_t}\,x_0,\ (1-\alpha_t)\mathbf{I}\big). \tag{2}

⚠️ Notation difference vs. DDPM. The DDIM paper drops the bar: what DDPM writes as αˉt=αtαt−1⋯α1\bar\alpha_t = \alpha_t\alpha_{t-1}\cdots\alpha_1 (a cumulative product), DDIM simply writes as αt\alpha_t. They are the same quantity — DDIM's αt\alpha_t is DDPM's αˉt\bar\alpha_t. We follow the DDIM (bar-less) convention from here on.

An important aside: the endpoint is never actually reached

The noising target is to travel from the data distribution pdatap_{\text{data}} to the prior pprior∼N(0,I)p_{\text{prior}} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}). But reaching the prior exactly requires infinitely many steps. So as t→∞t \to \infty, αt→0\alpha_t \to 0 only in the limit, which means

xT ∼˙ N(0,I)(approximately, not exactly).x_T \ \dot\sim\ \mathcal{N}(\mathbf{0}, \mathbf{I}) \quad (\textit{approximately}, \text{ not exactly}).

The endpoint xT∼N(0,I)x_T \sim \mathcal{N}(\mathbf{0}, \mathbf{I}) is an approximation that is never truly attained. In experienced-driver terms, the planned route is slightly "off." With T=1000T = 1000 the error is tiny — but it's nonzero, and this approximation can subtly hurt generation quality.

Formulas 3–6 — the reverse posterior

We want the reverse (denoising) distribution q(xt−1∣xt)q(x_{t-1} \mid x_t), but we only know the forward direction. The standard recipe: Bayes' rule to turn unknown into known.

q(xt−1∣xt)=q(xt∣xt−1) q(xt−1)q(xt).(3)q(x_{t-1} \mid x_t) = \frac{q(x_t \mid x_{t-1})\,q(x_{t-1})}{q(x_t)}. \tag{3}

The marginals q(xt−1),q(xt)q(x_{t-1}), q(x_t) are unknown data distributions — dead end. Fix: condition on x0x_0.

q(xt−1∣xt,x0)=q(xt∣xt−1,x0) q(xt−1∣x0)q(xt∣x0).(4)q(x_{t-1} \mid x_t, x_0) = \frac{q(x_t \mid x_{t-1}, x_0)\,q(x_{t-1} \mid x_0)}{q(x_t \mid x_0)}. \tag{4}

Adding x0x_0 made the numerator's first term unknown again — but the Markov property rescues us: xtx_t depends only on its neighbor xt−1x_{t-1}, not on the distant x0x_0, so q(xt∣xt−1,x0)=q(xt∣xt−1)q(x_t \mid x_{t-1}, x_0) = q(x_t \mid x_{t-1}) (the x0x_0 vanishes), leaving a fully-known fraction:

q(xt−1∣xt,x0)=q(xt∣xt−1) q(xt−1∣x0)q(xt∣x0).(5)q(x_{t-1} \mid x_t, x_0) = \frac{q(x_t \mid x_{t-1})\,q(x_{t-1}\mid x_0)}{q(x_t\mid x_0)}. \tag{5}

Now all three pieces are known (Formulas 1 and 2). Plug in the Gaussian density 12πσexp⁡ ⁣(−(x−μ)22σ2)\frac{1}{\sqrt{2\pi}\sigma}\exp\!\big(-\frac{(x-\mu)^2}{2\sigma^2}\big), grind through elementary algebra, and the reverse posterior comes out Gaussian too:

q(xt−1∣xt,x0)=N(xt−1; μ~t(xt,x0), β~tI).(6)q(x_{t-1} \mid x_t, x_0) = \mathcal{N}\big(x_{t-1};\ \tilde\mu_t(x_t, x_0),\ \tilde\beta_t \mathbf{I}\big). \tag{6}

By reparameterized sampling, knowing xtx_t and x0x_0 gives xt−1x_{t-1}; iterating xT→xT−1→⋯→x0x_T \to x_{T-1} \to \cdots \to x_0 produces the final image. At inference x0x_0 is unknown, so a model predicts it — equivalently predicting the noise ϵ^\hat\epsilon (the two are interchangeable; the "hat" marks a model prediction). Full derivation is in the DDPM reverse-process post.

Formulas 7–8 — the loss (the "driver")

Training the driver = the loss function. Start from maximum likelihood — we want the model to assign the highest probability to generating the images we want (the running "generate basketball-playing Kunkun" gag: maximize pθ(Kunkun)p_\theta(\text{Kunkun})). Take a log⁡\log, flip max to min with a minus sign (it's a loss, lower is better), and after the full DDPM derivation:

Lγ=Eq(x0:T) ⁣[DKL(q(xT∣x0) ∥ p(xT))⏟network-free constant+∑t≥1DKL(q(xt−1∣xt,x0) ∥ pθ(xt−1∣xt))](7)L_\gamma = \mathbb{E}_{q(x_{0:T})}\!\left[ \underbrace{D_{\mathrm{KL}}\big(q(x_T \mid x_0)\,\|\,p(x_T)\big)}_{\text{network-free constant}} + \sum_{t\ge1} D_{\mathrm{KL}}\big(q(x_{t-1}\mid x_t, x_0)\,\|\,p_\theta(x_{t-1}\mid x_t)\big)\right] \tag{7} =∑t=1TEq(x0), ϵt∼N(0,I) ⁣[γt ∥ ϵθ(αt x0+1−αt ϵt⏟= xt, t)−ϵt ∥22].(8)= \sum_{t=1}^{T}\mathbb{E}_{q(x_0),\,\epsilon_t\sim\mathcal{N}(\mathbf{0},\mathbf{I})}\!\left[ \gamma_t\,\Big\|\,\epsilon_\theta\big(\underbrace{\sqrt{\alpha_t}\,x_0 + \sqrt{1-\alpha_t}\,\epsilon_t}_{=\,x_t},\ t\big) - \epsilon_t\,\Big\|_2^2\right]. \tag{8}

Here ϵθ\epsilon_\theta is a network (parameters θ\theta) that predicts the noise ϵt\epsilon_t injected by Formula 2, and γt\gamma_t is a tt-dependent coefficient with γt=1\gamma_t = 1 in DDPM.

Two fine points (both already covered in the DDPM loss post):

  • The sum runs over t≥1t \ge 1, not DDPM's t>1t > 1, and the separate "L0L_0" term has vanished. That's not an error — the t=1t = 1 term folds into the sum via the Kunkun rule p(x1∣x1,… )=1p(x_1 \mid x_1, \dots) = 1: "Given Kunkun practiced two and a half years, what's the probability Kunkun practiced two and a half years? Obviously 1."
  • The network input ϵθ(xt,t)\epsilon_\theta(x_t, t) takes the timestep tt and xtx_t (written out as αtx0+1−αtϵt\sqrt{\alpha_t}x_0 + \sqrt{1-\alpha_t}\epsilon_t). The predicted target ϵt\epsilon_t is the noise in this xtx_t expression — don't confuse it with the adjacent-step noise.

The key observation (and the cliffhanger)

Stare at the loss in Formula 8 for three seconds. Notice:

The loss depends only on the distribution q(xt∣x0)q(x_t \mid x_0) (Formula 2).

This is the crux. If DDIM wants to free-load DDPM's loss — i.e. reuse a pretrained DDPM model — then DDIM's loss must match DDPM's up to at most a constant coefficient (γt\gamma_t, which is just 11 in DDPM anyway). Concretely:

DDIM may change the sampling procedure (to skip steps), but it must not change the marginal q(xt∣x0)q(x_t \mid x_0). Touch that distribution and the loss LγL_\gamma changes — and the free-loading breaks.

Hold that thought.

From this three-factor analysis (route / car / driver), the diagnosis is clear: DDPM's reverse process is shackled by the Markov chain. In car terms, DDPM's car is a clunky SDE-car — it can only crawl one step at a time, making generation hugely inefficient. That is exactly the problem DDIM sets out to fix: upgrade the car.

How does DDIM remove the Markov shackle and turn the SDE-car into an ODE-car — while keeping q(xt∣x0)q(x_t \mid x_0) untouched so it can still free-load the DDPM model? That's the reveal, coming up in Part 2.

Notes taken from a video tutorial on DDIM (Part 1 of the series).

References

  • Song, J., Meng, C., & Ermon, S. (2021). Denoising Diffusion Implicit Models. ICLR 2021. arXiv:2010.02502
  • Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. NeurIPS 2020. arXiv:2006.11239