Understanding diffusion models by implementing DDPM

1. Why did you choose diffusion models?

I chose diffusion models because I wanted to understand generative modeling from its mathematical foundations rather than treating image generation as a black-box capability of a pretrained model.

I began by studying the original DDPM paper, Denoising Diffusion Probabilistic Models by Ho, Jain, and Abbeel. What interested me was the central idea: a model can learn to generate images by reversing a gradual process of adding noise to data.

Unlike approaches that generate an image in a single forward pass, DDPM learns a denoising process that progressively transforms noise into a structured sample. This creates an interesting connection between probabilistic modeling, neural networks, optimization, and iterative inference.

I wanted to understand how the forward diffusion process is constructed, how the reverse process is learned, and why predicting noise provides a useful training objective.

The project became an exercise in reproducing an established generative modeling method, understanding its underlying mathematics, and implementing a working image-generation pipeline in PyTorch.

2. What did you implement from scratch?

I implemented the core DDPM pipeline in Python and PyTorch, using the original paper as the primary reference.

My work covered the following components:

• Forward diffusion: Gradually corrupting training images with Gaussian noise according to a predefined noise schedule.
• Noise scheduling: Implementing the mechanism that determines how noise levels change across diffusion timesteps. I used external guidance while developing this component.
• Denoising U-Net: Implementing the neural network responsible for predicting the noise added to an image at a given timestep.
• Training objective: Training the denoiser to predict the noise used to corrupt the original image.
• Reverse diffusion: Implementing the iterative sampling process that uses the trained network to progressively denoise a sample.
• Image generation: Sampling from random noise and visualizing the resulting generated images.
• Dataset integration: Preparing image datasets for training and running the model on MNIST, CIFAR-10, and a reduced CelebA-HQ subset.

The implementation was built using PyTorch, NumPy, and Matplotlib.

The important distinction is that I implemented the core method and learned how its components fit together, while using the paper and external guidance to help resolve implementation details. This was a paper-based implementation rather than a claim of inventing a new diffusion architecture.

3. What mathematical ideas did you work through?

The most important part of the project was understanding how the probabilistic forward process makes the reverse generative process tractable to learn.

The forward diffusion process

Let \(x_0\) denote a clean training image. At each timestep, the process adds Gaussian noise:

Here, \(\beta_t\) controls the variance of the noise added at timestep \(t\).

Defining \(\alpha_t=1-\beta_t\) and \(\bar{\alpha}_t=\prod_{s=1}^{t}\alpha_s\), we can sample a noisy image directly at any timestep:

This closed-form expression was particularly useful because it allows training examples to be constructed at randomly sampled timesteps without simulating every preceding diffusion step.

Learning the reverse process

The generative model learns to reverse the corruption process, starting from Gaussian noise and progressively recovering image structure.

The reverse transition is parameterized by a neural network that predicts the noise associated with the current sample and timestep.

The model therefore learns a mapping of the form:

where \(\theta\) represents the denoiser's parameters.

The training objective

The simplified DDPM training objective is:

Training minimizes the mean-squared error between the actual injected noise and the noise predicted by the U-Net.

Working through these equations helped me connect the probabilistic formulation to the practical implementation: sample an image, choose a timestep, add noise, predict the noise, calculate the loss, and update the network.

It also helped me understand why the noise schedule and the timestep representation are fundamental to the learning problem rather than merely implementation details.

4. What was your U-Net architecture?

I implemented a U-Net-style denoising network conditioned on the diffusion timestep.

The U-Net provides an encoder-decoder structure that processes images at multiple spatial resolutions. Its skip connections allow information from earlier encoder stages to be combined with decoder features, helping preserve spatial detail during reconstruction.

Timestep conditioning is essential because the denoiser must learn different behavior depending on how much noise has been added to an image.

At a high level, the architecture consists of:

• An image-processing path that extracts increasingly abstract features.
• A timestep-conditioning mechanism that informs the network about the current diffusion step.
• A decoder path that reconstructs spatially organized features.
• Skip connections between encoder and decoder stages.
• An output layer that predicts a noise tensor with the same spatial dimensions as the input image.

I implemented the denoiser as part of the overall DDPM training and sampling pipeline.

The exact channel widths, number of resolution levels, residual-block configuration, normalization layers, attention layers, and timestep-embedding dimensions should be taken from the source code. I would not claim a particular number of attention blocks or architectural modifications without verifying the implementation.

5. Which datasets and training configurations did you use?

I trained the diffusion model on three image datasets with different characteristics.

MNIST: Grayscale handwritten digits. This provided a relatively simple setting for testing whether the model could learn a meaningful image distribution.

CIFAR-10: Small RGB images spanning ten object categories. This introduced more varied visual structure and a more challenging generation problem.

CelebA-HQ subset: A reduced collection of face images. This allowed me to explore diffusion-based generation on more visually complex, structured imagery.

The implementation used PyTorch, with NumPy for numerical operations and Matplotlib for visualizing generated samples and experimental outputs.

The training workflow included constructing noisy examples, predicting noise with the U-Net, calculating the training loss, and updating model parameters.

The exact image resolutions, number of training images, epoch counts, batch sizes, optimizer settings, training duration, and hardware used for each dataset should be recovered from the training scripts and logs before publication.

These details matter because performance across datasets cannot be compared fairly without accounting for differences in image resolution, data volume, model capacity, and training budget.

6. What experiments and ablations did you perform?

The primary goal was to implement and run the baseline DDPM method successfully across multiple datasets.

The project involved working through the diffusion schedule, training objective, denoising architecture, and iterative sampling procedure to produce generated images.

Noise scheduling was an important implementation consideration, and I explored the role of the scheduler while developing the pipeline. However, I would distinguish implementing and using a schedule from conducting a controlled experimental comparison between schedules.

I do not have verified evidence of a completed, systematic ablation study comparing linear and cosine schedules, varying timestep counts, changing U-Net architectures, or benchmarking alternative sampling procedures.

If I extended the project, I would conduct controlled experiments comparing linear and cosine schedules while keeping the architecture, training budget, dataset split, and evaluation procedure fixed.

I would also investigate the effect of the number of diffusion timesteps and compare generation quality against sampling latency. These experiments would help quantify the trade-off between computational cost and sample quality.

7. What results can you substantiate?

The concrete outcome was a working DDPM implementation that successfully trained and generated images across MNIST, CIFAR-10, and a reduced CelebA-HQ subset.

This demonstrated that the implementation could connect the forward corruption process, learned denoising, and iterative reverse sampling into an end-to-end generative modeling pipeline.

I would prioritize showing actual generated samples and the implementation itself as evidence of the work.

Although an 18% FID improvement has been mentioned in earlier project descriptions, I do not currently have verified baseline and final FID values or the evaluation protocol needed to substantiate that claim. I would omit it from the public portfolio until the underlying results can be recovered.

For a rigorous evaluation, I would report FID with the baseline and final values, dataset and sample counts, the evaluation implementation, and the sampling conditions. I would also distinguish visual inspection from quantitative evaluation: plausible-looking samples alone do not establish an improvement in the learned distribution.

8. What problems did you encounter?

One of the main challenges was translating the mathematical formulation of DDPM into a correct implementation.

Noise scheduling: The schedule must be applied consistently in both the forward process and reverse sampler. Errors in the cumulative noise coefficients can prevent the model from learning the intended denoising process.

Timestep conditioning: The network needs to distinguish between lightly corrupted images and samples dominated by noise. Conditioning on the timestep is therefore central to the denoising task.

Iterative sampling: Generation requires repeatedly applying the learned reverse transition. This makes inference more computationally expensive than generation methods that require a single model evaluation.

Dataset differences: MNIST, CIFAR-10, and face images have different image statistics and levels of visual complexity. A working model on one dataset does not automatically imply equivalent performance on another.

Implementation complexity: The forward process, training objective, noise schedule, U-Net, and reverse sampler must agree on tensor shapes, timestep conventions, and noise coefficients. Debugging these interactions requires understanding the mathematics as well as the code.

These challenges made the project a useful exercise in implementing a probabilistic generative model from a research paper rather than relying exclusively on a high-level pretrained pipeline.

9. What did you learn, and what would you investigate next?

The project gave me a more concrete understanding of how generative models can learn complex data distributions through a sequence of relatively simple denoising operations.

It strengthened my understanding of probabilistic modeling, Gaussian noise processes, neural-network optimization, and the relationship between training objectives and sampling procedures.

It also reinforced an important lesson about research: successfully reproducing a method is a meaningful technical milestone, but it is different from demonstrating a new research contribution. The next step is to develop controlled experiments that isolate specific design choices and measure their effects.

If I continued the project, I would investigate:

1. Noise schedules: How linear and cosine schedules affect training dynamics and sample quality.
2. Sampling efficiency: How fewer reverse steps influence generation quality and inference time.
3. Model capacity: How U-Net depth and channel width affect quality, computational cost, and memory consumption.
4. Evaluation methodology: How FID and qualitative inspection complement one another when comparing generated samples.
5. Training stability: How optimization settings and training duration influence the consistency of generation.
6. Learned representations: How the denoising network's behavior changes across timesteps and whether intermediate features reveal useful structure.

These directions would turn the baseline implementation into a more systematic investigation of the design choices behind diffusion models.

10. Where can people see the work?

GitHub repository: https://github.com/HardikLovesTech/DiffusionModels

The repository is the main artifact for examining the implementation.

For my portfolio, I would include:

• A link to the source code.
• A diagram of the forward and reverse diffusion processes.
• The U-Net architecture and timestep-conditioning explanation.
• A grid of generated samples from MNIST, CIFAR-10, and the CelebA-HQ subset.
• Training-loss curves, if available.
• A technical explanation of the DDPM objective and sampling procedure.

I would label generated images by dataset and training configuration and avoid presenting unverified quantitative comparisons.

The goal is to make the implementation inspectable and communicate what I learned from reproducing the method, while keeping the distinction between implementation work and original research explicit.