tanelp/tiny-diffusion: A minimal PyTorch implementation of probabilistic diffusion models for 2D datasets.

A minimal PyTorch implementation of probabilistic diffusion models for 2D datasets. Get started by running python ddpm.py -h to explore the available options for training.

A visualization of the forward diffusion process being applied to a dataset of one thousand 2D points. Note that the dinosaur is not a single training example, it represents each 2D point in the dataset.

This illustration shows how the reverse process recovers the distribution of the training data.

I have run a series of ablations experiments on hyperparameters, such as learning rate and model size, and visualized the learning process. The columns in the graphs represent the checkpoint epoch, and the rows indicate the hyperparameter values. Each cell displays one thousand generated 2D points.

The learning process is sensitive to the learning rate. At first, the model’s output was poor, causing me to suspect a bug. However, simply changing the learning rate value resolved the problem.

The current model configuration doesn’t work well on the line dataset, which I consider the most basic among them. The corners should be clear and sharp, but they are fuzzy.

A longer diffusion process results in a better output. With fewer timesteps, the dinosaur is incomplete, missing points from the top and bottom.

The quadratic schedule does not yield better results. Other schedules like cosine or sigmoid should also be considered.

The capacity of the model doesn’t seem to be a bottleneck, as similar results are obtained across various hidden layer sizes.

As in the hidden size ablation run, the capacity of the model does not seem to be a limiting factor.

The model benefits from the timestep information, but the specific method of encoding the timestep is not important.

The use of sinusoidal embeddings for the inputs helps with learning high-frequency functions in low-dimensional problem domains, such as mapping each (x, y) pixel coordinate to (r, g, b) color, as demonstrated in this study. The same holds true in the current scenario.

Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!