yousef-rafat/miniDiffusion: A reimplementation of Stable Diffusion 3.5 in pure PyTorch

miniDiffusion is a reimplementation of the Stable Diffusion 3.5 model in pure PyTorch with minimal dependencies. It’s designed for educational, experimenting, and hacking purposes.
It’s made with the mindset of having the least amount of code necessary to recreate Stable Diffusion 3.5 from scratch, with only ~2800 spanning from VAE to DiT to the Train and Dataset scripts.

-Files: The main Stable Diffusion model code is located in dit.py, dit_components.py, and attention.py. The dit.py file contains the main model, dit_components.py contains the embedding, normalization, patch embedding, and help functions for the DiT code, and attention.py contains the Joint Attention implementation.
The noise.py is where the Euler Scheduler is located for solving the ODE of Rectified Flow.

The text encoders are in t5_encoder.py and clip.py, and their tokenizers are both in tokenizer.py. The metrics.py implements the Fréchet inception distance (FID).

The common.py is a place for helper functions for training, the common_ds.py is an implementation of an iterable dataset that converts image data to trainable data for the DiT model.

-Folders: The model folder saves the model’s checkpoint and logs after training. The encoders folder saves other modules’ checkpoints (e.g., VAE, CLIP).

⚠️ Warning:
This repository still has experimental features and requires more testing.

• Implementations of VAE, CLIP, and T5 Text Encoders
• Implementation of Byte-Pair & Unigram tokenizers

• Multi-Modal Diffusion Transformer Model
• Flow-Matching Euler Scheduler
• Logit-Normal Sampling
• Joint Attention

Get the repo

git clone "https://github.com/yousef-rafat/miniDiffusion"

Install Dependencies

pip install -r requirements.txt

Install Checkpoints for Models

• Add a Hugging Face Token in get_checkpoints.py before running the script.

python3 encoders/get_checkpoints.py

This project is under the MIT License and is made for educational and experimental purposes.

Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!