nathan-barry/gpt2-webgl: A browser-based, WebGL2 implementation of GPT-2 with transform block and attention matrix visualization

A browser-based, WebGL2 implementation of GPT-2.

• Full GPT-2 small (117M) forward pass in the GPU via WebGL2 shaders
• BPE tokenization using js-tiktoken in the browser (no WASM fetch)
• Simple Python script to download the pretrained weights

• Node.js ≥ 16.x and npm
• Python ≥ 3.8
• A modern browser with WebGL2 support (Chrome, Firefox, Safari, Edge)

We rely on HuggingFace’s transformers to pull down the official GPT-2 weights and emit raw Float32Array blobs:

1. Install Python dependencies:

   pip install torch numpy transformers

2. Run the downloader:

   python download_weights.py

   This will fetch:

   • wte.bin (token embeddings)
   • wpe.bin (positional embeddings)
   • c_attn_q_w_0.bin … c_attn_q_w_11.bin
   • c_attn_k_w_0.bin … etc.
   • lm_head_w.bin, lm_head_b.bin
   • And a generated manifest.json mapping names → URLs

We use Vite to bundle TS, serve ESM modules & handle js-tiktoken:

1. Install JS dependencies:
2. Start the local dev server:
3. Open your browser at

Any changes under src/ will trigger HMR and live-reload.

.
├── public/                  # static assets served at `/`
│   └── weights/             # GPT-2 weight binaries + manifest.json
├── src/
│   ├── gpt2_webgl.ts        # WebGL2 inference + shaders + tokenizer
│   └── main.ts              # bootstrap: loads manifest, sets up UI
├── download_weights.py      # Python script to fetch & dump weights
├── index.html               # (copied by Vite) entrypoint HTML
├── vite.config.ts           # Vite config
├── package.json
└── tsconfig.json

MIT

Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!