Forget megabucks Nvidia GPUs, apparently all you need to run an LLM is a Pentium II CPU from 1997

2025-05-07 05:53:00
www.pcgamer.com

Conventional wisdom says you need a mountain of Nvidia GPUs at about $50,000 a pop to have a chance of running the latest AI models. But apparently not. EXO Labs (via Indian Defence Review) claims to have got the Llama 2 LLM up and running on a Windows 98 box circa 1997 courtesy of a mere Pentium II processor. Hurrah! The catch? It’s running about 20,000 times slower than on a modern GPU. Haroo.

Apparently, Exo Labs picked up the machine for just under $120 on eBay, following which perhaps the biggest headache was getting peripherals to work, what with the legacy PS2 ports and just a single USB input.

Indeed, getting the required files onto the machine was a serious headache. Then there was compiling the files in a format which was compatible with the Pentium II’s ancient instruction set.

Anywho, with the code and hardware sorted, it was time to run Llama 2. Reportedly, the 260K parameter version of the model achieved 39.31 tokens per second on the Pentium II, while the larger 15M parameter version hit just 1.03 tokens per second.

They even tried a partial data model run using a one billion parameter version of Llama 3.2 which returned a glacial 0.0093 tokens per second. To put that into context, there are references to the one billion parameter 3.2 model hitting 40 tokens per second on Arm CPUs and 200 tokens per second on GPUs.

In other words, it’s running about 20,000 times slower on the Pentium II. But hey, it’s running. The comparison isn’t perfect, there are all kinds of variables in terms of how the models are set up. But that 20,000 times figure probably gives the right idea in terms of the performance delta in rough order of magnitude terms.

Indeed, while it’s impressive to get a modern LLM running on such an old CPU, the performance gap is a reminder that speed matters. Actually, it’s a bit like 3D gaming.

Compiled correctly, you’d could no doubt get Cyberpunk 2077 running in full path-traced mode on a Pentium II at 4K. But you’d probably be looking a frame rate similar to the P II’s 0.0093 tokens per second performance. At which point, it’s all a bit academic.

But maybe it would be fun to watch the pixels being rendered, one-by-one. On the other hand, completing a benchmark run could take years. Perhaps we’ll leave all that, for now.

Source Link

Take your gaming to the next level! The Redragon S101 RGB Backlit Gaming Keyboard is an Amazon’s Choice product that delivers incredible value. This all-in-one PC Gamer Value Kit includes a Programmable Backlit Gaming Mouse, perfect for competitive gaming or casual use.

With 46,015 ratings, an average of 4.6 out of 5 stars, and over 4K+ bought in the past month, this kit is trusted by gamers everywhere! Available now for just $39.99 on Amazon. Plus, act fast and snag an exclusive 15% off coupon – but hurry, this offer won’t last long!

Score this incredible deal now!

Start your free Amazon Prime trial
today and unlock unlimited streaming and more!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!

BITCOIN

Bitcoin Logo