Researchers create reasoning model for under $50, performs similar to OpenAI’s o1

Why it matters: Everyone’s coming up with new and innovative ways to work around the massive costs involved with training and creating new AI models. After DeepSeek’s impressive debut, which shook Silicon Valley, a group of researchers has developed an open rival that reportedly matches the reasoning abilities of OpenAI’s o1.

Stanford and University of Washington researchers devised a technique to create a new AI model dubbed “s1.” They have already open-sourced it on GitHub, along with the code and data used to build it. A paper published last Friday explained how the team achieved these results through clever technical tricks.

Rather than training a reasoning model from scratch, an expensive endeavor costing millions, they took an existing off-the-shelf language model and “fine-tuned” it using distillation. They extracted the reasoning capabilities from one of Google’s AI models – specifically, Gemini 2.0 Flash Thinking Experimental. They then trained the base model to mimic its step-by-step problem-solving process on a small dataset.

Others have used this approach before. In fact, distillation is what OpenAI was accusing DeepSeek of doing. However, the Stanford/UW team found an ultra-low-cost way to implement it through “supervised fine-tuning.”

This process involves explicitly teaching the model how to reason using curated examples. Their full dataset consisted of only 1,000 carefully selected questions and solutions pulled from Google’s model.

TechCrunch notes that the training process took 30 minutes, using 16 Nvidia H100 GPUs. Of course, these GPUs cost a small fortune – around $25,000 per unit – but renting works out to under $50 in cloud compute credits.

The researchers also discovered a neat trick to boost s1’s capabilities even further. They instructed the model to “wait” before providing its final answer. This command allowed it more time to check its reasoning to arrive at slightly improved solutions.

The model is not without its caveats. Since the team used Google’s model as its teacher, there is the question that s1’s skills, while impressive for its minuscule cost, may not be able to scale up to match the best AI has to offer just yet. There is also the potential for Google to protest. It could be waiting to see how OpenAI’s case goes.

Keep your entertainment at your fingertips with the Amazon Fire TV Stick 4K! Enjoy streaming in 4K Ultra HD with access to top services like Netflix, Prime Video, Disney+, and more. With an easy-to-use interface and voice remote, it’s the ultimate streaming device, now at only $21.99 — that’s 56% off!

With a 4.7/5-star rating from 43,582 reviews and 10K+ bought in the past month, it’s a top choice for home entertainment! Buy Now for $21.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!