New AI Reasoning Model Trained on Minimal Budget

AI language models are evolving into widely available tools, highlighted by the emergence of open-source projects like DeepSeek. A new model named S1 has recently been developed by researchers at Stanford and the University of Washington, trained with less than $50 in cloud computing resources.

S1 positions itself as a competitor to OpenAI’s o1. This model is referred to as a reasoning model because it generates answers by logically considering related questions to validate its conclusions. For example, if tasked with estimating the cost to replace all Uber vehicles with Waymo's fleet, S1 would break the problem into manageable parts, such as identifying the current number of Ubers on the road and the manufacturing cost of a Waymo vehicle.

As reported by TechCrunch, S1 is built on a standard language model and has been trained to reason by analyzing questions and their answers from Google's Gemini 2.0 Flashing Thinking Experimental. This Google model visually demonstrates the reasoning process behind its answers, allowing the developers of S1 to train their model with a modest dataset of 1,000 thoughtfully selected questions and answers. This method effectively teaches S1 to replicate the reasoning techniques of Gemini.

In a notable twist, the researchers improved S1's reasoning capabilities with a simple technique: they instructed the model to "wait" during its processing phase. Adding this word helped S1 arrive at more accurate responses, indicating that there is still much to explore in enhancing AI performance through straightforward adjustments.

This development suggests that while there are concerns about AI models reaching their limits, there are still significant opportunities for simple yet effective improvements. Some advancements in computer science now revolve around discovering the right keywords or commands to refine model performance.

There has been controversy surrounding OpenAI's response to the Chinese DeepSeek team, which has reportedly trained its models based on OpenAI's outputs. This situation is ironic, as major models like ChatGPT were developed using information sourced from the internet without formal permission. This issue is ongoing in legal disputes, particularly with publishers like the New York Times, who are seeking to protect their content from unauthorized use. Moreover, Google officially forbids other companies, including S1, from utilizing the outputs of its Gemini model for training.

While S1 demonstrates impressive performance, it does not imply that one can construct a fully functional model from scratch for under $50. Instead, S1 benefits from the extensive training that went into developing Gemini, serving as a shortcut in the process. A fitting comparison would be to think of a reduced version of an AI model as akin to a JPEG image: it captures essential information but can lose detail in the compression. Current large language models also grapple with numerous accuracy issues, especially when attempting to provide comprehensive answers sourced from the internet. Even key figures at Google often overlook fact-checking the AI-generated text.

Despite these limitations, models like S1 could play vital roles in on-device applications, such as those seen in Apple Intelligence. The increasing availability of affordable, open-source models like S1 raises discussions about what this trend implies for the tech industry. The question remains: could OpenAI face a decline if its models can be easily replicated? Proponents argue that commoditization was always on the horizon for language models, and that companies like OpenAI and Google can thrive by creating valuable applications based on these models. With over 300 million users engaging with ChatGPT weekly, this platform has become a hallmark of modern chatbots and a pioneering approach to search functionalities. The key differentiator will likely be the user interface that leverages these models, like OpenAI’s Operator, which can browse the web on behalf of users, or unique datasets like xAI’s access to X (formerly known as Twitter).

It is also essential to remember that while models may become inexpensive to train, the actual process of inference—the computation needed to respond to user queries—remains costly. As AI technology becomes cheaper and more accessible, it is expected that the demand for computing resources will grow significantly. OpenAI’s ambitious project to establish a $500 billion server farm will not be wasted, provided the current excitement surrounding AI is not just a passing trend.

AI, Model, Training