- Chinchillas are cuddly and cute.
- Chinchilla is also an established way to build huge AI models using mountains of data.
- There's at least $3 trillion riding on whether this approach continues or not.
About five years ago, researchers at OpenAI discovered that combining more computing power and more data in ever-larger training runs produces better AI models.
A couple of years later, Google researchers found that adding more data to this mix produces even better results. They showed this by building a new AI model called Chinchilla.
These revelations helped create large language models and other giant models, like GPT-4, that support powerful AI tools such as ChatGPT. Yet in the future, the "Chinchilla" strategy of smashing together oodles of computing and mountains of data into bigger and longer pre-training runs may not work as well.
So what if this process doesn't end up being how AI is made in the future? To put it another way: What if the Chinchilla dies?
Building these massive AI models has so far required huge upfront investments. Mountains of data are mashed together in an incredibly complex and compute-intensive process known as pre-training.
This has sparked the biggest wave of infrastructure upgrades in technology's history. Tech companies across the US and elsewhere are frantically erecting energy-sucking data centers packed with Nvidia GPUs.
The rise of new "reasoning" models has opened up a new potential future for the AI industry, where the amount of required infrastructure could be much less. We're talking trillions of dollars of capital expenditure that might not happen in coming years.
Recently, Ross Sandler, a top tech analyst at Barclays Capital, and his team estimated the different capex requirements of these two possible outcomes:
- The "Chinchilla" future is where the established paradigm of huge computing and data-heavy pre-training runs continue.
- The "Stall-Out" alternative is one in which new types of models and techniques require less computing gear to produce more powerful AI.
The difference is stunning in terms of how much money will or will not be spent. $3 trillion or more in capex is on the line here.
The reason is "reasoning"
"Reasoning" AI models are on the rise, such as OpenAI's o1 and o3 offerings, DeepSeek's R1, and Google's Gemini 2.0 Flash Thinking.
These new models use an approach called test-time or inference-time compute, which slices queries into smaller tasks, turning each into a new prompt that the model tackles.
Reasoning models often don't need massive, intense, long pre-training runs to be created. They may take longer to respond, but their outputs can be more accurate, and they can be cheaper to run, too, the Barclays analysts said.
The analysts said that DeepSeek's R1 has shown how open-source reasoning models can drive incredible performance improvements with far less training time, even if this AI lab may have overstated some of its efficiency gains.
"AI model providers are no longer going to need to solely spend 18-24 months pre-training their next expensive model to achieve step-function improvements in performance," the Barclays analysts wrote in a recent note to investors. "With test-time-compute, smaller base models can run repeated loops and get to a far more accurate response (compared to previous techniques)."
Mixture of Experts
When it comes to running new models, companies are embracing other techniques that will likely reduce the amount of computing infrastructure needed.
AI labs increasingly use an approach called mixture of experts, or MoE, where smaller "expert" models are trained on their tasks and subject areas and work in tandem with an existing huge AI model to answer questions and complete tasks.
In practice, this often means only part of these AI models is used, which reduces the computing required, the Barclays analysts said.
Where does this leave the poor Chinchilla?
The "Chinchilla" approach has worked for the past five years or more, and it's partly why the stock prices of many companies in the AI supply chain have soared.
The Barclays analysts question whether this paradigm can continue because the performance gains from this method may decline as the cost goes up.
"The idea of spending $10 billion on a pre-training run on the next base model, to achieve very little incremental performance, would likely change," they wrote.
Many in the industry also think data for training AI models is running out — there may not be enough quality information to keep feeding this ravenous chinchilla.
So, top AI companies might stop expanding this process when models reach a certain size. For instance, OpenAI could build its next huge model, GPT-5, but may not go beyond that, the analysts said.
A "synthetic" solution?
The AI industry has started using "synthetic" training data, often generated by existing models. Some researchers think this feedback loop of models helping to create new, better models will take the technology to the next level.
The Chinchillas could, essentially, feed on themselves to survive.
Kinda gross, though that would mean tech companies will still spend massively on AI in the coming years.
"If the AI industry were to see breakthroughs in synthetic data and recursive self-improvement, then we would hop back on the Chinchilla scaling path, and compute needs would continue to go up rapidly," Sandler and his colleagues wrote. "While not entirely clear right now, this is certainly a possibility we need to consider."
