What If LLMs Could Continue Learning?
Here’s a question of paramount importance in AI: what if large language models could continue learning new information even after they’re done training?
That’s the question that researchers at Writer, a $2 billion-valuation startup developing AI tools for enterprises, found themselves asking six months ago. Those researchers have now developed a new type of LLM, called “self-evolving LLMs,” that they claim can continue learning and updating its parameters, or the connections between parts of the model that determine how it answers questions, even after it’s deployed.
The development, if it proves as good as it sounds, might provide a solution to the growing concerns that traditional methods to improve LLMs aren’t working as well as they used to. Those questions have prompted researchers at labs like OpenAI, Google and Meta Platforms to experiment with new methods to keep the “scaling laws” party going.
But if LLMs could continue learning new information even after they’re done training, researchers wouldn’t have to spend hundreds of millions of dollars tweaking and retraining updated versions of their models every few months. This could also open the door to models that can personalize themselves based on new information they learn about their users.
Here’s how Writer’s new approach works. Traditional transformers—the model architecture that underlies popular LLMs like GPT and Claude—are made up of “layers,” or special “filters” that help the model learn different aspects of the input data. Within each of those layers, Writer’s self-evolving LLMs also include a “memory pool,” which stores important information from past interactions, said Writer cofounder and CTO Waseem Alshikh.
Each time the LLM receives new information it hasn’t seen before, it updates those memory pools throughout all its layers so it can refer to that information in future interactions, Alshikh said. (Importantly, researchers have some control over what information the LLM does and doesn’t learn, so you can’t just troll an LLM with fake facts, Alshikh said.)
Developing a self-evolving LLM increases training costs by 10% to 20%, Alshikh said, but doesn’t require additional work once the LLM is trained, unlike other methods to update models with new information like retrieval augmented generation or finetuning.
This type of model leads to some interesting results on popular benchmarks—specifically, the model’s performance on benchmarks actually improves every time it’s tested, since it learns the information on the benchmark over time, Alshikh said. For instance, the first time the model was tested on a common math benchmark, it got less than 25% of the questions correct. The third time it was tested, its accuracy jumped to nearly 75%.
Don’t expect researchers to start abandoning traditional transformers quite yet, though. This type of tech is still extremely early—Alshikh told me that it’s being tested with just two customers in beta right now—and there are plenty of kinks to work out.
Most notably, the more information the LLM learns, the worse it gets at refusing to answer dangerous questions, Alshikh said. Intuitively, you can think of it as—the new information that the model learns over time will begin to override the original data, such as safety instructions, it was trained on. That’s not great news for businesses that want to incorporate AI into customer-facing products.
Another issue is that these models have a limit to how much new information they can learn. However, Alshikh argues that this isn’t as big of a concern for businesses, as they’re typically just trying to update an LLM with their own private information rather than all the latest data on the web. And, he added, if you make the memory pool around 100 billion to 200 billion words, it’s enough for the LLM to learn for at least five to six years for the typical enterprise.
Even so, that does spotlight the still evolving issue of how businesses will use—and pay for—software that’s continuously changing and updating over time. Already, AI businesses are experimenting with new pricing models, such as outcome-based pricing, to get ahead of this shift.
Either way, it seems like other AI leaders are interested in this question as well. Microsoft AI chief Mustafa Suleyman dropped hints on a recent podcast that AI with "near-infinite" memory that "just doesn't forget" is coming in 2025. (We’ll believe it when we see the company’s long-awaited MAI-1 model, though.)
