Khosla-Backed Startup Claims Breakthrough With Largest-Ever AI Model on an iPhone
The Takeaway
- PrismML said it has shrunk a 27-billion-parameter AI model to run on an iPhone.
- Startup says its technique enables complex AI tasks on-device without performance loss.
- Apple has talked to PrismML about ways it could use its technology.
Apple is on a quest to shrink powerful AI models to run on iPhones, which could cut down on cloud computing costs and enhance user privacy. But a small startup that emerged from stealth mode earlier this year says it recently got an AI model running on an iPhone bigger than any previous mobile model.
The startup, PrismML, said it has shrunk down Qwen 3.6, an open-source large language model developed by Chinese internet giant Alibaba, to run on an iPhone 17 Pro. The model has 27 billion parameters, which are roughly similar to the synapses in a brain and can help determine the complexity of the data a model can process. In contrast, most models that run on mobile phones have only a few billion parameters active at a time.
The largest AI models, which can measure in the trillions of parameters, are still far too big to run on mobile devices. But the model PrismML has working on an iPhone is capable of tasks like complex chat, reasoning, fully autonomous agents and software coding, the startup said. The open-source model will be available for download next week on Tuesday.
The milestone, which hasn’t been previously reported, reflects a broader push to get AI running on devices instead of expensive high-powered servers in data centers. Microsoft, Amazon, Meta Platforms and others are spending hundreds of billions of dollars racing to build those data centers to keep up with the demand they’re anticipating for AI.
Apple, though, has largely stayed on the sidelines of the costly data center race while also being a vocal proponent of making sure as many of the iPhone’s AI functions as possible run on the devices rather than in the cloud. The company believes on-device AI will better allow it to deliver on its privacy and security promises to customers.
In an interview, Babak Hassibi, CEO of PrismML, predicted that the vast majority of AI will eventually be processed on devices.
“Imagine a world, maybe three years from now, where 95% of the intelligence that you need is available to you locally, on your phone, on your laptop, on your appliances, and it’s really on the last maybe 5% of high-end stuff that you’ll need to go to the cloud,” Hassibi said. “I think that’s how people are seeing the way forward.”
Shrinking models to run on devices, he added, “fundamentally changes the economics of AI.”
Some prominent AI investors have bought into the startup’s vision. Khosla Ventures, which was OpenAI’s first venture capital investor, also put money into PrismML’s $16.25 million seed round earlier this year. In an interview, Vinod Khosla, the firm’s founder, said he became interested in PrismML because the company offers a “fundamental breakthrough.”
“We bet heavily on transformer models in 2018 when we invested in OpenAI, but what are the new ways to build AI?” Khosla said. “Our team is always looking for new approaches.”
PrismML uses a mathematical trick to shrink the Qwen 3.6 model to a fraction of its original size. Shrinking models typically results in worse performance, but the company claims its technique for miniaturizing AI model sizes doesn’t hinder their performance. PrismML has compressed the size of Qwen 3.6 to less than 4 gigabytes, down from around 54.
PrismML is a spinoff of the California Institute of Technology, where Hassibi, a professor of electrical engineering at the school, and his co-founders conducted the mathematical research used in the startup’s technology. Caltech owns the patents behind the technology but licenses them exclusively to PrismML.
PrismML plans to continue shrinking larger AI models, even at the scale of a trillion parameters, which will bring it into the realm of cutting-edge models such as OpenAI’s GPT and Anthropic’s Claude, said Hassibi.
PrismML’s approach may particularly appeal to Apple. At the company’s Worldwide Developers Conference in June, it announced its long-delayed Siri overhaul based on Google’s Gemini models. The most advanced parts of Siri are still so big that they require Apple to tap into Nvidia chips running in Google Cloud.
As part of the new Siri announcement, Apple said some of the iPhone’s new AI capabilities would run on devices. One new on-device Apple model has 20 billion parameters but uses a so-called sparse architecture, in which only 1 billion to 4 billion parameters are active at a time. In the case of PrismML’s on-device model, all 27 billion parameters are active at the same time.
In the past, Apple has run into stumbling blocks with its on-device model efforts. Last year, as it was trying to shrink its internal AI models to fit on the iPhone, the company discovered that performance took a dramatic hit, The Information has reported.
Apple is currently on the hunt for acquisitions of companies that can help it run more AI on device, The Information previously reported. Apple has held meetings with PrismML about ways it could use its technology, people familiar with the talks said.
There are other approaches to on-device AI. For example, startup Argmax is focused on deploying AI on device for tasks like processing voice and images, and then sending that processed information up to the cloud for the more complex reasoning work.
One big reason for that hybrid approach is because the large models running in the cloud are still changing so fast, with updates arriving on a weekly basis. Proponents of that approach argue that AI running entirely on devices won’t get the advantage of the latest, most advanced models, which run in the cloud.