Apple Can ‘Distill’ Google’s Big Gemini Model
While we have reported that Apple can tweak, or fine-tune, a version of Google’s Gemini AI so that it responds to queries the way Apple wants, the agreement gives Apple a lot more freedom with Google’s tech.
In fact, Apple has complete access to the Gemini model in its own data center facilities. Apple can use that access to produce smaller models that power specific tasks or are small enough to run directly on Apple devices so they can run the tasks faster, said a person who has direct knowledge of the arrangement.
The process of producing such models is called distillation, which essentially transfers knowledge from one large language model, which acts like a teacher, to another model that acts as a student.
In this case, Apple can ask the main Gemini model to perform a series of tasks to produce high-quality results, or answers, including the model’s step-by-step “chain of thought” or reasoning process. Apple can feed those answers or results to a smaller, cheaper model as training data to produce a distilled model.
Since Apple has full access to Gemini, its student model can also learn to imitate the internal computations that Gemini uses to arrive at its answers, which can be more effective than just imitating the answers it spits out.
This results in smaller models that roughly approximate the performance of their state-of-the-art teachers but require significantly less computing power to run.
However, the process is tricky because Gemini is generally optimized for chatbot and enterprise or coding applications, and those don’t always align with what Apple wants Siri to be, this person said.
The deal doesn’t mean Apple has given up on its own development of unique AI models, which still continues in the Apple Foundation Models team, the person said. It isn’t clear how much Apple is investing in that effort.
It may be tempting to think that the foundation models team would consider using the distilled version of Gemini as a base to develop new models. But the person involved in the process was skeptical it would be of much use to that team because it doesn’t seem to be developing its own direct Gemini competitor.
It isn’t clear exactly what the AFM team wants to develop, but it’s safe to say that part of the team’s goal will be to also develop its own small models that work on Apple devices.
With Gemini powering Siri answers for now, Siri will have the ability to answer questions, tell stories, provide emotional support or help people accomplish tasks such as booking travel. (For more details on the AI features Apple will launch with Gemini’s help, see this article.)
Some features, including Siri’s ability to remember past conversations it had with a customer, or proactive features that could suggest they leave home to avoid traffic ahead of an airport pickup, are expected to be announced at the company’s annual developer conference in June.
While Apple and Google arguably forged the most mutually fruitful tech partnership of the last 20 years, the new arrangement doesn’t mean their competition in AI will end.
Just as OpenAI and Apple are each trying to develop personal AI devices for consumers, such as smart glasses and a wearable pin (read about them here and here), Google will surely try to launch its own new devices that could loosen the iPhone’s grip on the mobile consumer device market.