Why OpenAI is Hiding its Reasoning Model’s ‘Thoughts’
In the artificial intelligence race, competition is fierce. Every big AI developer is watching rivals like a hawk and trying to reverse-engineer or copy their best work, as we’ve reported.
What can a leader like OpenAI do to keep its edge? The company’s newly released reasoning model, o1-preview, aka Strawberry, shows one way: by hiding the ball on how the model actually solves problems.
The blog post announcing o1-preview last week said the model uses an “internal chain of thought” to break down problems into simpler steps before solving them. As we have previously shown, developers have used “chain-of-thought prompting” to get existing large language models, including OpenAI’s GPT-4, to perform better on complex or multistep queries.
The new reasoning model does this on its own, but without showing its work to the customer. Instead, the o1 model shows a “model-generated summary of the chain of thought,” which implies that its thoughts are rewritten by a different model altogether before the customer sees them.
OpenAI said it decided to keep the raw chain of thought hidden primarily because that would allow its employees—and only its employees—to “read the mind” of the model to understand how it operates. OpenAI said it doesn’t want the model’s unfiltered thinking to be shown because it might contain unsafe thoughts and that the company wants to “monitor” the model to make sure it’s not being treacherous, such as by “manipulating” the customer.
But OpenAI didn’t hide the fact that another factor in its decision was a “competitive advantage.” That’s understandable.
The move implies that o1-preview could perform even better if its chain of thought wasn’t filtered, as customers might be able to get more out of it by refining their questions in response to what the model was thinking.
We’re curious whether OpenAI will find ways to limit the purported performance dip from hiding the chain of thought before it launches a fuller version of the model, o1.
Some developers have said they are annoyed by the hidden chain of thought because they could get billed for something they can’t see. OpenAI charges developers based on how many tokens—words or parts of words—its models process and spit out in the form of answers.
Still, reviews of o1-preview continue to be largely positive among developers who post about it on X.
The good reception to o1-preview ups the ante for rivals such as Google. The search company has already had a tough time winning over customers for its Gemini LLMs because of how alarmingly difficult Google makes it to use them, as my colleague Erin reported on Monday.
Thanks to OpenAI, the mountain Google must climb to attract business customers just got taller.