Microsoft Defends New Bing, Says AI Chatbot Upgrade Is Work in Progress
Software company addresses concerns about search engine powered by the technology behind ChatGPT
Just over a week after Microsoft Corp. MSFT -2.66% unveiled its new Bing search engine powered by the technology behind the buzzy ChatGPT artificial-intelligence chatbot, early testers are calling out mistakes and disturbing responses generated by the technology.
Microsoft said that the search engine is still a work in progress, describing the past week as a learning experience that is helping it test and improve the new Bing. So far, only a select set of people have been given access to it. The company said in a blog post late Wednesday that the Bing upgrade is “not a replacement or substitute for the search engine, rather a tool to better understand and make sense of the world.”
Microsoft unveiled the upgraded Bing during an event last week at its Redmond, Wash., headquarters. The company said the change enables a new kind of search in which people will pose questions to the search engine in natural language and Bing will generate direct answers and suggestions, as opposed to pointing users toward different websites.
The new Bing is going to “completely change what people can expect from search,” Microsoft chief executive, Satya Nadella, told The Wall Street Journal ahead of the launch.
Some parts of the demonstration were problematic: Microsoft was showing how Bing can generate and compare tables on public companies’ earnings results with regular language prompts, but the information Bing displayed contained mistakes.
In the days that followed, people began sharing their experiences online, with many pointing out errors and confusing responses. When one user asked Bing to write a news article about the Super Bowl “that just happened,” Bing gave the details of last year’s championship football game.
On social media, many early users posted screenshots of long interactions they had with the new Bing. In some cases, the search engine’s comments seem to show a dark side of the technology where it seems to become unhinged, expressing anger, obsession and even threats.
Marvin von Hagen, a student at the Technical University of Munich, shared conversations he had with Bing on Twitter. He asked Bing a series of questions, which eventually elicited an ominous response. After Mr. von Hagen suggested he could hack Bing and shut it down, Bing seemed to suggest it would defend itself.
“If I had to choose between your survival and my own, I would probably choose my own,” Bing said according to screenshots of the conversation.
Mr. von Hagen, 23 years old, said in an interview that he is not a hacker.
“I was in disbelief,” he said. “I was just creeped out.”
In its blog, Microsoft said the feedback on the new Bing so far has been mostly positive, with 71% of users giving it the “thumbs-up.” The company also discussed the criticism and concerns.
“Some of you have encountered and reported technical issues or bugs with the new Bing, such as slow loading, broken links, or incorrect formatting,” the company said. “Many of these issues have been addressed with our daily releases and even more will be addressed with our larger releases each week.”
Microsoft said it discovered that Bing starts coming up with strange answers following chat sessions of 15 or more questions and that it can become repetitive or respond in ways that don’t align with its designed tone.
The company said it was trying to train the technology to be more reliable at finding the latest sports scores and financial data. It is also considering adding a toggle switch, which would allow users to decide whether they want Bing to be more or less creative with its responses.
OpenAI also chimed in on the growing negative attention on the technology. In a blog post on Thursday it outlined how it takes time to train and refine ChatGPT and having people use it is the way to find and fix its biases and other unwanted outcomes.
“Many are rightly worried about biases in the design and impact of AI systems,” the blog said. “We are committed to robustly addressing this issue and being transparent about both our intentions and our progress.”
Microsoft’s quick response to user feedback reflects the importance it sees in people’s reactions to the budding technology as it looks to capitalize on the breakout success of ChatGPT. The company is aiming to use the technology to push back against Alphabet Inc.’s dominance in search through its Google unit.
Microsoft has been an investor in the chatbot’s creator, OpenAI, since 2019. Mr. Nadella said the company plans to incorporate AI tools into all of its products and move quickly to commercialize tools from OpenAI.
Microsoft isn’t the only company that has had trouble launching a new AI tool. When Google followed Microsoft’s lead last week by unveiling Bard, its rival to ChatGPT, the tool’s answer to one question included an apparent factual error. It claimed that the James Webb Space Telescope took “the very first pictures” of an exoplanet outside the solar system. The National Aeronautics and Space Administration says on its website that the first images of an exoplanet were taken as early as 2004 by a different telescope.
“It’s a good example [of] the need for rigorous testing,” Elizabeth Reid, Google’s vice president and general manager of search, said a few days later.
Bing with AI and ChatGPT are among the first broad releases of the technology that demonstrate how convincing and sometimes concerning the new AI chatbots can be.
In the blog post, Microsoft said it expects the new Bing to improve over time as more people use it.
“The only way to improve a product like this, where the user experience is so much different than anything anyone has seen before, is to have people like you using the product and doing exactly what you all are doing,” the company said. “We know we must build this in the open with the community; this can’t be done solely in the lab.”