TheChek News Flow

Google’s upcoming computer-using agents aims to help consumers with everyday tasks, in contrast to agents from rivals that focus on software engineering and other work tasks.

Those plans are tentative and could change.

Google is developing artificial intelligence that takes over a person’s web browser to complete tasks such as gathering research, purchasing a product or booking a flight, according to three people with direct knowledge of the product. The product, code-named Project Jarvis, is similar to one Anthropic announced this week, these people said.

Google plans to preview the product, also known as a computer-using agent, as early as December alongside the release of its next flagship Gemini large language model, which would help power the product, two of the people said.

The timeline for releasing Jarvis—which shares its name with Tony Stark’s AI assistant in “Iron Man”—shows how Google is still playing catch-up to startup rivals, despite the fact that Google researchers invented much of the underlying AI technology. In another recent example, Google is still developing AI with so-called reasoning capabilities, which OpenAI launched in September after hiring a researcher who helped invent the reasoning method at Google in 2022.

As a result, Google’s Gemini chatbot has badly lagged ChatGPT, and businesses have flocked to OpenAI’s LLMs, making it difficult for Google’s Gemini models to catch up. Last week, in a bid to make Google’s development of AI more efficient, the company moved the team behind its Gemini chatbot to DeepMind, its main AI group.

AI developers have positioned agents—AI systems that can complete complex tasks without needing human supervision—as a next step for the industry. Enterprise software firms Salesforce, Microsoft and Workday, which purchase LLMs from OpenAI and others, are racing to use the technology to develop agents. They have said the agents will automate simple business tasks, though they are still considered largely experimental.

Anthropic and Google are trying to take the agent concept a step further with software that interacts directly with a person’s computer or browser. OpenAI also has been developing similar software for the better part of the year.

Google’s agent, similar to the one Anthropic launched, responds to a person’s commands by capturing frequent screenshots of what’s on their computer screen, and interpreting the shots before taking actions like clicking on a button or typing into a text field, two of the people said.

There are key differences between the two companies’ agents. Anthropic has said its product can operate different applications installed on a person’s computer, while Jarvis can only operate a web browser and has been tailored to Google’s Chrome browser, the two people said.

Returning Shoes

And Jarvis, at least for now, targets primarily consumers who want to automate everyday, web-based tasks, the three people said. At Google’s developer conference this spring, for example, CEO Sundar Pichai suggested that a future version of Gemini could take several actions on its own to help someone return a pair of shoes.

By contrast, Anthropic positioned its agent as something that could help software engineers and other office workers do their jobs faster, though it said people could also use it for personal tasks like planning a day trip with a friend.

Initially, Google may release Jarvis to a small group of early testers to help identify and fix its shortcomings, two of the people said.

The agent currently operates relatively slowly because the model needs to think for a few seconds before taking each action, according to two of the people with direct knowledge of the product.

Google will need to convince people that its AI agent would safely handle their personal data, including login passwords and credit card information, which it requires so it can visit different sites to complete tasks or make purchases based on a customer’s request. LLMs have been known to produce errant answers: Google’s use of LLM-powered conversational answers in its search engine, for instance, initially resulted in numerous blatant errors.