The Electric: Behind the Threat to Tesla’s Optimus Robot From Toyota and Boston Dynamics
At the 1939 World’s Fair in New York, Westinghouse Electric unveiled Elektro, a 7-foot-tall robot that walked, spoke, smoked cigarettes and gestured with its arms. It was all a show—Elektro’s witty conversation flowed from phrases recorded on a vinyl disc inside its massive body. But the robot wowed crowds that piled in to witness its apparent smarts.
Eight and a half decades later, Toyota and Tesla have joined a race to commercialize walking, talking robots that resemble humans, are powered by AI and will handle many mundane factory and household tasks. But in many ways, the current versions of the robots resemble Elektro—they still need a hidden hand.
The upshot: Versatile humanoid robots seem unlikely to be available to mainstream consumers for years, if not decades.
At an event on Oct. 10, Tesla CEO Elon Musk showcased the company’s latest version of Optimus, a humanoid robot he unveiled three years ago. As he spoke, numerous Optimus robots walked into the crowd and danced on a stage, and Musk promised that future versions would have capabilities similar to those of C-3PO, the empathetic, diplomatic and sentient robot in “Star Wars.” Tesla would sell billions of such robots, Musk predicted in July, driving the company’s valuation to tens of trillions of dollars, up from $705 billion last week.
But while the Optimus robots shuffled around on their own at the event, human workers behind the scenes appeared to be driving almost everything else they did, conversing through built-in speakers and directing the robots’ arms and hands as they mixed drinks at a bar, according to guests at the event and AI experts. When guests asked, Optimus robots themselves fessed up that they were operated remotely by humans, according to guest videos posted on X. “Regarding speech, I believe this was entirely remote controlled and done with human voices through some sort of radio,” William Stein, a tech analyst with Truist Securities, told me.
Six days after the Tesla event, the Toyota Research Institute, the Japanese carmaker’s research and development arm, announced a partnership with robot maker Boston Dynamics to develop humanoid robots for use in factories and homes. Gill Pratt, CEO of TRI, told me that new advances in AI may make it possible to train humanoid robots to handle numerous mechanical tasks, even those they haven’t explicitly been taught to do.
But that’s probably years away. Pratt said that, just as the World’s Fair visitors did in the 1930s, people still often have unrealistic expectations of how close researchers are to developing robots with near-human abilities.
“We are anthropomorphizing tremendously onto the [robots],” Pratt said. “And we are kind of fooling ourselves and fooling each other by wishful thinking about what we wish that they could do versus what they actually can do, which is very, very little right now.”
Show, Don’t Tell
Musk has not disclosed the technology behind Optimus apart from saying it shares the same motors, gearboxes, batteries and AI software as Tesla’s self-driving cars, and he did not respond to an email seeking comment about Optimus.
The Toyota–Boston Dynamics team uses a new type of AI system similar to the large language models that power ChatGPT, but instead employs large behavioral models to train robots by physically showing them a task: for example with a video. This approach, called diffusion policy, is much less work than writing mountains of line-by-line instructions in code, said Shuran Song, a robotics professor at Stanford University who collaborates with Pratt.
The field is in a primitive state: Pratt’s team has used diffusion policy to teach robots specific tasks, like folding a T-shirt. But they often must show the robot how to fold the shirt 100 times before it can do so reliably.
Song said this points up how complex it is to develop an all-purpose humanoid robot: The robot will technically know how to fold the T-shirt, but it will get confused the next time it’s asked to do so on a bed with a different blanket or sheets, or in a different position, or if the weather visible through the window is not the same.
So trainers show the robot how to fold the shirt in every possible condition they can think of. “When you collect 100 examples of doing that one task, the robot can do that task,” Song said.
And that’s only in one house. Take the robot to a different house and you would have to train it again, Song said. Multiply that process by the number of tasks you’d like a robot to do—serve drinks, answer the door, iron a dress, vacuum the floor—and it becomes clear why humanlike robots are not around the corner.
Making matters worse, unlike with AI chatbots and driverless car technology, there is no ready warehouse of training data for humanoid robots. Researchers are just beginning to create the data.
Toyota’s partnership with Boston Dynamics aims to take the development of such robots to the next stage. Pratt said the companies have assigned the partnership 10 employees each, who will work in a Boston Dynamics facility in Waltham, Mass.
The companies start with a robot called Atlas, developed by Boston Dynamics. At a glance, it appears far more agile than Optimus. In the Oct. 10 event, Optimus robots didn’t fall over or bump into anything, but they also just trundled around, stood still, and generally looked as though they were mired in molasses. By comparison, Atlas flips, leaps, rolls, dances and is generally athletic, as you can see in this video.
The next challenge, Pratt said, is training robots to piece together discrete skills picked up in different tasks to handle a situation they have never seen before. “We have found ways, and we believe it has a lot of potential to go much further, of combining different tasks into general competence at mechanical things,” he said.
One of the most difficult things to replicate is the human hand, with its five fingers, all moving dexterously. No one in robotics has managed it, and even when robot hands get pretty far, the fingers sometimes break off in usage and generally are not robust.
Song suggests equipping humanoid robots with pincers rather than hands and fingers. She picked up a lightweight device on a desk behind her. It had two big pincers and a GoPro camera on top. She called it a handheld gripper and said it could instantly train a robot.
The idea is that when you buy your humanoid robot, you would also purchase one of these grippers. At home, you would take the gripper in your hands, turn on the GoPro, and go about whatever tasks you wanted the robot to carry out, like folding your laundry. Within limits, the robot would be able to replicate what you did after just one go, because it would be custom trained for your home. “Everybody can just carry this gripper and collect a lot of data for the tasks they care about,” she said.
In his Oct. 10 event, Musk predicted that every person on Earth would eventually buy an Optimus, which would become Tesla’s biggest product, surpassing EVs and its future Robotaxis. “I think this will be the biggest product ever of any kind,” he said.
Musk didn’t say when all of this would happen. I asked Song how long she thought it would take before there were all-purpose humanoid robots everywhere. Would our grandchildren see them? “I feel probably years,” she said, “but I don’t think it’s next generation. I think within our generation. When I’m getting old, I will probably have a robot at home.”