The Sora Saga Has Only Just Started
A tiny sneak preview of OpenAI’s groundbreaking video tool last month has sent outsize shockwaves across countless industries. Here’s how Sora is already changing everything from Hollywood to videogames to Madison Avenue.
A month ago, Greg Pilon, a 44-year-old visual effects artist on such films as “Deadpool 2,” thought he heard the tick-tock of a doomsday clock for his industry.
What set the gears in motion? His initial glimpse of Sora, OpenAI’s new text-to-video artificial intelligence tool, unveiled by CEO Sam Altman. In a 10-minute demo video, OpenAI showed the tool instantly spinning simple strings of words into realistic footage: a sweeping drone shot of the Amalfi Coast, a close-up of a man pondering his life at a Parisian cafe, a clip of a groovy kangaroo at a disco and many other scenarios.
Sora appeared far more advanced than any other generative AI video software, and while Pilon had figured it would take years before the other, cruder programs would endanger his livelihood, Sora sparked a fear of imminent obsoletion. “Well, I’m gonna get replaced in two years now,” he recalled thinking to himself.
OpenAI hasn’t even given a release date for Sora, but the tool’s mere unveiling has roiled every industry built around video—from Hollywood to videogames to Madison Avenue, among many others. The most prominent example: A week after Sora’s debut, prolific movie and television producer Tyler Perry canceled an $800 million expansion of his Atlanta studio, with the billionaire saying the technology had potentially eliminated the need for 12 new soundstages.
Yet Perry’s very public move stands out as an exception. Most of Sora’s immediate and longer-term consequences are playing out in private, as conversations with more than a dozen founders, executives and investors within those industries over the past several weeks underscore. Behind closed doors, the sophisticated tool’s imminent arrival has intensified conversations around existing AI; sowed significant anxiety and confusion about what changes Sora will eventually bring; and deepened the problems for a slew of industries beleaguered by broader woes, including budget cuts, layoffs and labor unrest.
Are the fears overblown? Maybe. As much as the Sora reveal seemed like a showcase of strength for OpenAI, it was also a clever piece of marketing—an important factor for a company that must constantly burnish its image as it seeks more employees and more capital. Speaking of capital, it will certainly cost OpenAI dearly to run Sora, and that puts some limits on its disruptive powers. And there’s always the chance that the product won’t connect with consumers as well as OpenAI’s Dall-E image generator and ChatGPT have.
Yet the odds of significant tumult ahead increase based on the fact that it’s not only Sora working on such technology. Startups like Runway and Pika Labs, which have taken in $236 million and $55 million, respectively, from the likes of Google and Lightspeed Venture Partners, have for months been offering AI that can produce clips as long as 15 to 16 seconds. (Sora can reportedly produce minutelong footage.) And two weeks after Sora’s demo, Israeli startup Lightricks announced its text-to-video offering, LTX Studio, and Alibaba debuted its own nascent generative AI video tool, Emo.
“We’re going to see 10 more Sora-type announcements between now and June,” said Mike Grandinetti, a former software engineer who now teaches a Harvard University class on AI and business. “We just have to strap ourselves in.”
Pilon has had to steer through turbulent times before. He initially went to the Conservatory of Recording Arts and Sciences to become an audio engineer, but as audio-editing software advanced in the 2000s, his work prospects dwindled, and Pilon made a break for Hollywood, hoping to try his hand at film and visual effects editing next. He proved pretty good at it and worked on shows including “Game of Thrones,” adding snow to various landscapes as the series’ plot went from summer to winter, and “The Vampire Diaries,” drawing the veins in the eyes of bloodthirsty teens. It’s the type of work Sora might soon be able to do.
After looking at the Sora video, he realized it could have the power to do in minutes what took him months. “If it’s as good as they’re claiming it is…I don’t think anyone expected it to be this far along,” said Pilon, who only just returned to work last week after last year’s Hollywood labor strikes shuttered production. “What we thought was maybe a decade away now is maybe a few years.”
Like Pilon, Ben Kvalo has knocked around Los Angeles’ creative corridors. For a time he led Netflix’s push into videogames after he held earlier roles at two major video companies, 2K and Blizzard Entertainment (now part of Microsoft after that company made a $69 billion deal to buy parent Activision Blizzard last year).
Last May, Kvalo left Netflix and departed California for Wisconsin, where he formed his own videogame publishing company, Midwest Games. Even at a remove from his industry’s epicenter, he can foresee Sora’s possible consequences. In particular, he believes concept artists, who work with videogame developers to sketch out the initial characters and settings for a game, could lose work. “Artists have certain styles that they are good at doing,” whereas generative AI “can do whatever style or can try a completely different mix,” he said.
Kvalo, who expects to see about 40 pitches from indie videogame developers a month as he determines which games his company will release, is readying himself for a surge of AI-generated visuals in those pitches. He believes smaller developers could benefit the most: Smaller teams, who never had the resources to hire concept artists in the first place, are suddenly able to pitch games that “appear at a level above and beyond” where they’re at and secure funding they wouldn’t have otherwise been able to access.
Moreover, Kvalo believes one day generative AI could get to a point where games can have what he calls “sentient” nonplayable characters. The NPCs that populate a game’s world—say, the limping zombies in The Last of Us—are typically restricted to short canned lines and basic movements. If AI could allow those characters to spontaneously invent dialogue and improvise actions, a videogame’s storyline would seem more robust.
“You’re able to deliver the content that allows [players] to engage with [a game] over and over and over and over,” he said. “That is a dream scenario for a lot of game developers.”
On Madison Avenue, many executives talk about Sora less as a harbinger of doom than as a possible godsend. As opposed to the slow-paced auteurism that underpins many movies and games, the ability to quickly develop pitches—possibly with multiple visual styles—and change course fast based on client wishes is what fuels the ad world.
Scott Maiocchi, a senior vice president at creative agency (add)ventures, has already seen his firm, which has worked with major clients like CVS Health and Stop & Shop, use image-generating AI tools such as Midjourney and Stable Diffusion to develop interesting visuals and inspiration images. (In one instance, the process led to an image of a photorealistic man painting a sunset; in another, it rendered the painting man in Corporate Memphis style, drawn with bright colors in a minimalist way.)
Sharing his screen on a video call, Maiocchi gleefully showed off how easy it is for (add)ventures employees to use the custom AI tool. In a Slack channel called addAI, he typed a prompt: a description of a woman captured through a rain-droplet–covered camera lens. The Slack channel immediately spit out a picture created by Dall-E, OpenAI’s image generator.
“Create a 30-second commercial script from that,” he wrote next, watching ChatGPT produce a script. From there, he could give the AI more specific instructions, like “Start the script with a close-up of the woman’s face” or “Add a product tagline to the picture.”
“Anybody in the company right now can use this and play with it,” he said. “It’s not for final delivery on a final product because it’s just not really game ready,” alluding to the litany of copyright infringement lawsuits against generative AI companies.
To name just a few: The New York Times is suing OpenAI and Microsoft for allegedly training their models off the paper’s articles. In September, a group of famous authors including George R.R. Martin and Michael Connelly joined a class-action lawsuit against OpenAI for the same reason. And Getty Images is suing Stability AI, accusing it of using over 12 million copyrighted Getty photographs in its model. There’s a pervasive fear among ad agencies like (add)ventures that using generative AI advertisements could leave them vulnerable to similar litigation, and it’s hard to imagine that Sora and other video AI tools will be any safer to use.
But he believes the models could be ready for commercial use sooner rather than later. Maiocchi predicted that within two years, AI video software will be good enough to take raw commercial footage and, with some text instruction, spit out a final edit. “It’s going to be just that holistic change of how content is actually created,” he said.
Maiocchi has already been experimenting with Pika, RunwayML, ElevenLabs and other generative AI video tools, and he is excited to get his hands on Sora to see if it will “enhance the storytelling process.”
He believes so deeply that generative AI is the future of advertising that he’s added AI skills—or at least a willingness to explore AI technology—to (add)ventures job descriptions.
Similarly, commercial film director Justin Poirier, who has worked with brands such as Bose, CVS and Hasbro, has been using generative AI to build mood boards. Before, he spent hours searching for inspirational images, a particular challenge when he’s working on an animation and needs to show the client options for what the art could look like.
It makes sense, he said, that AI models, which pull from so many artists’ work, would work well in the advertising pitch process. “Most clients want to see that something has worked in the past,” said Poirier. “In a certain sense, advertising and marketing has always relied on the creativity of things that came before it.”
In the future, Poirier believes brands will likely use AI-generated films for b-roll, instead of sifting through Getty or Shutterstock for stock footage as he recently did to find office shots to set the scene for an insurance company’s ad. “Not every project has the budget to hire a drone to shoot a city skyline, but once AI video is widely available, you’re going to be able to make one in seconds,” he said. “To be clear, I’m not advocating for not shooting or using stock, but I’m sure that some clients will start mandating it.”
Advertising executives aren’t the only ones peering through the Sora storm clouds and focusing on the silver lining. In Hollywood, director Scott Mann has been thinking about the technology’s potential for years and predicts it will “have the biggest impact on our industry since the invention of the film camera.”
In 2018, he co-founded Flawless, which makes AI filmmaking tools. The company’s most buzzy product fixes dubbing issues spotted in film editing—saving on costly reshoots—and can also dub movies for foreign markets, using AI to edit actors’ faces so their lips align with the new language.
Mann has deployed the tool in his own films. In his most recent production, “Fall,” a thriller about a pair of friends trapped atop a 2,000-foot communication tower, the characters swore often, and the film initially earned an R rating. He used Flawless software to dub over more than 30 instances of the F-word in “Fall” without requiring reshoots or having to accept that the performances wouldn’t match the audio. The new version received a PG-13 rating, expanding the film’s potential audience, and “Fall” grossed nearly $22 million on a $3 million budget, a roughly 7 times return.
Mann, an action-movie director whose films include 2015’s Robert De Niro–led “Heist,” acknowledges that the technology will radically change some jobs—but believes it will lower production costs so significantly that it could lead to more original, creative projects getting made. “We need a better rate of return, where you’re able to produce and create better, bigger films for less,” he said. “Using the right tools, this is going to save the industry—not destroy it.”