Sora's movie-making magic, directed by AI
Published Date: 2/20/2024
Source: axios.com

OpenAI's Thursday unveiling of Sora, which turns verbal commands into remarkably life-like movie clips, sent shockwaves through both the tech and media worlds.

Why it matters: Other firms have their own text-to-video tools, but OpenAI has upped the wow factor.


  • Everyone knew this was coming — but not this soon.

What's happening: AI as a tool for the execution of human intentions seemed to take another leap with Sora's ability to take a couple of sentences from a user and crank out a convincing video up to a minute long.

  • It's not just that the images look real — it's that they feel like movies, the way ChatGPT's conversations sometimes feel like human speech.
  • Sora has what Wired's Steven Levy called "an emergent grasp of cinematic grammar."

What they're doing: "We're teaching AI to understand and simulate the physical world in motion," OpenAI's article introducing Sora declared.

  • For the company's researchers, teaching neural networks to develop a human toddler's grasp of basic physics — motion and gravity, continuity over time, common sense knowledge like "you can't unbreak an egg" — is a key step on the road to the grail of artificial general intelligence.

The big picture: Just as Dall-E raised the hackles of visual artists and ChatGPT triggered soul-searching (and lawsuits) by authors and writers, Sora is giving moviemakers thrills and chills.

  • It seems to promise profound shortcuts around expensive special effects and could upend many of the crafts that built Hollywood.
  • It also raises deep questions about the unique human contribution to artistic creation, the place of human actors and storytellers in visual media, and where future audiences will turn to find pleasure and surprise.

Yes, but: Most of us can't yet play with Sora.

  • The firm says it's making the tool available to select "visual artists, designers, and filmmakers to gain feedback on how to advance the model to be most helpful for creative professionals" — and also to "red team" researchers, who will assess "harms and risks."
  • Skeptics argued that OpenAI had cherrypicked its demo examples and that Sora's output didn't appear substantially different from that of Midjourney and similar tools.

Like each breakthrough in generative AI that preceded it, Sora summons an unholy trinity of problems: it could be violating creators' rights; it could flood the internet with misinformation and hate speech; and it "hallucinates" errors.

  • "The training data is from content we've licensed and also publicly available content," an OpenAI researcher told Wired — but intellectual property holders are already challenging how much "publicly available content" AI builders can use to train their programs.
  • "Despite extensive research and testing, we cannot predict all of the beneficial ways people will use our technology, nor all the ways people will abuse it," OpenAI said in announcing Sora. "That's why we believe that learning from real-world use is a critical component of creating and releasing increasingly safe AI systems over time."

Of note: Even as engineers work to perfect the output of tools like Sora, creative artists will be much more interested in playing with the mistakes.

  • For instance, OpenAI asked Sora for a scene with "New York City submerged like Atlantis. Fish, whales, sea turtles and sharks swim through the streets of New York."
  • Sora presented a finny phalanx flying between skyscrapers above the surface of a street-level sea. There are some bubbles in the air, too, so maybe there's a new halfway-zone between sea and sky?
  • The video is obviously"wrong" — but also weirder and wilder than the prompt's request.

My thought bubble: I watched OpenAI's Sora demos early on a California February morning and was appropriately astounded.

  • But then I looked up from my laptop. Outside my San Francisco Bay-facing window, the sunrise was catching the Bay Bridge and SF's skyline with golden lateral light under a canopy of salmon clouds. And I forgot all about the videos.