OpenAI’s Sora Creates Music Video for Indie Artist

Screenwriter and director Paul Trillo has showcased one of the first music videos entirely generated by OpenAI’s artificial intelligence, Sora.

The video was created for indie chillwave musician Washed Out (Ernest Weatherly Greene Jr.) and his new single, The Hardest Part. The four-minute clip is a series of rapid shots across various scenes, seamlessly stitched together to create the illusion of a continuous zoom.

AI Generates Entire Video

As part of the project, Sora generated over 700 clips, from which 55 individual segments were selected and edited into a complete video sequence using Adobe Premiere.

According to the director, achieving the desired result required crafting detailed prompts for the AI, which included not only the image itself but also camera angles and character movements.

“We fly through a bubble, it pops, we fly through gum and emerge onto an open football field,” Trillo wrote as a prompt for one short video segment.

The director noted that the video solely utilized text-to-video generation capabilities. He did not employ the increasingly popular technique of animating static frames with AI.

Trillo conceived the idea for the video nearly a decade ago but abandoned it at the time. Now, with the help of AI, he has brought it to fruition.

First official commissioned music video made with @OpenAI Sora for @realwashedout

This was an idea I had almost 10 years ago and then abandoned. Finally was able to bring it to life.

Watch the full video here https://t.co/sGpmMLVCul pic.twitter.com/J3RxRD9nzo

— Paul Trillo (@paultrillo) May 2, 2024

In an interview with the Los Angeles Times, Greene also shared his eagerness to explore new technologies and assess their impact on creativity.

“So if this is pioneering, I’m happy to be part of it,” the musician added.

A First of Its Kind?

OpenAI unveiled the Sora video generator in February 2024. The tool generated excitement on social media, yet it has not been released to the public.

Developers are concerned that the generator could be used to create manipulative and misleading videos. Consequently, the startup has limited access to the tool to security experts and a small group of “visual artists, designers, and filmmakers.”

In early April, OpenAI released the video Air Head, created by the Canadian creative studio Shy Kids using Sora. However, it heavily relied on other VFX and video editing tools, such as rotoscoping in Adobe After Effects.

Nonetheless, the emergence of such videos indicates a desire among creatives to harness new AI tools for expression and storytelling. Meanwhile, many criticize the technology and OpenAI, particularly for infringing on copyrights by scraping and training algorithms on artists’ works without their consent or compensation.

Previously, research showed that people struggle to distinguish AI-generated videos from real footage.