Meta previews Movie Gen, AI tools that turn dreams into videos

Meta on Friday previewed new AI tools called Movie Gen that can create videos, automatically edit them and overlay them with AI-generated sound to create a cohesive video clip.

Movie Gen works with written text prompts, an image, or an existing video as input. There is also the option to add a personal image so that users can see themselves in the video.

Related: Meta inserts AI images with personalized images into your Facebook and Instagram feeds

After the AI ​​works its magic and generates a video, a user can enter a text prompt to create a custom audio soundtrack that plays with the video.

^{A look at a video that Meta Movie Gen created from an image. Photo credit: Meta}

While Meta says Movie Gen's high-resolution videos are “the first of their kind in the industry,” that doesn't mean the AI ​​is perfect when it comes to creating long videos with different aspect ratios. The AI ​​can currently only generate videos that are up to 16 seconds long – and it doesn't always manage to classify them.

In a demonstration to the New York Times, Meta's AI tool made a mistake. Although it managed to create a video of a dog talking into a phone in a park, the AI ​​made a mistake by placing a human hand around the phone instead of a dog's paw.

? Today we introduce Meta Movie Gen: the most advanced media foundation models yet.

Developed by AI research teams at Meta, Movie Gen delivers cutting-edge results across a range of features. We are excited about the potential of this research area… pic.twitter.com/NDOnyKOOyq

— AI at Meta (@AIatMeta) October 4, 2024

Chris Cox, chief product officer at Meta, explained in a Threads post that Movie Gen is “industry-leading” in video quality, but Meta isn't ready to release the tools because they're too expensive and the videos currently take too long to generate.

Meta is sharing what it has right now because the results are “going to be pretty impressive,” Cox wrote.

Meta isn't the first to introduce a text-to-video AI generator tool – ChatGPT maker OpenAI did so in February with its text-to-video model Sora.

In July, OpenAI, in collaboration with artists and entrepreneurs, released several YouTube videos that showed how Sora could create amazing short films.

Related: Mark Zuckerberg is better at explaining AI than his competitors