Emu Video Could Bring Meta Closer to AI-Generated Movies

Emu Video is the latest iteration of Meta’s picture generation tool, Emu, and its release hints that the tech giant may be moving closer toward the domain of artificial intelligence-generated films. Emu Video is capable of producing animated GIFs that are just four seconds long based on a caption image or a photo that has been matched with a description.

Emu Edit, which was just recently unveiled, is a companion product to Emu Video that provides customers with an artificial intelligence model that enables the editing of created video. The user has the option of indicating the corrections they desire using natural language, and Emu Edit will implement these revisions, delivering a smooth and user-friendly editing experience for individuals who are exploring AI-generated material.

Meta’s Emu Video

In a relatively short amount of time, Meta’s entrance into the realm of generative artificial intelligence has seen a quick transition from the generation of images to the generation of videos.

Emu was revealed to be the company’s core model for image production during the announcement that took place at Meta Connect. It plays a role in a variety of generative AI experiences, such as the AI image editing tools for Instagram and the Imagine feature featured within Meta AI.

The Emu model is utilized, and a text-to-video generation strategy that is founded on diffusion models is utilized as part of Emu Video. This unified design is capable of responding to a wide variety of inputs, such as text only, image solely, or a combination of text and image.

The procedure consists of two stages: first, the generation of images in response to a text prompt, and then, the generation of videos in response to both the text and the image that was generated. The effectiveness of the training video creation models is significantly improved by this “factorized” or divided method.

In contrast to earlier models, which relied on intricate cascades, Meta’s method used only two diffusion models to make 512×512 films at 16 frames per second for four seconds each.

According to the claim made by the multinational technology corporation, human evaluations point to a strong preference for Meta’s video creation capabilities. Users reportedly prefer this model over previous work due to its higher level of quality and greater fidelity to the text prompt.

Emu Edit

In addition, Meta presents Emu Edit, an innovative method for streamlining picture alteration activities that provide greater capabilities and precision to image editing. Meta developed Emu Edit. Emu Edit enables free-form editing by following a set of instructions, covering a wide range of activities like local and global editing, background removal and addition, color and geometry modifications, and more.

Particularly noteworthy is the fact that Emu Edit prioritizes accurate changes, making certain that pixels that are irrelevant to the instructions remain unmodified.

One of the most extensive datasets of its sort, Meta’s Emu Edit is constructed on a database that includes 10 million synthetic samples, making it one of the largest datasets available. According to Meta, the model achieves state-of-the-art outcomes in both qualitative and quantitative evaluations for a variety of image editing tasks, demonstrating superior performance in comparison to the methods that are currently in use.

Although Meta emphasizes that the work being done right now is fundamental research, many different use cases could be possible. Emu Video and Emu Edit are two of the technologies that could make it possible for people to create animated stickers and GIFs, as well as improve the content they share on social media, without having to have sophisticated technical knowledge.

“Emu Video, Emu Edit, and other new technologies like them could help people express themselves in new ways. This could range from an art director coming up with an idea for a new concept to a creator spicing up their most recent reel to a best friend exchanging a one-of-a-kind birthday greeting. And we believe that is something that should be celebrated,” stated Meta in a statement.