Adobe showed off the capabilities of its upcoming artificial intelligence tools for working with video, including the ability to create videos from still images. The Firefly family model is responsible for these functions, which will be integrated into Adobe Creative Cloud applications and will be able to edit audio and video.
Image source: youtube.com/@AdobeVideo
The commercial shows footage created using the Firefly family of AI, which generates videos based on text descriptions – back in April, Adobe announced that such a set of functions would appear, but the company has only demonstrated their capabilities now. It will be possible to generate a video based on the description and adjust the results using “camera controls” that simulate changes in the viewing angle, its movement and the distance from which it is filmed. There is a function for generating a video from a static image, which is set as a sample; You can also generate intermediate frames if a gap is created during video editing.
The video quality in the demo is on par with OpenAI’s Sora model, which Adobe may also add to Premiere Pro’s video editor as a third-party module. But in its current incarnation, Firefly’s capabilities are still limited – the model generates video up to five seconds long. An important advantage of Adobe’s model is that the materials it creates are “commercially safe.” This means that the AI was trained on licensed and publicly available materials, as well as content from the Adobe Stock collection – the company assures that the video created in this way can be used unrestrictedly for commercial purposes.
Functions for generating videos from a text description or a static image will appear in the beta version of the Firefly app this year. Designed for video, the Firefly family will be integrated into Creative Cloud, Experience Cloud and Adobe Express applications. The company also showed off several videos created using the Generative Extend feature, which extends the length of existing video footage, similar to Photoshop’s Generative Expand feature for preparing background images. Adobe did not specify when the new features will become available to users, but promised to release them this year.