Adobe’s AI Now Generates Soundtracks and Voice-Overs for Videos
▼ Summary
– Adobe is introducing Generate Soundtrack and Generate Speech AI tools in its redesigned Firefly app to help filmmakers add backing tracks and narration to videos.
– Generate Soundtrack analyzes uploaded videos and creates synchronized instrumental audio clips, with users able to select music styles or describe desired vibes via text prompts.
– Adobe’s AI tools are trained on licensed content, ensuring they are commercially safe and avoiding copyright infringement risks faced by competitors like Suno and Udio.
– Generate Speech allows users to create voiceovers from text with over 50 voices in 20+ languages, offering customization options for speed, pitch, and emotion.
– Adobe is developing a Firefly video editor with a multitrack timeline that integrates AI features for generating and editing clips, set to launch in private beta next month.
Adobe is introducing powerful new generative AI audio capabilities designed specifically for video creators. These innovative tools allow filmmakers to automatically generate fitting background music and professional voiceovers, streamlining the production process significantly. The features are launching within a redesigned Adobe Firefly AI application, alongside development of a comprehensive web-based video editor that integrates multiple AI functions into a straightforward editing workflow.
The Generate Soundtrack tool, now available in public beta, analyzes uploaded video footage and produces several synchronized instrumental audio clips. Creators can guide the musical style by choosing from preset genres such as lofi, hip-hop, classical, or EDM. Alternatively, they can type descriptive text prompts requesting a specific emotional tone, perhaps more sentimental or aggressive. The system can even suggest an initial prompt after reviewing the video content, helping users who are new to describing music.
Alexandru Costin, Adobe’s generative AI lead, explained the approach aims to build a new creative skill. The company wants to simplify music prompting, offering both an automated suggestion based on the clip and an interactive “Mad Libs” method where users select the vibe, style, and objective for their project. Each prompt generates up to four unique audio variations, with clips lasting a maximum of five minutes.
A crucial advantage is the commercial safety of the generated audio. The underlying Firefly AI model was trained exclusively on licensed content purchased from intellectual property owners. This careful approach significantly reduces the risk of copyright strikes that could lead to video removals. This positions Adobe favorably against some AI music startups currently facing lawsuits for training their models on protected material without permission. While Generate Soundtrack currently focuses on background scores, Adobe is developing additional tools for a more complete AI-assisted music production suite.
Simultaneously launching in public beta is the Generate Speech tool for creating voiceovers from text. It offers over fifty unique voices powered by either Adobe’s own Firefly Speech Model or technology from ElevenLabs, supporting more than twenty different languages. Users maintain precise control, adjusting speech speed, pitch, and emotional delivery. The tool also allows for manual pronunciation corrections, which is particularly useful for proper names or regionally specific terms.
Looking further ahead, Adobe is building a dedicated Firefly video editor. Described as a multitrack timeline editor for generating, organizing, trimming, and sequencing clips, it will combine the company’s AI tools for soundtracks, voiceovers, and titles into one web-based application. The editor will include frame-by-frame editing capabilities and various style presets, creating a centralized hub for AI-enhanced video creation. A private beta for the Firefly video editor is scheduled to begin next month, with interested users able to join a waitlist for early access.
(Source: The Verge)
