Nvidia Unveils AI Model for Audio Generation

Nvidia has unveiled an AI model designed for creating music and audio, capable of altering voices and generating new sounds.

The technology, known as Fugatto, is aimed at creators of music, films, and video games.

The neural network can generate sound effects and music based on prompts. For instance, it can create “audio of a trumpet barking like a dog.” Another example is the sound of “deep, rumbling bass pulses combined with periodic high-frequency digital chirps — akin to the sound of a giant intelligent machine awakening.”

A distinctive feature of Nvidia’s solution is its ability to analyze and modify existing sound. For example, it can transform a melody played on a piano into human singing.

“If we think about synthetic audio over the last 50 years, music sounds different now thanks to computers and synthesizers. I believe generative AI will bring new possibilities to music, video games, and ordinary people who want to create something new,” commented Bryan Catanzaro, Vice President of Applied Deep Learning Research at Nvidia.

The new model is trained on a dataset from open sources. The company is considering ways to present it to the public.

“Any generative technology always carries some risks because people might use it to create things we would prefer they didn’t,” Catanzaro emphasized.

Earlier, Google DeepMind announced the development of an AI-based technology for creating video soundtracks.