OpenAI is building a music generator — and it’s getting help from Juilliard students

OpenAI is developing its own artificial intelligence (AI) music generator capable of creating audio from text and audio prompts. The company is enlisting students from the prestigious Juilliard School to help train the system.

The Juilliard students were to annotate musical scores, according to a report by The Information published October 24. This annotated data will teach the AI to understand musical structure, tone, and rhythm.

The tool could allow users to add guitar accompaniment to existing vocal tracks or generate original soundtracks for videos. One source familiar with the project told The Information that ad agencies might use it to brainstorm lyrics, create jingles from music samples, or match video styles.

OpenAI has not confirmed when it plans to launch the tool. The company also hasn’t revealed whether it will operate as a standalone app or integrate into existing products like ChatGPT or the video-generation tool Sora. OpenAI declined to comment on the reports.

OpenAI will soon be in direct competition with other players in generative music. Google launched the second generation of its Lyria model in May, which customers can access via Google Cloud for tasks such as creating ad soundtracks. Startup Suno, which sells subscriptions to an AI music generator, reportedly generates around $150 million in annual recurring revenue—nearly four times higher than a year ago.

This isn’t OpenAI’s first venture into music generation. The company developed two earlier models, MuseNet in 2019 and Jukebox in 2020, but neither is currently available through ChatGPT. More recently, OpenAI has focused its audio work on text-to-speech and speech-to-text capabilities.

Copyright concerns loom over AI-generated music. The Recording Industry Association of America has sued competitors Suno and Udio, alleging they trained their models on copyrighted songs without permission. The labels are seeking up to $150,000 per infringed song, potentially totaling billions in damages.

Both Suno and Udio claim their use of copyrighted material falls under the fair use doctrine. Universal Music and Warner Music have been negotiating AI licensing deals with these companies and Google.

OpenAI takes some precautions with copyrighted material. ChatGPT doesn’t share full lyrics of certain songs, offering to summarize them instead. After launching Sora, Altman said the company would give copyright holders “more granular control” over character generation and share revenue from video generation with rights holders.