Apple Introduces Apple Music Sing

Re:Apple Introduces Apple Music Sing by JohnBooty on Tuesday December 06, 2022 @11:37AM (#33882820)

     And "real-time lyrics" is _already_ in Apple Music.

Currently, Apple Music lyrics are synced to the current line of lyrics (if the lyrics metadata supports it)

This new feature appears to sync the lyrics to the current word of lyrics, which is what karaoke usually does. Not sure how big a deal that is to folks looking to do their own vocals.

I'm curious about how they're doing that word-for-word timing. Hardcoded metadata? ML? Is it a simple interpolation based on the existing line-by-line timing?

[ Parent | on HN ]

Re:Apple Introduces Apple Music Sing by hunter2_ on Wednesday December 07, 2022 @10:23AM (#33895372)

The metadata exists in special karaoke recordings, but assuming they're using original recordings not created/modified for karaoke, they'd have to create it on the fly:

I'd guess it's done using the same speech-to-text system used by voice assistants, which can certainly show the words it hears in near-realtime -- and way more quickly when it already knows what words it's listening for.

By the way, karaoke often highlights individual syllables, not just whole words.

[ Parent | on HN ]