I run a small food channel called Wait Until You Try. Every recipe gets a short vertical film and its own song, written for that one dish. The visuals and the music are AI-generated (every page says so), and each recipe is written up properly on the site, with grams and temperatures.

A couple of months in, we had seventy-odd songs and no way to sing along to any of them. So we built a karaoke page where each word lights up as it's sung: waituntilyoutry.com/karaoke.

This isn't a tutorial. The way we do it is our own, and I'm keeping the recipe in the kitchen, the same way a good pizzeria doesn't hand out its dough. But the problems turned out to be more interesting than I expected, and most of them apply to anyone who wants text to follow sound.

You can't look these songs up

With a famous song, synced lyrics already exist somewhere. Somebody has timed them by hand, and you can license or download them.

Our songs are a day old when they go live. Nobody has ever sung them except the track itself. So there's no database to borrow from, and timing seventy songs by hand, word by word, isn't something I wanted to do more than once. The timing had to come out of the audio.

A band is very good at hiding a singer

The obvious first try is to point a speech recognizer at the finished song. It goes wrong in two predictable ways.

Long instrumental intros were the first. Fifteen seconds of guitar, and the transcript would sometimes decide the first verse never happened.

Solos were the second. During a break with no vocals at all, it would hear words anyway, and report them with enough confidence that they were hard to filter out.

Same cause both times. To a system trained on speech, drums and pads look like a person talking very badly. The fix was to stop asking it to listen to the whole band at once. I'll leave it there.

Trust the lyric sheet, not the ears

Even on a clean recording, sung words get misheard. Vowels stretch, consonants disappear, and one held note can come back as two words. But we already know the lyrics, because they're written before the song exists.

So we never show what the machine thinks it heard. The words always come from our lyric sheet. The audio is only there to answer one question: when.

Choruses are where this gets tricky. The same line comes back three or four times, and any matching approach will happily attach a line to the wrong chorus. We ended up with a few sanity checks that catch the impossible cases, like a single sung line that supposedly stretches across half the song. Getting those checks right took longer than everything else put together.

Lessons from the player itself

The timing data is half the job. The other half is making it feel right on screen, and that part was all trial and error.

Exactly on time feels late. When a word lit up at its measured start, people said the lyrics were lagging. Your ear decides a word has started earlier than any measurement does, on the first bit of consonant. Lighting each word a touch early fixed the complaints.

Don't let the text jump. Our first version sized each line to fit. Short lines got huge, long lines got tiny, and the stage pulsed between them. Now each song picks one size and keeps it.

Not every word deserves fireworks. We tried effects on every loud word and it looked like a slot machine. Now they only go off on the big chorus moments. Fewer effects made the good ones land harder.

Let the singer lead. Words get a little extra weight when the vocal leans on them. People notice that one quickly without being able to say why.

Where it stands

All 72 songs are synced. Each one ships as a small static file that the browser loads when you pick the track, so nothing heavy happens while you're listening. A new song goes through the same process the day it comes out.

If you want to try it, start with the pizza one: Ninety Seconds of Naples Fire. The intro is long on purpose. There's a short page on how the films are made, and a set of plain text pages for AI agents at /for-agents, which probably deserves its own post.

I'm happy to talk about the problems in the comments. The code is staying in the kitchen for now.

Disclosure: the films and songs on the site are AI-generated, and this post was written with an AI assistant.