News for hackers. Stuff that matters.
Re:Is Word Error Rate a Good Metric for Speech Recognition M... by lunixbochs on Friday September 10, 2021 @10:19AM (#28481918)
You could weigh insertions by how much perplexity they add (sum), deletions by how much perplexity they remove (-sum), and replacements by how big the ppl difference is in the replaced word (abs(sum)). And report this as a 4-part score (combined mean, then separate i/d/r). Lower is better.

Theory being you don't want to add or remove confusing words, but common stop words are less of an issue.

I'm not sure how this interacts with a multi word replacement, where the new words together make sense but independently make no sense to the LM.

There are no comments on this comment yet.