Thanks for the explanation. I had some assumptions but wasn’t totally sure how this was trained.

How would you make it sniffle in a natural way, too? It’s not a usual speech mannerism, and the way he does it is distinct. I wouldn’t know how to efficiently represent it with text. Maybe it’s easier than I’m imagining.

Just stop the audio output every 5 seconds and include a sniffle sounds, at least that's what it sounds like in real life haha

The TTS model is trained on two things: speech samples and their transcript. If you add enough sniffle-symbols every time a sniffle appears in the speech, I am confident the model would pick up on that. And then you would be able to replicate a sniffle in the generation part. The more time-consuming bit would be to add in the training data for the language model those sniffle-symbols, so that they would be organically added in the text in the text-generation phase.

But seriously, it's not worth it. I think he's a brilliant man with an idiosyncratic speech, let's leave it to that.

I agree, I personally don't hear his sniffles when I'm listening to him intently. It's irrelevant. I was mostly curious if and how, generally speaking, a model could be trained to sniffle. Now that you describe it though it seems fairly clear, so thanks!

in all seriousness how do you not hear them?

I assumed it would difficult to include, it was just something I noticed about him