I'll probably throw away the waveform. I'm just looking for ways to use the audio to pace the text output, and also to tie the speech and text together as much as possible, so there's never a gap between them for your focus to jump between.
How are you synchronizing to each word?
I wonder if Google would consider expanding their TTS a bit to provide syllable data and other rich breakdown information.
I like the waveform, btw.
I'm using Polly for the moment, and it provides word and viseme boundaries. Enough to animate face poses. I'm not taking full advantage of the visemes here: it's only affecting the saturation of the waveform color. Too subtle to notice, with the way things kind of flicker with this one.
I have other ideas I want to play with, but don't really have time, and I may just have to drop that line of thinking, because I plan to move to one of the wavenet implementations, which don't provide speech marks data.
( edit: oh, with dark and light themes! )
Fuck it. Slater is magic software...
and it's interface is like a tiling 'window manager' for any number of screens (anything with a browser), for threads of interaction, that are modeled as a story, like a conversation... but composes natural language with rich controls, forms, and stateful widgets. and super monochrome... but also controls your browser and TV and stuff...
I'm really inspired by the design work on the LightPhone 2, btw.
Super lame, early video: https://youtu.be/_m1zEbQieYQ