nielsinho

Unity is integrating a similar but more production-ready tool, Muse Animate, with their Muse platform. It does text-to-animation, and can fill in motion if you just provide keyframes. See usage here: https://youtu.be/tMCPz_yI7pY?si=OXAyxgGxHbCDywHm

I stumbled across ScriptBots when I was a kid, and became obsessed with implementing my own variants, and subsequently with neural networks. This one [1] is close to Karpathy's original, and this one [2] turns the world 3D and the beings are slightly more advanced. Both are implemented in MoonScript/Lua and run with Love2D. In part due to this this YouTube video, I work as a Machine Learning Engineer today.

[1] https://github.com/nilq/genesis [2] https://github.com/nilq/plane

TorToiSe is composed of many large models: GPT-2 for text encodings, as well as a large VQVAE encoder + large diffusion model decoder.

Only the big spaghetti inference code (+ weights) has been published, so there's a high entrance barrier for re-training / improving it.

TorToiSe (https://github.com/neonbjb/tortoise-tts) produces the best quality speech of any freely available model. However, its long inference times makes it impractical for voice chatbots like Gdansk.

Tissemand.

It happens quite often with TorToiSe that it collapses in this way. Especially for unseen tokens that wouldn't have appeared in the training data, which likely consisted of a lot of transcribed stuff and read text like audio books. Trying to make it laugh by prompting it with "hahaha" (which you won't really see in mentioned data) often gets you demon and zombie noises.

It uses the TorToiSe TTS model for generation. It's simple to generate conditioning voice latents using short audio samples. Likely transcribed JRE episodes were part of the TorToiSe training data, explaining how it's so good at recreating his voice characteristics in particular.

The book "Jellyfish Age Backwards" is very good and covers as well as sorts modern science in anti-aging and health. Definitely recommend this book.

You can take a look here: https://youtu.be/27PYlj-qNb0

Are you sure it's Gmail sending you those advertisements?