It's a really interesting technical challenge. On the one hand, changing your timbre to another human voice is much more complicated than basic pitch shifting, so we ended up using deep neural networks for a kind of simultaneous speech-recognition and speech-synthesis approach (though training this system to preserve e.g. the emotional complexity of your input speech while still changing out the timbre convincingly is difficult - we use adversarial training on the raw audio waveform, which is powerful but also pretty much unknown territory compared to images).
On the other hand, it's important to run with very low latency on your device while you're playing, which means that we can't simply "throw the biggest network we can" at the problem. So we have a tradeoff between model latency, which is easy to characterize, and audio quality / voice skin plausibility, which is pretty ambiguous and subjective.
Finally, as this kind of tech improves the potential for misuse becomes an important problem, so we need to build in protections (like watermarking the audio) that can help prevent fraud while not compromising the speed of the algorithm or the quality of the output audio.
Legitamate question: (absolutely not rhetorical, I don't know the answer!)
What the potential misuse of a gender voice changer? If someone wants to change their perceived gender in a video game (presumably to avoid harassment), should that be detectable? If both genders should ideally receive identical treatment anyway, is there any harm in swapping?
That said, there _are_ some interesting things to be careful around, even for changing your gender, or age, or other basic variables around your voice. We're mostly worried about what impact this would have for communities built around these kinds of commonalities: for example, is it okay for a child to masquerade as an adult in an adults-only social group? I don't think there's a clear answer to all of those situations - but until we see more use of realistic voice skins in the real world, we're playing it safe and building in these kinds of tools!
Faceapp comes to mind[1]. There's also Artistry Virtual Beauty[2] but it is a bit less advanced. I believe that Facebook Messenger also have some similar features.
> A low gruff voice does not bode well coming from a feminine avatar. Catches too many off guard.
A lot of sound cards those days come with tools that allow you to modify your audio input. Often, they'll have a female settings where it changes the pitch to make your voice sound more like a woman's.
[1] https://play.google.com/store/apps/details?id=io.faceapp&hl=...
[2] https://play.google.com/store/apps/details?id=com.perfectcor...
I remember back in the VRML/SecondLife heyday where you could pick an avatar that was another gender. I still do that. But the emphasis for voice chat in modern games and MMORPGs make it far more difficult. A low gruff voice does not bode well coming from a feminine avatar. Catches too many off guard.