One of the premises of the novel's universe is that computational power is generally absurdly plentiful, but communications bandwidth over interstellar distances is not. Most communications are in plain text (modeled after USENET) but in some cases, "evocations" are used to extrapolate video and audio from an ultra-compressed data stream.
The trouble, of course, is that it's not very obvious what aspects of the image you're seeing are real, and what aspects were dreamed up by the system doing the extrapolating.
It would be quite obvious unless the raw data before extrapolating is destroyed, for which there are no reason nor is it possible to stop others in the vincinity from receiving this raw data.
But saying more would be spoilery...
> WaveNetEQ is a generative model, based on DeepMind’s WaveRNN technology, that is trained using a large corpus of speech data to realistically continue short speech segments enabling it to fully synthesize the raw waveform of missing speech.
I don't think you need to train for each person specifically, you can just train a model for all heads, then maybe transmit a few high quality pics when the call starts, and interpolate from that afterward.
Downside could be some uncanny valley if the models are not very high quality.
But if I had to make a prediction, I'd expect we'll get much more value from higher bandwidth, ultra high definition streaming and features like 3d cameras / virtual reality. I think we have a tendency to really underestimate how important high definition is for human communication.
Low latency is probably more important to me.
Recently I seem to have a 3 second delay on many VC calls at work (and just for me it seems), and I either end up interrupting people or feeling reluctant to talk at all since it becomes impossible to time gaps and conversations right.
Despite that I get a crystal clear HD picture for all participants, but I'd happily sacrifice video quality (in fact I'd accept audio only in some cases) to get a more real time experience (disabling video doesn't seem to have any effect).
If you're really willing to sacrifice video completely, at least for Zoom, and probably for lots of other videoconferencing solutions, you can call into meetings with your phone. In fact, I think Zoom allows you to join with the computer for video and the phone for audio, which might be the best of both worlds.
Slight issue in Toronto has been cellular system overloading and calls not being completed. But once on, no problem.
I can’t blame the providers though. How could they have predicted that people would use the service they’re paying for?
That can be controlled, since these compression algorithms usually work by making a prediction and sending the difference between the prediction and the actual value.
That works both for lossless compression - where the difference is sent in full - and lossy as well - where only the most important part of the difference is sent.
https://www.nvidia.com/en-us/geforce/news/nvidia-dlss-2-0-a-...
To a human, for example, the exact pattern of every strand of hair isn't important at all -- all that matters is that the hairstyle and hair color stays the same.
The algorithm can also not worry about encoding and re-constructing skin blemishes because humans would possibly actually enjoy not having to put on makeup for a video call.
The MPEG-4 part 2 actually had something like that, called "face and body animation (FBA)". As far as I know, there are no implementations in widespread use.
https://people.xiph.org/~jm/demo/lpcnet_codec/
Sure enough, it was discussed on HN when it came out last year. I think I missed it then.
For those who didn't catch this from the URL, this is by Jean-Marc Valin, of Speex and Opus fame.
An utterly brilliant great idea!
Train a model locally ahead of time and upload it to a server, then whenever you have a call scheduled the model is downloaded in advance by the other participants.
Now, instead of having to send video data, you only have to send a representation of the facial movements so that the recipients can render it on their end. When the tech is a little further along, it should be possible to get good quality video using only a fraction of the bandwidth.