Deepfaking Reality in Real Time

Synthesised simulacra

New AI-based video compression techniques enable a 1000x reduction in data usage for approximately the same quality.

Instead of sending every frame, the sender transmits one image of the speaker plus a stream of facial key points, and AI at the other end redraws the face to match. What arrives is, in effect, a deepfake of the person you are talking to, made in real time.

That means radical savings in bandwidth and storage costs for streaming services, lower barriers to entry for competitors, and a lifeline for those forced to host their own content due to censorship.

YouTube was not making a profit for Google as of 2015 due to enormous bandwidth costs, and it likely still is not. If the economics shift enough that YouTube stops abusing monetisation in asinine ways, creators could see real benefit.

The forced digital transition of the past year has put global networks under strain. YouTube and Netflix alone account for roughly 8.7 per cent and 12.6 per cent of global bandwidth usage respectively. Better compression gives us extra capacity and lower latency, especially on expensive intercontinental backbones.

The technique also opens new markets for streaming in remote and developing regions, for users on EDGE/3G or expensive metered connections, for peer-to-peer calls on Zoom and Skype, and for streaming gaming platforms like Stadia, where the brains of your console live somewhere in the cloud.

It also enables high fidelity remote experiences and avatar piloting even without stable 5G connections (chatting online is bad enough when it breaks, but losing connection whilst operating in a whole other environment could be catastrophic).

Similar techniques such as DLSS (Deep Learning Super Sampling) are also revolutionising gaming, enabling one to run a game with all the fancy graphical features but at a tiny resolution (truly tiny, equivalent to a NES or C64), and using AI techniques to dynamically upscale to high definition. It’s now computationally cheaper to do AI upscaling than to run at the output resolution.

AI is not a cornucopia, but it is a powerful force multiplier, already enabling us to do a great deal more with fewer resources.