Re:DeepFaceLab: A tool that utilizes ML to replace faces in ...
by Oras on Tuesday September 03, 2019 @10:36AM
(#20866865)
Is it the same technology used by Zao? The example in the gallery does not look as good as the video here https://twitter.com/AllanXia/status/1168049059413643265
[ Parent
| on HN ]
Re:DeepFaceLab: A tool that utilizes ML to replace faces in ...
by throwawaywego on Tuesday September 03, 2019 @11:04AM
(#20867164)
I think this tech is too complex to run on mobile devices in 8 seconds for video transfer from 1 selfie.
What I think Zao does is preprocess the videos (manually or highly accurate facepoint detection). They pre-calculate the transforms (standard facemorphing algorithms with opacity/alpha tweaks), and shading depending on the scene lighting. Then they just need a good frontal selfie, or do some frontalization, and keypoint detection, and the rest can be rendered/computed without much resources, following a pre-defined script.
If more advanced than facemorphing, then perhaps something more like: https://github.com/facebookresearch/supervision-by-registrat... (pre-fitting a 3D face mask, then texturing it with your selfie)
Re:DeepFaceLab: A tool that utilizes ML to replace faces in ...
by JoblessWonder on Tuesday September 03, 2019 @02:47PM
(#20869788)
Yeah, I think the big question is if Zao only allows pre-selected "scenes" that they have already done the processing on or if they allow you to upload any video.
From the results, I think you are exactly right in how they are accomplishing those videos.