Learning dynamic 3D geometry and texture for video face swapping

Re:Learning dynamic 3D geometry and texture for video face s... by yorwba on Friday October 14, 2022 @06:31AM (#33201721)

There's little point in rendering normals computed from a triangle mesh to a normal map. The point of using textures is that they enable greater detail without having to add many vertices. With the right normal map, you can make a flat triangle look curved or bumpy.

Computing a normal map from video that still looks good if you change the lighting is a bit more difficult.

[ Parent | on HN ]

Re:Learning dynamic 3D geometry and texture for video face s... by dorkwood on Friday October 14, 2022 @09:13AM (#33203073)

> There's little point in rendering normals computed from a triangle mesh to a normal map.

I disagree. It's a common workflow that I've used many times. The idea is you bake the normals from a high poly model with millions of verts and then apply the texture to a low poly model with matching uvs.

> Computing a normal map from video that still looks good if you change the lighting is a bit more difficult.

Yes, I agree. But it's also difficult to generate a clean mesh, and we can see that they've succeeded in doing that. The point I tried (and failed) to make is that if they can generate a mesh like that from a video, then normals shouldn't be too much of a stretch.

[ Parent | on HN ]