Learning dynamic 3D geometry and texture for video face swapping

Re:Learning dynamic 3D geometry and texture for video face s... by dorkwood on Friday October 14, 2022 @02:18AM (#33200198)

Generating normals is not particularly difficult once you have triangles. I believe the method is to take one corner of the triangle, get the two vectors pointing towards the two other corners, and then get the cross product of these two vectors. This gives you a new vector that is perpendicular to the triangle's face, i.e. the normal.

[ Parent | on HN ]

Re:Learning dynamic 3D geometry and texture for video face s... by J-_-_b on Friday October 14, 2022 @02:34AM (#33200272)

Normals is not the same as normal maps.

[ Parent | on HN ]

Re:Learning dynamic 3D geometry and texture for video face s... by dorkwood on Friday October 14, 2022 @05:29AM (#33201387)

You're right. One's a storage method for normals.

[ Parent | on HN ]

Re:Learning dynamic 3D geometry and texture for video face s... by virtualritz on Friday October 14, 2022 @05:13AM (#33201296)

You are confusing a per vertex information with a per texel one.

[ Parent | on HN ]

Re:Learning dynamic 3D geometry and texture for video face s... by dorkwood on Friday October 14, 2022 @05:34AM (#33201416)

No I just thought he was implying normals were hard to generate.

In your case you need a couple more steps: generate uvs and render to texture.

[ Parent | on HN ]

Re:Learning dynamic 3D geometry and texture for video face s... by yorwba on Friday October 14, 2022 @06:31AM (#33201721)

There's little point in rendering normals computed from a triangle mesh to a normal map. The point of using textures is that they enable greater detail without having to add many vertices. With the right normal map, you can make a flat triangle look curved or bumpy.

Computing a normal map from video that still looks good if you change the lighting is a bit more difficult.

[ Parent | on HN ]

Re:Learning dynamic 3D geometry and texture for video face s... by dorkwood on Friday October 14, 2022 @09:13AM (#33203073)

> There's little point in rendering normals computed from a triangle mesh to a normal map.

I disagree. It's a common workflow that I've used many times. The idea is you bake the normals from a high poly model with millions of verts and then apply the texture to a low poly model with matching uvs.

> Computing a normal map from video that still looks good if you change the lighting is a bit more difficult.

Yes, I agree. But it's also difficult to generate a clean mesh, and we can see that they've succeeded in doing that. The point I tried (and failed) to make is that if they can generate a mesh like that from a video, then normals shouldn't be too much of a stretch.

[ Parent | on HN ]