News for hackers. Stuff that matters.
Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by XorNot on Thursday June 04, 2020 @09:31PM (#23424116)
So this sort of thing seems like a big deal for indie video game devs.

One of the big problems at the moment is that photorealistic texturing basically requires finding and mapping a real life actor, which is all sorts of expensive.

This looks like a way to get past that in a super-efficient way: just mock up what "look" you're going for with a character, hit render and out pops all the data needed for a texture (I suspect we're not too many iterations way from also getting bodies, meshes and animation skeletons).

This would make it a huge field-leveller for catching up for HD content production.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by erikbye on Friday June 05, 2020 @03:25AM (#23425930)
> One of the big problems at the moment is that photorealistic texturing basically requires finding and mapping a real life actor, which is all sorts of expensive.

As a hobby game dev myself I find this to not be a problem at all. I think human skin is largely a solved problem. It largely boils down to creating a proper material/shader, the material will have many layers of textures. These days, with the proper tools, this is easy, and generally more of an artistic affair than technical.

In fact, I will claim that texturing for games, in general, is largely a solved problem.

Look into both Substance suite and Quixel's offerings, watch/read tutorials. Both Algorithmic and Quixel revolutionized texturing and made AAA quality possible for indies, just like Unreal Engine made AAA quality available for indies.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by dwighttk on Friday June 05, 2020 @06:58AM (#23427166)
> I think human skin is largely a solved problem.

Can you post an example? All the fake skin I’ve seen is still marching into the uncanny valley (video at least)

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by erikbye on Friday June 05, 2020 @08:02AM (#23427514)
To clarify, largely do not mean completely solved, right? It's not perfect, yet. Either way, I should have qualified by saying I think it's a largely solved problem as far as the technical implementation.

What an artist does with that implementation is always another matter, and it will always be down to the artist to create a pleasing final result. Specularity is one thing I think artists gets wrong all the time. At the artistic level, games often deliberately go for a more cartoony look, too.

And the uncanny valley thing is very subjective, no?

https://docs.unrealengine.com/en-US/Resources/Showcases/Digi...

I think this is more than good enough for real-time games. Of course, you will probably say, "no that sucks, uncanny valley!"

One thing to notice here is that the albedo (diffuse) map (texture) used can be very crude/simple, as it is simply one small piece of the puzzle.

If you have looked at a video, or even screenshots, like the digital human ones, where the focus is solely on the subject, often close-up, that is very different to see it running in a game, where you are less likely to notice small faults.

These days, I think facial rigs and animation is the cause of uncanny valley, to a higher degree than skin shaders.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by dwighttk on Saturday June 06, 2020 @09:27AM (#23439029)
Those do look a lot better than what I'd seen.

Skin (and other stuff) still looks pretty weird to me.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by fxtentacle on Friday June 05, 2020 @02:48AM (#23425749)
There's a huge copyright issue waiting for you when you use this kind of tech outside of research.

The way how these AIs work is that they have memorized aspects of celebrity photos and then recombine them as needed. That means if your sketch looks in any way like a celebrity from the training data set, then the AI will likely reuse parts of those photos, which would make your generated texture a derived work, meaning you'd have to pay royalties to the celebrities in the dataset.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by londons_explore on Friday June 05, 2020 @04:22AM (#23426231)
No court has yet decided on this.

Previous research has shown that if you search the entire training dataset for the image most similar to one generated by a generative model like this, the images actually look pretty visually different. To a non-expert, they'd say the images are similar, but neither copied the other.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by visarga on Friday June 05, 2020 @09:50AM (#23428492)
> the AI will likely reuse parts of those photos, which would make your generated texture a derived work

There are two inputs here: a collection of photos and a model. It's not being derived solely from the original photos. The scientist had a contribution as well, which cost him money and time to develop. As long as it's not wholesale copying and there is significant creative input (selection, changes, recombinations, etc) then I think it should be ok.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by sillysaurusx on Friday June 05, 2020 @03:51AM (#23426060)
That's an open question, and big companies will face it before us smaller fish. BigCos are quite incentivized to win that fight, too.

In legal terms, if your AI model is "sufficiently transformative," you don't need to worry about anything like that. But if your AI model is overfitted and is just memorizing, then yes, you're right.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by idontevengohere on Thursday June 04, 2020 @10:50PM (#23424540)
What sort of data is needed for a texture?
Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by MrLeap on Thursday June 04, 2020 @11:26PM (#23424797)
I'm not in the triple A game design industry, more like some good grade, game-ish design industry, so what I have to say may not apply to the state of the art.

Generally, a material is made up of _several_ textures that serve different purposes. Common textures (often called 'maps') used to build materials are albedo maps, surface normal maps, metallic/roughness maps, subsurface scattering maps (this one is important for realistic looking human skin!). There's others, and sometimes aesthetics/requirements require shader authors to _make up maps_.

Consider a texture map just like, a precomputed data cache. You can encode pretty much anything in them. Why, in the gritty gore and carrion filled trenches, I've created systems where artists can use maps to annotate parts of models they think suck. That map was used in a camera AI system that tried to avoid looking at parts artists are ashamed of, adjust depth of field threshholds depending on the depth of bad stuff... That kind of thing. (That texture seam is too gross for the sizzle cinematic, there's NO TIME to fix it. Just smudge the camera, or have it lose focus as it sweeps through! shameMap.png to the rescue).

The limit to the kind of data you can _use_ is only your imaaaagination..

For people, for most aesthetics, at minimum you need a color or diffuse or albedo map.

Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by sillysaurusx on Friday June 05, 2020 @12:12AM (#23425003)
Love the shameMap idea!
Re:DeepFaceDrawing Generates Photorealistic Portraits from F... by MrLeap on Friday June 05, 2020 @12:27AM (#23425074)
Feel free to use the technique. I only hope that you'll also call it shameMap, because it's funny.