News for hackers. Stuff that matters.
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by amelius on Wednesday March 27, 2024 @04:37AM (#39836744)
Can GANs prevent the ">10 fingers" problem?
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by samus on Wednesday March 27, 2024 @07:18AM (#39837652)
A lot of the problems of Stable Diffusion arise because of using a text encoder that isn't up to the task to encode the meaning sufficiently precise. For example, if you mention the word "green" in the prompt, it often carries over to the whole image instead of the object that is being described. Numbers suffer from a similar problem. Rendering text is compromised by tokenization.

One problem with hands might be that they are comparatively small. The model easily gets the big picture right (head, arms, legs), but hands are so-called high-frequency details and are additionally featured in lots of different positions, which are seldom sufficiently described in the captions of the training data.

Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by fzliu on Wednesday March 27, 2024 @05:08AM (#39836918)
The >10 fingers problem should be solved via higher quality data or conditioning rather than a better model architecture.
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by onion2k on Wednesday March 27, 2024 @05:28AM (#39837016)
Are you suggesting AI generates images of people with more than 10 fingers because there are too many pictures of people with more than 10 fingers in the input data? That seems unlikely..
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by notresidenter on Wednesday March 27, 2024 @07:06AM (#39837591)
People holding hands, people high-fiving, etc.
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by amelius on Wednesday March 27, 2024 @08:20AM (#39838075)
Yes, but if AI cannot solve the fingers problem, then can it reliably generate images of things which we have relatively few example images of? We have an enormous amount of images of hands.
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by Drakim on Wednesday March 27, 2024 @05:49AM (#39837128)
In some pictures not all 5 fingers on a hand are visible, so maybe it appears that humans have a variable amount of fingers?
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by josefx on Wednesday March 27, 2024 @06:04AM (#39837230)
Humans with less than 5 fingers per hand do indeed exist. How does that lead to a default of 7?
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by Drakim on Wednesday March 27, 2024 @07:04AM (#39837581)
I meant more because people are holding objects or their hand is just at an angle where all fingers are not visible in the picture.

I don't understand how neutral networks operate, but my layman's guess is that when you sometimes see hands with 5 fingers visible, sometimes 4, sometimes 3, sometimes 2, sometimes 1, and sometimes 0, then it's not immediately apparent that it means that every hand has between 5 and 0 fingers.

Think of it this way, if the AI has ever only seen houses with a maximum of 10 windows in it's entire training set, is it so unthinkable that it sometimes draws a house with 12 windows? that's a "sensible" understanding about how houses have a variable amount of windows. It just doesn't work for fingers.

I'm sure the same issue would arise if humans had other body parts that came in large quantities, but almost everything else is either 1 or 2 like the nose or eyes.

Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by itishappy on Wednesday March 27, 2024 @04:10PM (#39843983)
Counting fingers is how humans do it, not necessarily how AI does. On a five-fingered hand, fingers are more likely to have neighbors than not. Why wouldn't the default be infinite fingers?
Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by jobigoud on Wednesday March 27, 2024 @12:07PM (#39841050)
Higher quality could also mean the description attached to the original images in the dataset. If you were to describe a picture of someone you would never specifically call out the fact that they have 5 fingers per hand, we take it for granted, so that kind of information may never appear in the dataset.

But I think what the grant parent means by conditioning is non-textual conditioning, like ControlNet. This will always be more powerful than trying to describe something by text. Think about the description of a character in a novel vs the movie adaptation.

Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by mdp2021 on Wednesday March 27, 2024 @06:00AM (#39837201)
Outside of the work to study and improve available technologies ("how far can we push a hammer"),

you do not draw an individual with anomalous hands because you have an ontological model in which "humans normally have five fingers per hand".

Knowing "how the world works" is the appropriate source for subsequent expression of a representation.

Somewhere in the architecture a world model should be formed.

Re:MIT Unveils Gen AI Tool That Generates High Res Images 30... by poulpy123 on Wednesday March 27, 2024 @07:02AM (#39837563)
it's exactly the reverse: the issues with generative AI isn't the data anymore, but the models that are not able to understand the data