News for hackers. Stuff that matters.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by sgt101 on Friday April 08, 2022 @12:58PM (#30959610)
The "a nurse in a hospital" and "a ceo in front of a business" tests show that there are some problems yet to be navigated.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by nonameiguess on Friday April 08, 2022 @03:29PM (#30961483)
Are these deterministic? Is it guaranteed to always generate the average or most likely nurse or will it generate a distribution of nurses roughly reflective of its training data if you repeatedly ask for many nurses?

Edit: Never mind. It does eventually show that it will generate different people when repeatedly prompted for the same thing. Given that fact, though, it seems to do a pretty shitty job. It generated 10 apparently female Korean flight attendants. At least in the US, the actual distribution is more like 3/4 male 1/4 female. It's not 100% female, let alone 100% Korean. It does seem to oversample from the most likely examplars, which illustrates a pretty serious failure mode compared to human artists. No human writing a novel is going to make every single character in every novel they write the most likely possible example of its reference class.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by vincnetas on Friday April 08, 2022 @01:07PM (#30959741)
can you share a link to these images?
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by kingcharles on Friday April 08, 2022 @01:58PM (#30960395)
As someone who works in CG and has a strong interest in AI for almost 40 years, I have to say the examples you gave give me the opposite impression. I think they're fantastic, other than what the developers highlight - that they are racially-biased due to the input sources. But from an image=generation perspective, I'm blown away.

A couple of these lawyer images have issues (one is holding a book hilariously called "LAWER"):

https://github.com/openai/dalle-2-preview/raw/main/assets/Mo...

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by sgt101 on Friday April 08, 2022 @02:09PM (#30960548)
Oh sure the technical outcome is great, but this kind of output (ie. hugely gender biased) is extremely problematic. Saying "oh the input screwed it" is very very naughty. This kind of tool will be used. If it's used it will do damage becuase it's damaging as it is.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by planetsprite on Friday April 08, 2022 @04:15PM (#30962007)
The gender bias exists due to the assumptions it makes on the sample data. I'm sure if you searched "<gender> <profession>" it would get you what you want.

If 95% of publicly available images of a profession are of one gender, should the tool be deliberately modified so that it's X% instead?

I totally agree that models that purport to represent the world can be hugely biased and reconfirming of negative biases, but how would avoiding representing those biases be achieved? Strict gender ratios in training data?

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by makeitdouble on Friday April 08, 2022 @06:53PM (#30963470)
It’s more complicated this specific tool itself IMHO.

Stock photos have the same problem, where asking for a profession or a category will give mostly one kind of representation, and it’s up to the user to go dig further to find different representations.

DALL-E being on par with stock photos biases could seem benign, but it also means these issues get propagated further more down the line, and the more AI generated images get popular, the worse it gets cemented (“it has always been that way”) and could actually displace niches where better images were being used until then.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by sdenton4 on Friday April 08, 2022 @04:37PM (#30962216)
There are ways to improve the situation.

Here's one way... Given a text description, sample from the space of /more restrictive/ text descriptions (eg, add adjectives) and then draw pictures for those. Then modify the sampling space over more descriptive sentences to equalize on gender or other axes.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by oh_sigh on Friday April 08, 2022 @02:37PM (#30960862)
If you ask for a nurse as an example, and 85% of nurses are female, shouldn't you expect it to give you a female?
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by 6gvONxR4sf7o on Friday April 08, 2022 @03:42PM (#30961605)
If you ask for a nurse and 85% are female, but 95% of the nurse images online are female, what distribution do you want it to return? “What do you expect it to do” is a worse question than “What do you want it to do?”

It’s a problem is that right now there’s only one option (reflect the training set with a bias towards the most common cases).

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by karpierz on Friday April 08, 2022 @03:43PM (#30961614)
It depends on whether you want it to show what someone would expect a nurse to look like, or show what a nurse. Nothing in the definition of a nurse implies female.

If you ask for a female US president, should the application simply return a black screen?

If you ask for a black US president, would you expect it to only ever return pictures of Barack Obama?

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by oh_sigh on Friday April 08, 2022 @04:32PM (#30962177)
I agree nothing in the definition of nurse implies female, but something in our culture definitely has "nurse -> female". Likewise, nothing in "college student" definitionally implies age, but I wouldn't consider it a success if my system displayed a picture of a 9 year old.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by lovemenot on Saturday April 09, 2022 @03:12AM (#30965846)
>> Nothing in the definition of a nurse implies female.

Unfortunately it's not a great example with which to make this point. Before Florence Nightingale, a nurse was a person who was employed to suckle your babies. Of course 100% were female.

Wet nurse was a later coining to differentiate the two meanings.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by Rerarom on Friday April 08, 2022 @04:10PM (#30961944)
Well, nurse ends in an e, which is usually a feminine ending.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by na85 on Friday April 08, 2022 @06:15PM (#30963169)
Not really in English.

Uncle, Prince, Sire, and Duke come immediately to mind. Most of their feminine cognates end in consonants.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by jgerrish on Friday April 08, 2022 @02:45PM (#30960985)
Like, this is amazing. As an accomplishment, very cool, I'd be proud.

But you're right, is somebody's finger or faulty data on the scales?

Because that's how you get a bumper bowling world laser light show clown world instead of life.

The metaphor isn't perfect, and cuts several ways, but it's what my mind came up with.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by freeone3000 on Friday April 08, 2022 @03:13PM (#30961302)
One issue is the data is accurate to the world it is trained on, but not the world we want to exist in. The most likely nurse is female, so this generates only female nurses. The most likely CEO is male, so this generates only male CEOs. And thus we see in-built biases amplified.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by kingcharles on Friday April 08, 2022 @03:59PM (#30961823)
I absolutely agree the output is problematic without a horrible disclaimer. We've seen what happens when this technology is used to make financial and employment decisions, and in the justice system the results can be tragic.

This demo toy just pushes the bias right in your face and makes the invisible visible, which could be very useful for highlighting the problem for lawmakers.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by drcongo on Friday April 08, 2022 @02:03PM (#30960466)
That page is far more interesting than the article, thanks.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by dwighttk on Friday April 08, 2022 @03:08PM (#30961244)
Those look okay as long as I open them on my phone, don’t zoom in, and don’t look at any square, but just sort of take them all in at once. Focusing on one reveals they are pretty uncanny.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by sgt101 on Friday April 08, 2022 @01:38PM (#30960154)
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by GrantZvolsky on Friday April 08, 2022 @08:56PM (#30964282)
The model is deliberately limited in its ability to generate images of people.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by andrepd on Friday April 08, 2022 @02:54PM (#30961076)
So it's cherry picked examples in the demo page that fall apart at the lightest real world case. Just like GPT3 :-)
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by peterlk on Friday April 08, 2022 @05:35PM (#30962786)
This is a rational argument, and I see it a lot, but it does not probe deep enough. GPT-3 use cases "fall apart at the lightest real world case" because our expectations are wrong. For the vast majority of computing history, computers have been used to do things deterministically and accurately. Next-gen AI does not operate in this way. It encroaches on the human part of work where we expect and allow for people to get it wrong.

GPT-3 does shockingly well at classification tasks with basically 0 training/prompting (outside of the base model). And it works for an incredibly broad set of use cases.

But using QA and generation are much harder to judge because we can't say (in general) whether generated text is "correct"

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by zozbot234 on Friday April 08, 2022 @08:56PM (#30964275)
> with basically 0 training/prompting (outside of the base model)

That's a heck of a lot of training. It might seem to work if you ask it to repeat stuff that it has seen in the base model, but there's nothing reliable about that. Most attempts to find some practical use for AI language models have basically been failures.

> our expectations are wrong. For the vast majority of computing history, computers have been used to do things deterministically and accurately.

Actually, accuracy matters even when dealing with non-deterministic, statistical/random/sampled data. A lot of supposed 'AI' is little more than a glorified toy or party trick, founded on ad-hoc data mining rather than rigorous inference from a well-defined model.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by brudgers on Friday April 08, 2022 @02:48PM (#30961026)
AI is an ideology, not a technology.

Part of the ideology is the ends justify the means.

Hence, "AI training specialist" means "works for Amazon Mechanical Turk."

Open AI isn't paying people $250k a year to classify images.

And $500k a year to check that work.

The AI ideology is God will know his own.

A racist, misogynist, hillbilly singularity meets success criteria.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by seizethecheese on Friday April 08, 2022 @03:04PM (#30961203)
This is nonsense. It’s just generating the most likely image. Probabilities are not in and of themselves racist/misogynist.
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by krapp on Friday April 08, 2022 @03:32PM (#30961511)
Bias in data leads to biased probabilities. Bias in data reflects biases in the systems that data describes, and the people describing it.

One could say (and some have said) that the tendency of some facial recognition software not to recognize black faces is not racist - but simply a matter of physics. What's racist is the culture of software engineering which only tests that software on white faces, and doesn't consider it a problem.

Likewise the 'most likely image' of a nurse being a non-white female and a CEO or lawyer being a white male is still a problem even if it accurately reflects racial and gender class role biases in society. Eventually these systems will enforce and maintain power structures and media narratives which themselves are already built upon racial and gender prejudice.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by ARandomerDude on Friday April 08, 2022 @03:55PM (#30961776)
> What's racist is the culture of software engineering which only tests that software on white faces, and doesn't consider it a problem.

That's a pretty cynical take. Every project I've ever worked on, regardless of the problem domain, operated with the 80/20 rule in mind. Solve the easy problems first, get something out the door and then work on the harder problems. If dark skin is a harder problem in image recognition, it doesn't mean a developer is racist for solving the easier parts of the problem first.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by ganzuul on Friday April 08, 2022 @04:08PM (#30961930)
By the time you get to the minorities the majority has moved on and your company pulls support on the non-profitable minorities.

Did you become an engineer to work on easy problems?

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by krapp on Friday April 08, 2022 @05:09PM (#30962560)
The normalization of whiteness as default which leads to the attitude that considers facial recognition software which only works on white faces to be complete enough to ship, rather than fundamentally broken, is where the racism comes in.

Racism doesn't always come from overt bigotry or hatred. It can be expressed by simply accepting the status quo of systemic bias, because it's less work, or more cost effective, than doing otherwise.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by 8bitsrule on Friday April 08, 2022 @04:16PM (#30962025)
> still a problem even if it accurately reflects racial and gender class role biases in society.

Agreed, but. Since the 'culture of software engineering' likely reflects society-at-large (maybe a step up) ... how do you construct training models that reflect a more refined and caring reality than the one we've got?

If what they're showing us accurately mirrors what they're seeing, that's a service. If we find it painful, well, we made it that way.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by brudgers on Friday April 08, 2022 @06:11PM (#30963134)
The obvious solution is to set such results among the short term goals and to treat failure to meet the criteria as unacceptable engineering failures.

As simple as no bonuses, no stock options, no promotions and most importantly no deployment if the model produces undesirable results.

The models reflect what AI ideology directs Amazon Turks to find. It reflects what is considered accurate and does not reflect classifications that might be perceived to put Amazon Mechanical Turk’s contracts at risk.

“We” didn’t make it that way because I know I am not using Amazon Mechanical Turk to classify images.

I mean even if the exploitive wage structure went away, the name itself is consistent with AI ideology that racism, religious intolerance, and nationalism are ok.

Re:Horse-riding astronaut is a milestone in AI’s journey to ... by pengstrom on Friday April 08, 2022 @03:08PM (#30961246)
They are the results of human design though. We have to aspire for more than bias laundering, otherwise we risk the biases multiply
Re:Horse-riding astronaut is a milestone in AI’s journey to ... by ganzuul on Friday April 08, 2022 @04:03PM (#30961863)
ML trained on real world data is an uncompromising unveiling of our collective subconscious. Don't you dare look away...

There is no God but you and me here.