News for hackers. Stuff that matters.
Re:DALL·E now available in beta by danielvf on Wednesday July 20, 2022 @01:19PM (#32169025)
I am thrilled about DALL-E, and the new terms of service. However, how they implemented the improved "diversity" is hilarious.

Turns out that they randomly, silently modify your prompt text to append words like "black male" or "female". See https://twitter.com/jd_pressman/status/1549523790060605440

I don't know which emotion I feel more - applause at how glorious this hack is or tears at how ugly it is.

Good luck to them!

Re:DALL·E now available in beta by causi on Wednesday July 20, 2022 @01:38PM (#32169383)
Interesting. Considering this is now a paid product, is modifying user input covered by their ToS? If I was spending a lot of money on it I'd be rather annoyed my input was being silently polluted.
Re:DALL·E now available in beta by tptacek on Wednesday July 20, 2022 @01:59PM (#32169723)
Your input isn't being polluted by this any more than it is when the tokens in it are ground up into vectors and transformed mathematically. You just have an easier time understanding this transformation.
Re:DALL·E now available in beta by throwuxiytayq on Wednesday July 20, 2022 @02:46PM (#32170388)
Obviously, it's polluted. Undisputably. In a mathematical sense, an extra (black box) transformation is performed on the input to the model. In a practical sense (eg. if you're researching the model), this is like having dirty laboratory tools - all measurements are slightly off. The presumption by OpenAI is that the measurements are off in the correct way.

I'm interested in using Dall-E commercially, but I think some competitor offering sampling with raw input will have a better chance at my wallet.

Re:DALL·E now available in beta by tptacek on Wednesday July 20, 2022 @03:13PM (#32170748)
It's a fucking AI picture generator. The whole thing is a series of (literally) inscrutable black boxes. This is not a good argument.
Re:DALL·E now available in beta by throwuxiytayq on Wednesday July 20, 2022 @03:34PM (#32171019)
Yeah man, but literally the entire point of this AI picture generator is that it's, like, super accurate at rendering the prompt, and stuff.

I don't understand the relevance of the black box's scrutability - I just want to play with the black box. I am interested in increasing my understanding of the black box, not of a trust-me-it's-great-our-intern-steve-made-it black box derivative.

Re:DALL·E now available in beta by tptacek on Wednesday July 20, 2022 @03:39PM (#32171085)
You should make your own black boxes then. By all means, send your dollars to whatever service passes your purity test; I'm just saying that the idea that DALL-E is "polluting" your input is risible. It's already polluting your data at, like, a subatomic level, at dimensionalities it hadn't even occurred to you to consider, and at enormous scale.
Re:DALL·E now available in beta by tablespoon on Thursday July 21, 2022 @09:48AM (#32179352)
> Your input isn't being polluted by this any more than it is when the tokens in it are ground up into vectors and transformed mathematically. You just have an easier time understanding this transformation.

These kinds of modifications are obviously different. At least the mathematical transformations are attempting at least some level of fidelity to user input, these ones aren't (e.g. someone mentioned they're sometimes getting androgynous results and speculates the added terms are conflicting with the ones they provided in their input). Not all black boxes are equivalent.

Re:DALL·E now available in beta by zikduruqe on Wednesday July 20, 2022 @01:46PM (#32169517)
Don't spend money. Use https://www.craiyon.com
Re:DALL·E now available in beta by kuprel on Wednesday July 20, 2022 @01:54PM (#32169643)
This one is faster, I ported it https://replicate.com/kuprel/min-dalle
Re:DALL·E now available in beta by minimaxir on Wednesday July 20, 2022 @02:14PM (#32169915)
Additionally, it's also open-sourced on GitHub and can be self-hosted, with easy instructions to do so: https://github.com/kuprel/min-dalle
Re:DALL·E now available in beta by MrBra on Saturday July 23, 2022 @12:48AM (#32200473)
I can run DALL-E mini with the MEGA model with 12 GB of VRAM. What are the requirements for minDALL-E and MEGA, in terms of VRAM?
Re:DALL·E now available in beta by bhaney on Wednesday July 20, 2022 @01:50PM (#32169574)
This produces dramatically worse results in my experience.
Re:DALL·E now available in beta by minimaxir on Wednesday July 20, 2022 @02:03PM (#32169757)
Not worse, but different. It depends on the prompt but DALL-E mini/mega seems to do better then DALL-E 2 for certain types of absurd prompts, such as the ones in /r/weirddalle
Re:DALL·E now available in beta by causi on Wednesday July 20, 2022 @02:05PM (#32169801)
Yes, there are very sharp lines where it does and doesn't understand. It understands color and gender but not materials. I got very good outputs for "blue female Master Chief" but "starship enterprise made out of candy" was complete garbage.
Re:DALL·E now available in beta by elcomet on Wednesday July 20, 2022 @02:14PM (#32169920)
Definitely worse-quality. Maybe more diverse for some prompts yeah.
Re:DALL·E now available in beta by scott_s on Wednesday July 20, 2022 @02:22PM (#32170022)
[shudder]

I tried the first whimsical, benign thing I could think of: "indiana jones eating spaghetti." The results are clearly recognizable as that. But they are also a kaleidoscope of body horror; a Indiana Jones monster melted into Cthulu forms inhaling plates that are slightly not spaghetti.

Re:DALL·E now available in beta by practice9 on Wednesday July 20, 2022 @02:16PM (#32169937)
Thankfully it doesn't introduce any researcher bias, doesn't ban people from using it on the basis of country, doesn't use your personal data like phone number...

And the best of all - it does have a meme community around it, and you can always donate if you feel it adds value to your life

Re:DALL·E now available in beta by kingkawn on Wednesday July 20, 2022 @02:08PM (#32169829)
The racist pollution came long before this product was a glimmer in our eye.
Re:DALL·E now available in beta by konfusinomicon on Wednesday July 20, 2022 @01:28PM (#32169206)
as far as I can tell, they also concatenate "On LSD" to every prompt as well.
Re:DALL·E now available in beta by tshaddox on Wednesday July 20, 2022 @01:53PM (#32169625)
That Twitter thread is full of people saying "yeah that doesn't seem to be true at all" so I'm hesitant to jump to conclusions even if we're deciding to believe random tweets.
Re:DALL·E now available in beta by time_to_smile on Wednesday July 20, 2022 @02:01PM (#32169739)
This is funny because I work on a team that is using GPT-3 and to fix a variety of issues we have with incorrect output we've just been having the engineering team prepend/append text to modify the query. As we encounter more problems the team keeps tacking on more text to the query.

This feels like a very hacky way to essentially reinvent programming badly.

My bet is that in a few years or so only a small cohort of engineering and product people will even remember Dall-E and GTP-3 and someone cringe at how we all thought this was going to be a big thing in the space.

There's are both really fascinating novelties, but at the end of the day that's all they are.

Re:DALL·E now available in beta by throwaway4aday on Wednesday July 20, 2022 @02:30PM (#32170160)
How else would you specify the type of image you would like? Surely, if you were hiring a designer you would provide them with a detailed description of what you wanted. More likely, you would spend a lot of time with them maybe even hours and who knows how many words. For design work specifically to create a first mockup or prototype of a product or image it seems like DALL-E beats that initial phase hands down. It's much easier to type in a description and then choose from a set of images than it is to go back and forth with someone who may take hours or days to create renderings of a few options. I don't think it'll put designers out of work but I do think they'll be using it regularly to boost their productivity.
Re:DALL·E now available in beta by selestify on Wednesday July 20, 2022 @02:54PM (#32170479)
What are you using GPT-3 for in a commercial setting?
Re:DALL·E now available in beta by tablespoon on Wednesday July 20, 2022 @01:22PM (#32169077)
> Turns out that they randomly, silently modify your prompt text to append words like "black male" or "female".

I wonder what the distribution of those modifications is?

Re:DALL·E now available in beta by Hard_Space on Wednesday July 20, 2022 @01:35PM (#32169326)
Today, when DALL-E was still free, my Dad asked me to try a prompt about the Buddha sitting by a river, contemplating. I did about 4 prompt variations, and one of them was an Asian female, if that gives any idea about the frequency (I should note that the depiction was of a young, slim, and attractive female Buddha, so I'm not sure they have the bias thing licked just yet).
Re:DALL·E now available in beta by speedgoose on Wednesday July 20, 2022 @01:27PM (#32169173)
In my little testing, diversity in ethnicities was achieved but not realistic given the context. I also got a few androgynous people as I asked for a male or a female and another gender was appended.
Re:DALL·E now available in beta by bantou_41 on Wednesday July 20, 2022 @01:54PM (#32169639)
Diversity = black now? That’s even more racist.
Re:DALL·E now available in beta by xyzzyz on Wednesday July 20, 2022 @02:05PM (#32169799)
Diversity has meant exactly that all the way since Bakke.
Re:DALL·E now available in beta by Invictus0 on Wednesday July 20, 2022 @01:35PM (#32169333)
A dumb solution to a dumber problem.
Re:DALL·E now available in beta by mysore on Wednesday July 20, 2022 @01:20PM (#32169051)
it's a hard problem. at least they tried.
Re:DALL·E now available in beta by Jerrrry on Wednesday July 20, 2022 @01:29PM (#32169223)
It's not a "problem," it's an unwanted shard of reality piercing through an ideological guise.
Re:DALL·E now available in beta by gnulinux on Wednesday July 20, 2022 @01:44PM (#32169483)
How's it NOT a problem? If I'm trying to produce "stock people images", and if it only gives me white men, it's clearly broken because when I ask for "people" I'm actually asking for "people". I'm having difficulty understanding how it can be considered to be working as intended, when it literally doesn't. Clearly, the software has substantial bias that gets in way of it accomplishing its task.

If I want to produce "animal images" but it only produces images of black cats, do you think there is any question whether it's a problem or not?

Re:DALL·E now available in beta by Jerrrry on Wednesday July 20, 2022 @02:04PM (#32169781)
That is clearly overfitting due to unrepresentative training data.

The "issue" is a different one: that training data - IE, reality, has _unwanted_ biases in it, because reality is biased.

Producing images of men when prompting for "trash collecting workers" should not be much of a surprise: 99% of garbage collection/refuse is handled by men. I doubt most will consider this a "problem," because of one's own bias, nobody cares about women being represented for a "shitty" job.

But ask for picture of CEOs, and then act surprised when most images are of white men? Only outrage, when proportionally, CEO's are, on average, white men.

The "problem" arises when we use these tools to make decisions and further affect society - it has the obvious issue of further entrenching stereotypical associations.

This is not that. Asking DALLE for a bunch of football players, would expectedly produce a huddled group of black men. No issue, because the NFL are disproportionately black men. No outrage, either.

Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.

The "problem" is with reality being used as training data. The "problem" is with our reality, not the tooling.

Except in the cases where these toolings are being used to affect society - the obvious example being insurance ML algorithms. et al - we should strive to fix the issues present in reality, not hide them with handicapped training data, and malformed inputs.

Re:DALL·E now available in beta by less_less on Wednesday July 20, 2022 @02:33PM (#32170205)
> This is not that. Asking DALLE for a bunch of football players, would expectedly produce a huddled group of black men. No issue, because the NFL are disproportionately black men. No outrage, either.

This is not great. Only about 57% of NFL players are black, and the percentage is more like 47% among college players. It would be better to at least reflect the diversity of the field, even if you don't think it should be widened in the name of dispelling stereotypes.

> Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.

Only about 1/3 of US prisoners are black. (Not quite the same as "criminals" but of course we don't always know who is committing crimes, only who is charged or convicted.) That's disproportionate to their population, but it's not even close to a majority. If DALLE were to exclusively or primarily return images of black men for "criminals", then it would be reinforcing a harmful stereotype that does not reflect reality.

Re:DALL·E now available in beta by cmeacham98 on Wednesday July 20, 2022 @07:47PM (#32173966)
> Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.

"criminals" producing most black people actually would be a perfect example of bias in DALL-E that is arguably racism.

Black people commit a diproportionate amoumt of crime (for a variety of socioeconomic reasons I won't get into here), but even so white people make up a majority of criminals (because white people are the largest ethnic group by far).

Thus, a random group of criminals, if representive of reality, should be majority white.

Re:DALL·E now available in beta by TomWhitwell on Wednesday July 20, 2022 @02:19PM (#32169994)
In the UK… “The Environmental Services Association, the trade body, said that only 14 per cent of the country's 91,300 waste sector workers were female.” So 2x dall-e searches should produce 1.2 women.
Re:DALL·E now available in beta by CuriousSkeptic on Wednesday July 20, 2022 @02:16PM (#32169943)
> Asking DALLE for a bunch of football players, would expectedly produce a huddled group of black men

I think, for about 95% of the world football is synonymous with soccer. Its kind of interesting that you take this particular example to represent what reality looks like statistically

Re:DALL·E now available in beta by ceeplusplus on Wednesday July 20, 2022 @02:36PM (#32170260)
Black people comprise 12.4% of the US population, yet they are represented at substantially above that in "OpenAI"'s "bias removal" process. Clearly it has, as you put it, substantial bias that gets in the way of accomplishing its task.
Re:DALL·E now available in beta by mysterydip on Wednesday July 20, 2022 @02:01PM (#32169745)
That's what Jerrrry is saying. Framing the reality of diversity in the world as a "problem" is wrong.
Re:DALL·E now available in beta by ketzo on Wednesday July 20, 2022 @01:41PM (#32169442)
serious question: in what way is that not a “problem?”
Re:DALL·E now available in beta by TheFreim on Wednesday July 20, 2022 @02:03PM (#32169763)
It's not a problem in a few ways, let me know what you think (feel free to ask for clarification).

1. The training data would've been the best way to get organic results, the input is where it'd be necessary to have representative samples of populations.

2. If the reason the model needs to be manipulated to include more "diversity" is that there wasn't enough "diversity" in the training set then its likely the results will be lower quality

3. People should be free to manipulate the results how they wish, a base model without arbitrary manipulations of "diversity" would be the best starting point to allow users to get the appropriate results

4. A "diverse" group of people depends on a variety of different circumstances, if their method of increasing it is as naive as some of the are claiming this could result in absurdities when generating historical images or images relating to specific locations/cultures where things will be LESS representative

Re:DALL·E now available in beta by bobcostas55 on Wednesday July 20, 2022 @01:47PM (#32169541)
Well, it's a problem for the ideology.
Re:DALL·E now available in beta by stuckinhell on Wednesday July 20, 2022 @01:46PM (#32169518)
Everything is an ideological war zone now. That's the world we live in now.
Re:DALL·E now available in beta by Fnoord on Wednesday July 20, 2022 @02:10PM (#32169864)
Perhaps its a problem you don't care about?
Re:DALL·E now available in beta by protonbob on Wednesday July 20, 2022 @01:28PM (#32169205)
Honestly I would rather that they not try. I don't understand why a computer tool has to be held to a political standard.
Re:DALL·E now available in beta by Jerrrry on Wednesday July 20, 2022 @01:37PM (#32169370)
There are legitimate reasons to reduce externalizations of societies innate biases.

A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example.

This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias.

It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted.

when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate.

It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?

Re:DALL·E now available in beta by ceeplusplus on Wednesday July 20, 2022 @02:41PM (#32170320)
That's great, but by doing so you are also inadvertently favoring, in your example, the people with black names. For example, Chinese people save on average, 50 times more than Americans according to the Fed [1]. That would mean they would generally be overrepresented in loan approvals because they have a better balance sheet. Does that necessarily mean that Americans are discriminated against in the approval process? No.

My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no.

[1] https://files.stlouisfed.org/files/htdocs/publications/es/08...

Re:DALL·E now available in beta by Jerrrry on Wednesday July 20, 2022 @02:59PM (#32170543)
I would agree that it is not.

The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias.

Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture.

In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.

Re:DALL·E now available in beta by mh- on Wednesday July 20, 2022 @01:54PM (#32169638)
> A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example.

That's a great example, thanks. Also, I hope the teams working on that come up with a different solution...

Re:DALL·E now available in beta by whywhywhywhy on Thursday July 21, 2022 @05:34AM (#32177334)
Wouldn't the whole point of a "Mortgage AI" be to discriminate so the lenders hands could be clean.

Not that I agree with that but I don't see why you would build one otherwise, if you wanted discrimination free mortgages wouldn't the whole process by anonymized and minimal personal information rather than the current system of having to hand over every detail of your life.

Re:DALL·E now available in beta by daemoens on Wednesday July 20, 2022 @01:32PM (#32169283)
It's not a political standard though. There is actual diversity in this world. Why wouldn't you want that in your product?
Re:DALL·E now available in beta by mensetmanusman on Wednesday July 20, 2022 @01:37PM (#32169378)
Fix the data input side, not the data output side. The data input side is slowly being fixed in real time as the rest of the world gets online and learns these methods.
Re:DALL·E now available in beta by throwaway4aday on Wednesday July 20, 2022 @01:52PM (#32169614)
In a sane world we would be able to tack on a disclaimer saying "This model was trained on data with a majority representation of caucasian males from Western English speaking countries and so results may skew in that direction" and people would read it and think "well, duh" and "hey let's train some more models with more data from around the world" instead of opining about systemic racism and sexism on the internet.
Re:DALL·E now available in beta by astrange on Wednesday July 20, 2022 @04:35PM (#32171922)
That wouldn't necessarily fix the issue or do anything. A model isn't a perfect average of all the data you throw into its training set. You have to actually try these things and see if they work.
Re:DALL·E now available in beta by norwalkbear on Wednesday July 20, 2022 @01:48PM (#32169552)
I agree, the trust is broken now. Im going to skip on any AI that pulls that crap.
Re:DALL·E now available in beta by kache_ on Wednesday July 20, 2022 @01:31PM (#32169255)
While their heart is in the right place, I'd like to challenge the idea that certain groups are so fragile that they don't understand that historically, there are more pictures of certain groups doing certain things.

It's a hard problem for sure. But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist".

Let me be mindful of the bias, until we have a generally intelligent system that can actually do it. I'm generally intelligent too, you know.

Re:DALL·E now available in beta by UmYeahNo on Wednesday July 20, 2022 @01:42PM (#32169449)
>But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist".

That is a really, really, narrow viewpoint. I think what people would prefer is that if you query "Scientist" that the images returned are as likely to be any combination of gender and race. It's not that a group is "fragile", it's that they have to specify race and gender at all, when that specificity is not part of the intention. It seems that they recognize that querying "Scientist" will predominantly skew a certain way, and they're trying in some way to unskew.

Or, perhaps, you'd rather that the query be really, really specific? like: "an adult human of any gender and any race and skin color dressed in a laboratory coat...", but I would much rather just say "a scientist" and have the system recognize that anyone can be a scientist.

And then if I need to be specific, then I would be happy to say "a black-haired scientist"

Re:DALL·E now available in beta by throwaway4aday on Wednesday July 20, 2022 @02:11PM (#32169876)
Have you seen the queries that are used to generate actually useful results rather than just toy demonstrations? They look a lot more like your first example except with more specificity. It'd be more like "an adult human of any gender and any race and skin color dressed in a laboratory coat standing by a window holding a beaker in the afternoon sun. 1950s, color image, Canon 82mm f/3.6, desaturated and moody." so if instead you are looking for an image with a person of a specific ethnicity or gender then you are for sure going to add that in along with all of the details. If you are instead worried about the bias of the person choosing the image to use then there is nothing short of restricting them to a single choice that will fix that and even in that case they would probably just not use the tool since it wasn't satisfying their own preferences.
Re:DALL·E now available in beta by kache_ on Wednesday July 20, 2022 @01:47PM (#32169542)
This is a problem with generative models across the board. It's important that we don't skew our perceptions by GAN outputs as a society, so it's definitely good that we're thinking about it. I just wish that we had a solution that solved across the class of problems "Generative AI feeds into itself and society (which is in a way, a generative AI), creating a positive feedback loop that eventually leads to a cultural freeze"

It's way bigger than just this narrow race issue the current zeitgeist is concerned about.

But I agree, maybe I should skew to being optimistic that at least we're trying

Re:DALL·E now available in beta by numpad0 on Wednesday July 20, 2022 @02:03PM (#32169769)
Kind of funny that NN tech is supposed to construct some upper dimensional understanding, yet realistically cannot be expected to be able to generate gender and race indeterminate portrayal of a scientist.
Re:DALL·E now available in beta by micromacrofoot on Wednesday July 20, 2022 @01:47PM (#32169539)
Historically this is true, but it also seems dangerous to load up these algorithms with pure history because they'll essentially codify and perpetuate historical problems.