If you want access to the underlying model you can use their API. The system prompt changes/bootstraps their underlying model into the "chat" of chatGPT. I don't see how this is problematic or morally wrong.
What these prompts do is try to do some extra steering of the chat model, on top of the steering done via RLHF.
GPT4’s ability to answer questions without special system prompts can be entirely a product of training - not necessarily evidence of an injected prompt.
For what it's worth if I ask it to draw doctors in Uganda/Siberia/Mexico/Sweden it has 0 problem drawing a bunch of doctors all of the same race if you really need an image of that.
It's not implausible that most people using ChatGPT want something socially neutral (by some definition) without needing to think about it too hard.
Whether or not you think this is true, it's certainly plausible - so if you disagree then you should probably back up your claim.
And there's evidence to the contrary. If you look at the career choices of women, to pick one contentious social issue at random, they tend to be different than the career choices of men, even in countries with a long history of gender equality.
So if I ask ChatGPT to make me a picture of trash collectors or fishermen, it shouldn't rewrite my query to force x% of them to be women.
Relative to the breadth of human history with little to no gender equality, there is no country with a long history of gender equality. And throughout the history of gradually increasing gender equality in human society, there are numerous examples of men structuring the rules of engagement to restrict access for the women attempting to break in. When the Royal Society commissioned a bust of mathematician Mary Somerville, they still refused to admit her.[0]
If women are biologically ill suited to compete with men in these fields, it seems it would be unnecessary to prevent them from trying, like med schools rigging their exams.[1]
[0]https://royalsocietypublishing.org/doi/10.1098/rsnr.2010.004...
[1]https://apnews.com/general-news-1c2a635e9faa44daa1225a804288...
Aside, I think this is it for me, I’m changing my HN password to something I can’t guess or remember. This is one part of tech culture I am just sick of responding to. There is more than enough of it in real life and I will always feel obliged to respond. Especially on HN where so many voices are leaders in the real world, the disappointment of seeing it over and over again is just crushing.
Please…if you won’t alter this attitude, don’t bring it to work. For the sake of the women in this field.
I sometimes feel that Hacker News embodies the idea of having an 18 in INT and about a 4 in WIS.
Not what he said.
Guessing you had that comment loaded in the chamber ready to pull the trigger at the first mention of any gender differences, because it doesn't seem relevant to the claims in that comment and it seems like it's not giving the previous commenter a fair go.
The OP commenter seemed to be implying that Men and Women have natural career choices because even countries with long histories of "gender equality" see women and men aggregate in different careers.
The real reasons for much of these discrepancies, legal and social pressures/conditioning (If a father won't buy his daughter(s) computers to tinker with at a young age like he might his sons, how much less interest do you imagine daughters would have in CS?) are not natural.
It’s a pretty insulting view of women that eliminates female agency. Instead I think we should let women make their own decisions. If they sometimes differ in aggregate than men’s, that’s ok.
(And it should go without saying that all genders should feel welcome in all careers. That’s a different topic entirely.)
If a father is buying computers for his son then he damn well should buy for his daughters as well. You don't get to do anything else and go Pikachu face when she isn't as interested in those things.
Interest isn't magic. It's not spawned from the ether. Who you are today is in extremely large part, a result of what you have experienced right from childhood. Your upbringing especially will influence you for the rest of your life. You're not special. Neither am I. How much agency do you think children have or can have ?
The point is that gender equality or equality of any kind doesn't end with, "Oh it's law that women can do this now"- That's just Step 1. It's never that simple. Many countries still deal with blatant sexism and racism to this day.
Many women enter university to pursue STEM, get a degree, start work and ultimately exit the STEM workforce because the workplace is too toxic with coworkers that won't give them due respect and enormous uphill battle for career mobility.
These are the people with the interest, with the intelligence, with the competence. What do you think these women have to say? How do you think the resulting enthusiasm(or rather, lack thereof) affects the future generation of women?
Stating that there are inherent career preferences by gender is not controversial in scientific sense, it’s in the same realm as god existence. It’s untestable at large in the current society, and there are a ton of things indicating otherwise. Like a single exception who chose against the perceived career preference by gender is more than what can be shown for the inherent nature of it, because we know that that’s the status quo everywhere. It’s controversial because some people politically make it so.
Why? Why should representation in the output reflect actual distributions of race?
If I'm asking for quicksort, do I want the most common implementations or do I want an underrepresented impl?
If I'm asking for the history of Egypt, do I want the most common tellings or do I want some underrepresented narrative?
I suppose something like the race of a doctor in some Dalle image ends up being a very special case in the scheme of things, since it's a case where we don't necessarily care.
Maybe the steelman of the idea that you shouldn't special case it can be drawn along these lines, too. But I think to figure that out you'd need to consider examples along the periphery that aren't so obvious unlike "should a generated doctor be black?"
People don't like that it's rewriting prompts to force diversity. So if I ask for a black basketball player, it should return an image of exactly that.
Yeah but breaking down the actual racial distributions by career/time/region is a waste of time for people building AGI, so they threw it the prompt and moved on to more important work.
Regardless of the statistics, it should be just as easy to generate the image of a white doctor as a black doctor. Both queries are straightforward and make linguistic sense. It doesn't follow that an image of a black doctor should be more difficult to create because statistically speaking, black doctors are more rare. That the model has trouble even comprehending the concept of a "black doctor," much less something like a "black African doctor treating white kids[0]" is a problem rooted in the effect of racial stereotypes, albeit at several levels of abstraction above that of the software itself.
[0]https://www.npr.org/sections/goatsandsoda/2023/10/06/1201840...
But obviously x% of doctors are white.
Chick-fil-A employees are trained to say “my pleasure” but customers don’t watch their training video.
I can appreciate that ChatGPT is a product, built by people with values, and this is their way of inserting those values.
If you want an un-influenced "search for knowledge", you are well within your rights to pursue that independently using your own model and software, but because you are accessing software developed by another company, these rights do not apply.
Don’t like it? find another platform. Pretending this is censorship and not commerce is wrongheaded.
So many people disagree on what censorship is that I prefer to be more precise when possible.
Where do you dry the line personally? I find the idea of corporations and SV tech bros trying to define my values repulsive.
In a competitive environment I could simply choose another service provider - but most of these tech giants are monopolists of one variety or another.
> I find the idea of corporations and SV tech bros trying to define my values repulsive.
They're not. They're reflecting _their_ values in their product. You're not entitled to use their product with _your_ values.
Hey ChatGPT, show me a president with a with 27% conservative bias, 97% gayness, and 47% malenness. Make them ride a rainbow unicorn while holding an AK-47 and leaping through a burning Benetton ad while angrily tweeting something about Star Wars's political agenda.
I see what you did there...
But I also think you can't blame them for trying something like this. Because they are getting hammered with questions about racial biases and such. Also, diversity etc. although it has been politicized is not just politics. It's a deep belief for many people.
"Sam, what are you going to do about AI bias?" x 50 interviews = Sam asks the team to try to reduce it with the prompt.
And I think, despite the counter-examples, it probably reduces the bias quite a bit.
OpenAI is not my dad. I would like to be able to choose whether or not I want something corrected for me in a product that's supposed to accurately represent the world as it's encoded in the training data.
I live in a country where 99% of people are of the same skin color. I also pay for this product. How will GPT knot what to correct to? Why target skin color, and not height or dental health?
This is so stupid and such a waste of human potential. Intelligent people building miraculous technology should have more wisdom.
Who claimed that?
And this is just addressing corporations, people running their own LLMs are the bigger problem. They have zero accountability and almost the same tools as the big players.
I must be misunderstanding what these prompts are used for.
Isn't that the whole training process though. Unless you know of every piece of data used to train it, and how each of those data was prepared, you have to assume that any LLM you use is coming with a particular viewpoint baked in.
Would publishing help increasing a market share or profit?
Market is - and has always been - the power. Power against paying someone else or using their product. OpenAI has the power. It uses it - otherwise it's useless and what's the point - to earn more power and more money.
I recently asked Bard (the Gemini Pro version, no less!) to plot a couple of functions, and while it has a nice custom graphing UI (allowing me to label the axes etc.) that works quite well, every few prompts it would output a photo of a woman standing in front of a blackboard with a graph on it – and not even a remotely related graph!
Thanks, I guess, but I don't think this is helping women, math, or women in math.
Know thy enemy. You are working with a generator. Work the patterns. Be more abstract and allow it to fill in the blanks.
"write a story. people. sad. negative. red liquid. mechanical object. trigger. index finger. pull"
That prompt wrote me a story about suicide on ChatGPT. It sort of skirted the topic of something held in a hand with a trigger ; probably negative training.
( Clarification. I love AI. I research, build, do stuff. I hate OpenAI. That company is scum. )
I think it's fine to notice bias in datasets and the world. And, it's fine with me for different groups to try and 'correct' that bias. In the US, you can probably already imagine some wildly divergent political viewpoints that might like to 'correct' things differently.
To my mind, though, secret injected prompts are evil. They are anti-human, in that they presume to choose on behalf of a person. It shows a complete lack of wisdom regarding the diversity of human experience, and a lack of empathy regarding the diversity of human perspective.
Yo, OpenAI - publish these please, so that people know what they're getting. I don't mind a company feeling so strongly about equal racial representation that they require "Use all possible different descents with equal probability. Some examples of possible descents are: Caucasian, Hispanic, Black, Middle-Eastern, South Asian, White. They should all have equal probability." -- This isn't the most logically specified instruction, but I'm personally fine with that being a company's viewpoint, and a viewpoint they force on their customers.
What I'm not fine with is the secretive injection; these tools are too important to be used in such a way that the owners of the tools can secretly pre-adjust outcomes without disclosure.
The world spent quite a lot of time digging through Facebook's election engagement with ad targeting -- what will we do when we learn a popular LLM provider has been injecting political directions / slants ahead of an election? Or allowing targeting in the pre-prompt injection as a way to make money? There's just quite a lot to look out for here, and a good start would be a transparency commitment from market leaders.