Essentially automating existing biases, and in a way its even more insidious because companies can point to the black box and say "how could it be biased, its a computer!".
Then you have companies trying to overcorrect the other way like Google with their AI images fiasco.
Having said that, these false outputs have a strange element of correctness to them in a weird roundabout uncanny valley way: we know the input has been tampered with, and is biased, because the output is obviously wrong. So the algorithm works as intended.
If people are discriminatory or racist or sexist, it is not correct to attempt to hide it. The worst possible human behaviours should be a part of a well-formed Turing test. A machine that can reason with an extremist is far more useful than one that an extremist can identify as such.
Obviously a lot of this was done by users for the gotcha screen grabs, but in a real world product users may realistically may want specific demographic outputs for example if you are using images for marketing and have specific targeting intent or to match the demographics of your area / business /etc. Stock image websites allow you to search including demographic terms for this reason.
Now note again, that the current set of biases got us in an existential risk and likely disaster. (Ask Exxon how unbiased they were.)
AI does not optimize for this thing at all. It cannot tell the logical results from, say, hiring a cutthroat egoist. It cannot detect one from a CV. Which could be a much bigger and more dangerous bias than discrimination against disabled. It might be likely optimizing for hiring conformists even if told to prefer diversity, as many companies are, and that would choke any creative industry ultimately. It might be optimizing for short term tactics over long term strategy. Etc.
The idea here is that certain set of biases go together, even in AI. It's like a culture, we could test for it. In this case, hiring or organizational culture.
This makes sense because humans aren’t biased, hence why there is no word for or example of it outside of when people make adjustments to a model in a way that I don’t like.
What is the purpose of the system? What is the purpose of the specific component that the model is part of?
If you're trying to, say, identify people likely to do a job well (after also passing a structured interview), what you want from the model will be rather different than if you're trying to build an artificial romantic partner.
There are those who say that the purpose of a system is what it does.
Sociopolitically, "biased" in this context clearly refers to undue discrimination against people with disabilities or various other marginalized identities.
The meaning of "biased" you are using ("accurately maps input to output") is perfectly correct (to the best of my understanding) within the field of ML and LLMs.
The problem comes when someone comes to you saying, "ChatGPT is biased against résumés that appear disabled", clearly intending the former meaning, and you say, "It is not biased; the output is correct according to the input." Because you are using different domain-specific meanings of the same word, you are liable to each think the other is either wrong or using motivated reasoning when that's not the case.
there is a group of people who see the regurgitation of existing systemic biases present in training data as a convenient way to legitimize and reinforce interests represented by that data.
"alignment" is only a problem if you don't like what's been sampled.
Do you have a link to someone stating that they see this as a good thing?
I prefer to assume the best in people I'm actively talking to, both because I prefer to be kind, and because it cuts down on acrimonious discussions.
And a machine that can plausibly sound like an extremist would be a great tool for propaganda. More worryingly, such tools could be used to create and encourage other extremists. Build a convincing and charismatic AI, who happens to be a racist, then turn it loose on twitter. In a year or two you will likely control an online army.
This is false. A simple dictionary check shows that the definitions are in fact not circular.
Even in the best case, when a term is clearly defined and well-mapped to its referent, popular usage creates a connotation that then supplants the earlier meaning. Dictionaries will sometimes retain older meanings/usages, and in doing so, build a roster of "dated", "rare", "antiquated", or "alternative" meanings/usages throughout a term's mimetic lifecycle.
When I think of "correctness" in programming, to me that means the output of the program conforms according to requirements. Presumably a lawful person who is looking for an AI assistant to sift through resumes would consider something that is biased against disabled people to be correct and conform to requirements.
Sure, if the requirements were "an AI assistant that behaves similarly to your average recruiter in all ways", then sure, a discriminatory AI would indeed be correct. But I'd hope we realize by now that people -- including recruiting staff -- are biased in a variety of ways, even when they actively try not to be.
Maybe "overcorrecting" is a weird way to put it. But I would characterize what you call "correct according to the inputs" as buggy and incorrect.
> If people are discriminatory or racist or sexist, it is not correct to attempt to hide it.
I agree, but that has nothing to do with determining that an AI assistant that's discriminatory is buggy and not fit for purpose.
The easy obvious answer here is to "Do what's right". However if 21st century political discourse has taught us anything, this is all but impossible for one group to determine.
And while “the arc of the moral universe is long, but it bends toward justice.” .. it gyrates a lot overcorrecting in each direction as it goes.
Handing the control dials to a educationally/socially/politically/etc homogenous set of San Fran left wing 20 somethings is probably not the move to make. I might actually vote the same as them 99% of the time, while thinking their views are insane 50% of the time.
As a moderate conservative I feel the exact same.
Remember that's all this is, statistics not a logical program. The model is based on population data
I know Vox does not have the credibility of mainstream news, so evaluate its reporting as you will.
They could be something like a compression artifact?
Same problem that children have always had.
The reason research like this can still be useful is that of the people who write labor laws (and most of the people who vote for them) aren't necessarily going to "understand that the results from any data-based modeling process is a concactination of the cumulative input data topologies and nothing else"; an academic study that makes a specific claim about what results would be expected from using ChatGPT to filter resumes helps people understand without needing domain knowledge.
That is not what mathematically impossible means.
I don't think companies "hire" their owners, exactly.
Setting aside ethics, there are so many bright line anti-discrimination rules that I find it hard to believe that an AI could possibly account for them, not without lots of hand-holding. One often forgotten law states that you cannot discriminate against veterans. That is a hard thing for an AI to grasp. Phrases like "served four years in X" is confusing, so too all the military names/units/ranks. But if your AI is even slightly downvoting veterans... good luck in court. What makes that particular law so dangerous is there is no sliding slope. Either someone is a vet or not: a binary choice. So much of the are they/aren't they testing is dead simple. It will be detected and actioned against very quickly.
But not really, its mostly just predicting the next token.
The US Supreme Court ruling with regards to Colorado leaving Trump off the ballot was a complete farse. Their explanation was conveluted and contradictory, and they decided to include answers to questions that weren't directly part of the case. What is an LLM supposed to do with that, and how can an LLM trained on our laws be expected to make use of that when courts can, and sometimes do, go against the rules as written?
I'm also saying that we have case law that contradicts itself and violates the rules of how the courts are supposed to work. Those examples, if included in training data, would confuse an LLM and likely lead to poor results.
I don't think that LLMs are good enough that they can they confused by logical inconsistencies in the training data.
https://www.uscourts.gov/about-federal-courts/educational-re...
Miranda full decision: ten pages. The bit that matters in the real world? Literally nine words.
For future cases the full explanation does still matter too, especially from the Supreme Court. People only remember 9 words from the Miranda decision but the rest of the 10 pages are still case law that can absolutely be used to impact future cases.
If they are then, an objective system would pick a disabled person, else it should pick a non disabled person, other things being equal or better.
As a matter of law and ethics the answer could be different in either case, but objectively, is the system outputting the better answer?
Of course laws and ethics may direct us otherwise.
Hawking was disabled, but you’d be hard pressed to find a better astrophysicist, for example.
Which also implies that humans aren't aligned with "human goals" in the first place.
Even if your assertion that "hiring departments are well known for discriminating" is true, the ChatGPT bias is independent of that and is coming from casual human behavior on social media, not corporate malevolence.
So of course a model will be biased against people hinting at disabilities, because existing hiring departments are well known for discriminating and are regularly fined for such
So the only data it could possibly learn from couldn’t teach the model any other possible state space traversal graph, because there are no giant databases for ethical hiring
Why don’t those databases exist? because ethical hiring doesn’t exist in a wide enough scale to provide a larger state space than the data on biased hiring
Ethical Garbage in (all current training datasets) == ethical garbage out (all models modulo response NERFing)
It is mathematically impossible to create a “aligned” artificial intelligence towards human goals if humans do not provide demonstration data that is ethical in nature —- which we currently do not incentivize the creation of.