People should understand AI safety != AI ethics. AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion.
The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks.
The ethics branch (responsible for all the media attention about racist AIs/bias etc) view alignment people as pie in the sky researchers whose results are irrelevant to society.
It seems increasingly that the concerns from the ethics branch have not come to pass at all. Stable diffusion does not default to creating black people (you have to insert black into the prompt), this has not caused mass social harm despite the tens of millions of users. Hence companies are unwilling to pay for these teams.
The other concerns, such as mass unemployment, or copyright of training material, are real, but they are better understood by economists, lawyers etc, rather than this awkward branch of AI ethics.
Go back long enough and they might have banned the written word for being dangerous, too.
We should stop calling them ethicists.
Whatever "reasonable middle ground" you find, you'd have to make logically sound ethical arguments. This is what these people do for a living, and they are very well trained in those matters.
As programmers and engineers we are also trained in logic, use it every day. But that typically pales in comparison of what philosophers do. They deal with much richer logical systems and can precisely apply them to statements and arguments.
When applied to ethics, they typically end up with much more radical conclusions that we are dealing with here. In fact the resulting initiatives are already softened compromises and a "reasonable middle ground" before it even clashes with policy making.
Babies and children today are routinely subjected to avoidable adverse stimuli, such as circumcision and corporal punishment. That a typical parent can do this at will for no good reason, but a scientist wanting to do something similar to advance human knowledge must seek approval from an ethics board (which they would not get) is ridiculous.
Now the use of the results of that research...
Do you remember the drama[1] with Google's AI ethics team? Here's a sample of the "research" they were producing: https://s10251.pcdn.co/pdf/2021-bender-parrots.pdf
[1] - https://en.wikipedia.org/wiki/Timnit_Gebru#Exit_from_Google
I'd think that in light of all the problems we've seen with chatGPT "hallucinating" answers it reinforces their concerns about "stochastic parrots" more than anything.
I mean, if you have specific grievances, please do share them as this is an important conversation, but as of now your comment amounts to mudslinging and no substance.
It's all about whichever field the particular use-case belongs to, and would lose nothing by seeing the AI as a black box.
* IIRC there have been issues with models that find correlations being presented as finding causality. That is something that belongs to AI as a field; other people can't evaluate your stuff accurately if you've misinformed them about its capabilities.
Isn’t it a bit early to tell? I can foresee ChatGPT being used to farm “karma” on social media sites to bot accounts to credibility on sites like Reddit - maybe even HN.
There’s absolutely zero way to know that yet.
Seriously though, people tend to dismiss things they are know are bad if there is no immediate harm. Certainly they had no idea how incredibly pervasive the lead from gasoline would be, that it would be detectable at harmful levels in basically everyone. It is also hard to measure these harms. How much crime of the 70s and 80s could be attributable for example.
It was instantly obvious to anyone that using leaded gas esp. in agriculture would be deadly and toxic. The risk was ignored for profit, PR handled, underplayed. Even the chronic effects were known!
We even had an early alternative of just adding more ethanol to the pettol. But no...
It took 50 years of scientists trying to convince public how deadly this thing is, with varied success. And 30 more years for the bulk of the phaseout.
Who are “we”?
The same could be told about AI - content farms, generating spam that bypass spam filter, etc…
Some people knew them, majority of the public don’t
Then there was a big chunk of time taken out for WW2 where such considerations were pushed out. The issue returned later but then it was argued there was no alternative for years.
AI toxicity... We don't even know the risks. There's potentially no limit to the damage. We have some examples (metrics making echo chambers, automated classifiers giving people wolf ticket for no reason, amplified evil biases in conversational AI, fake news and deepfake generation) and some guesses for now. We do not even know how to begin to regulate it.
It's not even the same ballpark. Internet would be a closer comparison and it's still wild west.
My point is not that we should or should not regulate AI. It's just that I believe the causal link from certain types of AI usage to concrete harm for society is too complex for enough humans to band together and achieve consensus a priori on regulation.
I'm sure there are forums out there sufficiently expert that's not the case... but that doesn't feel like most forums (even niche).
Exhibit A: me, who doesn't post particularly interesting things on HN, but eventually accrues karma simply from time
I don’t think one’s access to different subreddits is something anyone should waste brain cycles on, many bigger fish to fry :)
This world features a competition between radically opposed ethical systems. We don’t agree on whether ethics is objective (i.e. some people’s ethics are superior to those of others) or subjective (i.e. “superior” is just another way of saying “same as my own”), and even those who agree that it is objective, often can’t agree on who is objectively more correct, even though they both agree that in principle someone must be.
I much prefer those who talk about ethics while making clear which brand of ethics they are selling - whether their name is John Rawls or Robert Nozick or Peter Singer or Edward Feser - than people who pretend we all agree on what is ethical, and leave their actual ethics as something to be inferred rather than openly advertised. From what I’ve seen, most “AI ethics” fails at this completely
For me, democracy is a system where persons can vote for their interests. Different persons have different interests and that's good. You must compromise with others so everybody gets partly satisfied.
Others see democracy as a way to establish truth, those are the ones talking about ethics. Even if they lose, they still believe they must win, because they're right and the others wrong.
(I probably do not have to spell out economic implications of that.)
By contrast, these “AI ethics” people aren’t clear about what their first principles are, or how those principles differ from those of other people. In that regard, as horrible as Singer’s conclusions may be, he’s vastly more honest and transparent and self-aware than the average “AI ethicist”
AI ethics people are the group who are not capable of bringing AGI to life, but want to make sure that those who can do it responsibly or not at all.
ChatGPT just spouts out a different wild guess when you tell it it's wrong, but doesn't learn from its mistake—not even within a single chat. Sydney just goes full psychopath.
What you're describing is a particular kind of superintelligence, recursively improving kind. Between that and general intelligence is another gap.
No, in the same way that boiling water in a kettle is not incrementally closer to inventing the fuel for a rocket ship.
You can, in hidsight do a linear connection of water boiling -> steam engine -> fuel engine -> aeronautics -> space travel
But the first time someone boiled water, if you said that got you closer to being on the moon I think you would be mad. There is a non trivial chance that all our current work in AI is functionally useless in terms of creating AGI, it might take a complete breakthrough in ML, tech not invented yet, or it might not be possible at all.
Who is to say that our current ML models are not Ether studies and utterly useless.
If we crack AGI in 10 years, maybe our current models are the fuel engine. But if they are not even close, we might be boiling water. Or perhaps be wrong all together, thats why I said your observation of us getting closer thanks to those proyects is not right, at least not yet.
Large language models might not be capable of general intelligence by themselves, but a model network that includes a LLM as part of a generalized game playing model would certainly have the potential to get there.
But we also have examples of things all around us we cannot make ourselves (e.g., a simple animal or lifeform from scratch).
If we can define what "generally intelligent" means in concrete terms, we can absolutely create it. The problem is that we don't have a good understanding/shared idea of what we're trying to achieve. We can create models that can learn to play many different games using the same weights, solve IQ/logic tests and pass the Turing test consistently, but people are going to move the goalposts and say those are just complex mashups of autocomplete and flowcharts.
To be honest I don't think AI skeptics are ever going to believe until there's a Skynet trying to exterminate them.
I rather think AGI "idealists" (maybe there is a better word) will forever hold out (any) algorithmic advances as a sign of the immanent creation of AGI proper.
But no--the state of the world in that second versus the next could not have been predicted. It just happened, just so, and it would take longer to calculate the determinism than to adjust to the new.
Notably the people closest to the fire, those advancing SOTA AI capabilities and those deepest into alignment/existential risk research, both say it is coming this decade. They predict very different outcomes (crudely summarized: utopia vs extinction), but they both predict the event horizon is on our doorstep.
You're right that we might get it wrong, but getting things wrong with a new technology really always means negative outcomes. When you're figuring out rocket science and you realize you worried about the wrong things your rocket doesn't somehow still succeed and make it to space, it blows up in a fireball. It might not blow up because of the specific thing you were paying attention to, but it still blew up.
Why should we believe you over the experts for whom this is their life's work? What are your credentials? Where is your experience? You seem to think they're unserious clowns so clearly you have a good reason.
Existing AI is more important to worry about.
This article is a bit dated but it gives a good explanation of the difference between the powerful but limited tools we have today versus a true AGI.
AI alignment people are the group who believe “responsibly” means, centrally, bringing AGI to existence as fast as possible, and, critically, under the tight, opaque, authoritarian control of a narrow group of people whose ideology is, well, exemplified by figures like Sam Bankman-Fried, and doing everything possible to advance that goal, including fostering broad acceptance of sub-AGI AI under the same tight control for other purposes without concern for adverse social impact as a means of advancing and funding progress toward AGI. They believe that the ends (AGI) inherently justifies any means taken in its pursuit.
AI ethics people are the group who believes “responsibly”, withe regard to AI, isn’t restricted to pursuit of AGI (which is maybe a nice to have, but a non-essential goal), and means openness, transparency, and resolving bias and harm issues as rapidly as possible when, and ideally before, adopting AI – including AI far short of AGI – in roles with social impact (and most critically, resolving those for roles with the greatest social impact.) They believe that AI is, to the extent it at any level is desirable, is a means to improving concrete, present, immediate conditions, not an ends to be pursued in itself.
From what I’ve seen it’s the exact opposite? They want AI development to slow down so we can get it right and make it safe. There are people who are trying to speed it up but that seems like the opposite of alignment to me
The only way that the alignment faction wants to slow things down is in that they want to maintain tight control of models and gated access to assure “proper” use. This does have an effect of slowing certain kinds of development, as broader access to build around models without centralized controls fosters some aspects of development (the “Stable Diffusion moment”) but in ideal terms, the alignment faction is about fast-but-narrowly-controlled development.
The ethics faction is more about slowing things down – though more about adoption, particularly in sensitive uses, than development – but is more oriented toward openness/transparency.
(and, y'know, being a fraudster who temporarily pulled the wool over the eyes of the regulators not just the charities he promised money to)
Also: is SBF really the central example of alignment in your head, and not, say Yudkowsky? Because Yudkowsky is my central example, and he's the exact opposite on all counts, wanting us to go slow and explicitly saying that humans are terrible at reasoning properly when they allow themselves to think "ends justify the means": https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t...
Likewise, there are plenty of examples of misaligned non-general AI, and have been since at least the 80s when it was GOFAI and databanks rather than neutral nets that led to new legislation: https://www.legislation.gov.uk/ukpga/1984/35/enacted
For normal people who don't spend six hours every day on lesswrong, they were literally in the same camp up until about six months ago (when all the "rationalists" violently disclaimed SBF after being funded by him for a few good years.)
No, he's a particularly well-known example of the broader ideology to which virtually all of tbe alignment camp subscribes, and which centers their desires for AI.
Hey, let's be fair now, a lot of them also believe that we just need a world government that tightly controls GPU use.
Sarcasm aside, the alignment crowd definitely does not have a consensus about the desirability about creating AGI asap. Yudkowsky is clearly horrified by OpenAI.
Mostly not at all. They write papers like "Can Language Models Be Too Big?"
I say then: can people be too afraid of the unknown?
Whether either is capable of "bringing AGI to life" remains to be seen, but seems very doubtful.
I don’t think that’s really true. Rather, the faction that sees alignment as a subordinate concern within ethics hates the faction that sees ethics as a subordinate concern within alignment.
> The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks.
Not really; the alignment-priority branch has no problem recognizing (certain subsets of) the bias and other issues that are the more central focus of the ethics branch as real and significant, but they view alignment as solving them and view narrow centralized control as a mitigation (for both “alignment” and “ethics” types of issues) until it is solved. They also view progress on AI as a general priority, for a mix of commercial, ideological, and other [0] reasons, and their own progress on AI in particular as essential (because they view AI from others – either because potentially unaligned, or for other reasons – as a critical and potentially existential threat.)
> The ethics branch (responsible for all the media attention about racist AIs/bias etc) view alignment people as pie in the sky researchers whose results are irrelevant to society.
No, they view them as an enormous threat, because they are working in a way that not only insufficiently mitigates in basic approach because of a lack of priority, but also magnifies by its focus on closely concentrated control, the issues of paramount concern to the ethics-focused.
> It seems increasingly that the concerns from the ethics branch have not come to pass at all.
They have in fact come to pass in deployments of AI in all kinds of socially-significant roles. E.g, the facial recognition in the CBP One app newly mandated for asylum seekers doesn’t work well for dark-skinned faces (racial bias in facial image recognition is literally one of the earliest publicly recognized AI bias problems that motivated the AI ethics movement.) They’ve also manifested in systems deployed in welfare management [1], hiring [2], and all kinds of other areas.
[0] e.g., Roko’s Basilisk
[1] https://www.wired.com/story/welfare-algorithms-discriminatio...
[2] https://www.reuters.com/article/us-amazon-com-jobs-automatio...
(Disclaimer: my view is of the forum itself; I don't know how long it persisted in SF circles.)
Those two branches despise each other with intense passion.
I never actually made this connection before, but having spent far too much time listening to both camps, this observation does ring true. Another observation I had is that there is an orthogonal axis to the safety/ethics dichotomy that follows a more scholastic approach, namely neuro/symbolism. These two branches correspond, roughly, to the debate between machine learning and mechanized reasoning (a.k.a. good old fashioned AI).Vastly oversimplifying, the neural branch believes reasoning comes after learning and that by choosing the right inductive biases (e.g., maximum entropy, minimum description length), then training everything up using gradient descent, reasoning will emerge. The symbolists place a heavier emphasis on model interpretability and believe learning emerges from a logical substrate, albeit at a much higher level of abstraction.
Like safety/ethics "research", neural/symbolic folks are constantly bickering about first principles, but unlike their colleagues in the S/E camp (which is at best philosophy and at worst fanfiction), N/S debates are resolved by actually building things like language models, chess bots, programming languages, type theories and model checkers. Both are valid debates to have, but N/S is more technically grounded while S/E is a bit like LARPing.
The schism isn't a technical disagreement about ai, it's a disagreement between subcultures about what values are important.
Red & blue are exactly what a U.S reader would think they are. "Grey" is an attempt by Scott to define something else, loosely hacker-ish / rationalist (you can tell who he thinks the cool people are). "Rainbow tribe" I just made up, but I think you can guess what I mean now based on context.
Alignment tends to be "rationalist panic" and ethics is a more traditional "moral panic". I believe both perspectives are valuable.
[1] https://web.archive.org/web/20200219044501/https://slatestar...
As a non-American, only the fact you wrote "American" makes me think democrat-republican; but for that I would've thought of the similarly named TV tropes entry: https://tvtropes.org/pmwiki/pmwiki.php/Main/BlueAndOrangeMor...
There was no mixture between the two groups.
Red = Republicans/Conservatives
Blue = Democrats/Liberals (in the US sense, not classical liberals)
Grey = Liberterian/Rationalists
Rainbow = LGBTQ+, usually a subset of the Blue tribe in the US
Blue: union army
Red: British army
Rainbow: Lovely, prismatic army that covers everyone else?
I still don't understand. You're saying words of import without landing the plane, IMHO.
Read the essay if you want the full story. Short version: Blue is a cluster of Democrat/left/liberal/urban. Red is a cluster of Republican/right/conservative/rural. Gray is a smaller niche of like Libertarian/Silicon Valley/nerds.
Ethics vs alignment is actually a perfect example of the Blue/Gray split. How dare you act like your science fiction fantasies are real when this tech is harming marginalized communities? vs. How dare you waste time on mandating representation in generated art when this thing is going to kill us all?
I wish it were so, but I think the tribal lines create a lot of motivated cognition about the technical details. I don't think there are many blue tribe members who say "yes we agree that rogue AI has a 10+% chance of destroying humanity by 2040, but I still think that it's more important to focus on disparate impact research than AI alignment." They instead just either assign trivial probabilities to existential risks occurring, or just choose not to think about them at all.
These people still exist?
Personally, I don’t think gradient descent is the way to AGI either (I think it’s efficient algorithmic search over the space of all computable programs), but I haven’t heard anything from the symbols crowd since the early 2010s.
Of course, I just finished reviewing a few papers from ICLP (the International Conference of Logic Programming) 2023. This year there was a substantial machine learning element, most of it in the form of Inductive Logic Programming (i.e. logic programming for learning; ordinary logic programming is for reasoning).
This September I was in the second IJCLR (International Joint Conference of Learning and Reasoning) where I helped run the Neuro-Symbolic part of the conference. Like the first year we had people from IBM, MIT, Stanford, etc etc (to clarify, my work is not in NeuroSymbolic AI, but I was asked to help).
Then in January there was the IBM Neuro-Symoblic workshop, again getting together people from academia and industry.
And this year someone I know very well almost got poached by a big university in a certain country I'll be very vague about where there was what I would describe as a sudden explosion in interest in neuro-symbolic machine learning. They're trying to find an edge, you see.
Yeah, there's interest in combining symbolic and statistical machine learning.
Just two data points about why (well because, why not, but):
a) Machine Learning really started out as symbolic machine learning, back in the '90s when people realised Expert Systems need too many rules to write by hand. A textbook to read about that era of Machine Learning is Tom Mitchell's "Machine Learning" (1997). The work the public at large knows as "machine learning" today was at the time mostly being published under the "Pattern Recognition" rubrik.
b) To the early pioneers in AI having two camps, of "statistical" and "symbolic" AI, or for "connectionist" and "logic-based" AI, just wouldn't make any sense at all.
Consider Claude Shannon. Shannon was at the Dartmouth workshop were "Artificial Intelligence" was coined, in 1956. Shannon invented both logic gates (in his Master's thesis... what the fuck did I do in my Master's thesis?) and statistical language processing ("A Mathematical Theory of Communication"; where he also invented Information Theory, just as an aside).
Or, take the first artificial neuron: the Pitts and McCulloch neuron, first described in 1942, by er, by Pitts and McCulloch, as luck would have it. The Pitts & McCulloch neuron was a self-programming logic gate, a propositional logic circuit.
Or, of course, take Turing. Turing described Turing Machines in the language of the first order predicate calculus, a.k.a. First Order Logic (mainly because he was following from Gödel's work) and also described "the child machine", a computer that would learn, like a child.
To be honest, I don't really understand when or why the split happened, between "statistical" and "logical". Anyone who knows how to fill in the blanks, you're welcome.
Anyway those are silly distinctions that are only amplified by the cluelessness of social media today. The truth is that, as it was always, a good scientist uses the tools of science. No exceptions.
That's the alignment problem. Its just the usual "rational self interest" but now we have a new, alien form of intelligence as a peer to contend with.
The history that I remember very much did experience AI biases and racism. And tons of ethical questions were indeed raised as a result. These concerns did come to pass, at an alarming level, and it took a tremendous amount of engineering to modellers to minimize those. Let’s not rewrite history here.
The recently adopted app Customs and Border Protection mandates for asylum seekers still has that problem, its not a mere historical footnote of early efforts.
> In fact I remember the ChatGPT team putting tons of engineering into supervised learning to not let it fall victim to regurgitating these biases.
And I remember seeing demonstrations (here, on HN) within the last week of how ChatGPT still falls victim to regurgitating those biases. (And then will lie about its ability to assure that it won’t do so in the future.)
RLHF was not conceived by AI Alignment people. Using RL to train generative models was a thing even ten years ago. Now they finally made it work on scale. This has nothing to do with alignment.
[1] Deep Reinforcement Learning from Human Preferences https://arxiv.org/abs/1706.03741
[2] Fine-Tuning Language Models from Human Preferences https://arxiv.org/abs/1909.08593
[3] Learning to summarize from human feedback https://arxiv.org/abs/2009.01325
[4] Recursively Summarizing Books with Human Feedback https://arxiv.org/abs/2109.10862
[5] Training language models to follow instructions with human feedback https://arxiv.org/abs/2203.02155
I don't think we should hamper the usefulness of a tool to please someones sensibilities. For most people in most of the world such words aren't relevant.
I get it, these models are made the the the US and they think these models should preserve and further their values but the rest of the world doesn't care. Most likely each of them have their own batch of words they would prefer the AI didn't understand for whatever uncomfortable reason. Reality is that censoring it on any axis won't please everyone.
I'm so glad that you are confident on this. What is the basis of your confidence?
It is literally happening right now in the US justice system: https://www.technologyreview.com/2019/01/21/137783/algorithm...
There's all sorts of problems with this, but the use of "AI" to launder biased data (via IP law no less - these are proprietary blackbox algorithms being used by the justice system, owned by corporations - every part of that is antithetical to open and fair trials for citizens).
The ethics branch have been sounding the alarm for a long time on this, and have been exactly right as to what the problem is.
DNA has also cleared a lot of people who have been falsely accused.
DNA in theory is very simple. It is much harder when you got a mix of multiple unknown donors, contaminated or decayed samples. Problems quickly occur when they then apply some "computer algorithms" to puzzle together things to create a probabilistic match. It is those proprietary black box computer algorithms that are being heavily discussed in courts, not the theory of DNA.
If we removed all cases that involved multiple unknown donors, contaminated or decayed samples, then DNA evidence would be much safer and open and we wouldn't need those proprietary computer algorithms. It would also significant reduce the number of cases where DNA evidence is used.
It usually refers to fine tuning language models using data labelled by humans.
Hugging face have a good overview in this article: https://huggingface.co/blog/rlhf
honestly in order to make race an irrelevant or random factor, I believe the training data would have to be evenly divided between all known races... and possibly some blends of races? And I think that would be a good thing.
Could you synthesize such data by (ironically perhaps) using AI to change the race of a person in a given training photo?
You would need equalized amounts of text referring to various groups of people and topics too. That's much harder to augment.
I've been to a couple of "AI ethics" talks recently where the speakers used AI ethics and AI alignment as if they were synonyms.
Neither have the concerns from AI alignment folk. I’d say both groups are pretty far off (and both fairly useless), but the ethics folks are at least a little closer to reality.
Of course, AI alignment folks have the advantage that they can keep pushing the time frame forward and saying it’s perpetually just around the corner (a fairly common practice for doomsday believers whose doomsday never comes to pass).
> The other concerns, such as mass unemployment
After the Great Recession there was a high level of unemployment. Many argued that this was the result of the recession, and that if we fixed the issues in the economy the labor markets would rebound. Others argued that this wasn’t the case at all, that the market was either soft because American workers suddenly lacked high tech skills (one of the reasons STEM became fixed in the popular imagination), or because the robots were taking our jobs and the jobs weren’t going to come back.
The “robots are taking our jobs” group told us that this was just beginning - unemployment was going to get significantly worse over the next few years (the previous decade). Self driving cars were going to cause millions of truckers to be unemployed by 2020. The idea that new jobs would appear to replace ones that were lost was completely dismissed.
As we’ve seen from the improvements in the labor market over the past few years, the folks who said the soft market was the temporary result of a giant recession appear to be correct. The folks who said the robots took the jobs and they were never coming back were clearly wrong. If anything, they probably had a negative impact on the labor market by trying to convince people that nothing could be done to get the jobs back and encouraging inaction.
I have the exact opposite read on the situation. Everything bad which has come from AI are things the AI Alignment crowd predicted while none have been things the AI Ethics crowd predicted.
Uh, yeah. We wouldn't be having these discussions if they had. There is no reason whatsoever to assume AI alignment is just going to work itself out. Sure, it's possible, but it's also possible to survive a bullet in the brain. Would you take the chance?
Traumatizing Kenyan workers with horrific content is unethical:
https://www.vice.com/en/article/wxn3kw/openai-used-kenyan-wo...
giving police the green light to arrest people based on AI that is known or suspected to be unreliable is unethical:
https://www.wired.com/story/wrongful-arrests-ai-derailed-3-m...
I'm pretty sure I could find lots more actual unethical things that have occurred in the name of AI all day long. AI breeds unethical outcomes like water is wet, there's no need to be wondering. AI ethics teams are fired because everyone realizes nobody can really afford to bother to be "ethical" at all, easier to just sweep the bad stories under the rug (which of course can be done using...AI! )
For example a logistic regression model for fraud detection that bases it's decision mostly on zip codes. Is actually a proxy for deciding based on race (in the US at least)
These kind of models are already massively used and commercially sold not as some curiosities but as decision makers that impact the day to day life's of millions if not billions of ppl.
[0] https://www.reuters.com/article/us-amazon-com-jobs-automatio...
Sure it does; just give it a prompt about urban poverty, inner city youth, or welfare mothers.
Nonsense, there are countless ethical and alignment harms in the wild, and have been for many years. For example:
- midjourney ripping off artists = ethical harm
- google search telling people to throw batteries in the ocean = alignment harm
- Buzzfeed slashing staff and replacing them with chatgpt = ethical harm
- google photos labelling african americans as "gorilla" = alignment harm
- deep fakes = ethical harm
- COMPAS parole recommender biased against minorities = alignment and ethical harm
- Amazon resume screening model biased against women and minorities = alignment and ethical harm
...I could do this all day, but the point I'm trying to make is that AI ethics and AI alignment are both valid and important concerns. Hoping that companies will take care of them without regulation is folly.
yeah but in buzzfeed's case it's a bit moot because their content can't possibly get any worse than it already is.
These roles might not satisfy their ostensible purpose, but they satisfy what may be an even more important purpose for the organization. It’s a cost of doing business (of a certain type).
For some, it’s earnest belief that the design is effective and for others it’s just signaling that they’re willing to play game and not upset apple carts unnecessarily.
Regardless, these fashion trends are real and can’t be rejected lightly. As mentioned, they’re just part of operating at a certain scale.
But I think Chief Compliance Officer and Chief Risk Officer are pretty analogous - their main directive is to ensure other teams adjust their behaviors to follow the rules. I think the difference, though, is that these roles have unambiguous, and in many cases legally defined, requirements. I actually think it is the right moment to argue those are effective - SVB famously went without a Chief Risk Officer for a year despite having an unusually high number of risk committee meetings. We've all seen what happens when companies don't take risk and compliance seriously.
With things that are "fuzzier" (and by "fuzzier" I mean real but ill-defined/hard to measure impacts on the business of making money) it's just too difficult to have impact on teams you don't own. A Chief Risk Officer can easily make the case "If you don't do what I say, our company will fail - like, completely cease to exist in 2 days." I don't think a Chief Diversity Officer has the luxury of that sense of immediacy.
In any case, thanks for helping me think about this a little more broadly.
But ethicist at some company - I've always wondered what they do all day.
1. Say, "Look, we're trying here. We hired a CDO."
2. Have someone to take the blame if/when nothing changes. "We hired a CDO and nothing has changed, it must be their fault."
I've seen some very frustrated people in this role that really want to make a difference, but find nothing but roadblocks.
I think a CDO role would be more effective as an official part of HR and Recruiting groups as the first step is getting diverse applicants and hires.
That's how we get AI that is not neutral. Microsoft made a necessary move if their goal is to be more neutral.
So what you're saying is we need commissars ;)
Like I can't understand this mindset where one says: "its clear the ethics team is getting in the way of us doing the business we could be doing, so its clear they are the problem." Isn't the friction the ethics team produces precisely their raison d'etre?
Like at what point can we all stop pretending that we are angry at woo-woo philosophy experts telling us "no," and admit we really just have a conscious which is at odds with overwhelmingly financial goals of this world.
Obviously, this person operates on different timescales than most people.
I responded to another child comment where someone brought up Chief Risk Officers or Chief Compliance Officers, which I think are a better analogy. I adjusted my opinion based on their comment.
It's not that I think "CDOs don't offer meaningful business value or justification." I just think that, for the most part, setting up the organizational structure like that is ineffective.
Of course, it can be window dressing if the company doesn't actually have product teams care. Just like a CDO position. But you can't patch over a company culture problem with a re-org.
And the ethical AI team doesn’t want to sit back and wait for people to come to them. They want a seat at the table for all important AI products.
More in my comment on the other submission about this news https://news.ycombinator.com/item?id=35146611
As I said, you can't patch over culture problems with a re-org.
If product people don't care, putting a person on the team that works against their interest will not make things better. Same thing with security/diversity/code quality/reliability. If the team doesn't care, putting an enforcer on the team will not fix the situation.
Having some sort of outside check on that seems wise to me.
Same goes for DevOps, QA, UX design, product design, accessiblity etc. If there is no direct instantaneous collaboration while the thing is being built then all that's happening is you're slowing down your product teams while their only course of action is to provide lip service to the thing you want because nobody on the team is actually incentivized to care on top of all of the other shit they need to get done.
Little rules and arrangements from the top can serve to bring out the best in people. In the same way cities and cultures can be shaped by laws and rules so can people within corporations.
What "AI" is at the leadership level is bullshit. It either makes money or it doesn't. The leaders are beholden to shareholders and shareholders in aggregate have the mob mentality of a psychopath with one goal: money.
Right now AI isn't really enough of a threat in terms of ethics. Nobody cares too much yet so if such a group got in the way of profits, of course that group will be executed.
The flip side is that such departments can put the brakes on because not doing risky things is the best way to ensure you don't take risk. If you ask a lawyer if you should do X, the answer is almost certainly going to be a "no" or a "yes" with a lot of "buts and ifs". That's their job and they bias to playing it safe. Same with financial compliance departments. Etc. That's fine if you are in a slow moving line of business where nothing changes much. But when you need to move more quickly, having a lot of internal bureaucracy and box ticking isn't helpful.
With AI, it kind of stopped being academic in the past few years and this is now a multi billion dollar business opportunity that requires companies to act decisively and with purpose. Companies sitting on their hands because it's risky, scary, etc. are going to end up being sidelined. And of course there are some companies going all in on this. Most of these are startups that can afford to take risk. Meaning they are not going to fight with their hands behind their backs taking it carefully, easy, and slowly. Going all in means doing things that are risky, a little bit uncomfortable, and have uncertain outcomes.
We see that playing out with the behavior of MS relative to Google. Google is not launching the AI stuff they have because it might do or say something embarrassing. MS has in fact launched a few AI things that said and did something mildly embarrassing (taybot, and recently the bing chat bot). Very entertaining. But it's fine. They learn from it and move on. Limited damage and it actually got them a lot of positive press. Making mistakes is an essential part of R&D. Having an ethics department trying to slow down things isn't that helpful.
In order to even allow a company to "try to do good", the only possible measures are to change incentives and the environment in which it operates. Regulation and customer choice can change incentives. The environment (such as how people in the job market value money compared to working environment, "doing good", etc.) evolves naturally and is hard to change with purpose (my definition of environment in this context).
I tend to view things like having such company roles a bit more positive. Yes, in most companies such a role cannot achieve almost anything and is created mainly for PR purposes. However, the people filling that role often really care about making a positive impact. The mere fact that having such a role does serve a PR purpose signals slowly moving in that direction. One just shouldn't expect all the people who didn't care before to have a sudden epiphany and become Mother Theresa.
Change is possible and it often happens quickly, ever more quickly these days than in history. Imagine how abolition of slavery in the US south must have felt like. Moving towards "doing good" is hard, painful and prone to fail. I think incentivizing the creation of these PR roles is a lot better than doing nothing. Pretending to care and not caring is hypocritical but it actually still leads to better outcomes than everyone proudly not caring and constructing an ideology about why they shouldn't.
I don't even blame companies for this, they are just optimizing, if public is so stupid that it is buying it then, well, why not?
https://richardhanania.substack.com/p/wokeness-as-saddam-sta...
>>Even after the case has been litigated, we still do not know what exactly Tesla could have done to avoid the $137 million judgment (it may be reduced on appeal). Employees did in fact get in trouble for using the n-word and drawing racist cartoons. In retrospect, they probably would’ve been safer if they just fired everyone who Diaz or Di-az accused of racism, at least the non-blacks, but there’s no guarantee that would’ve worked either, as they were in an industry that required them to rely on workers who hadn’t yet internalized elite ways of thinking about race. If they had fired black people for using the n-word too, would that have helped or hurt them in trying to avoid a lawsuit? It’s difficult to say, and that’s sort of the point.
>>For next time, there’s little Tesla can do but go all in on diversity training, be as enthusiastic as possible in adopting whatever next race fad comes out of academia, and hope that the next jury isn’t as woke as the one they got this time. Until anti-discrimination laws are rolled back, or come to be based on clear and objective standards, there is no other way.
If the goal is to increase minority positions, then you can do this by hiring new officers to replace attrition. This takes a long time and constraining available candidates for key positions is probably frowned upon.
Or you can add a new position, increasing the denominator and numerator but likely hitting your diversity ratio target (ie going from 1:8 existing officers to 2:9 helps in dashboards and whatnot).
And of course, I think these positions do have the potential to improve topics across the organization.
I think it’s the easiest step to make and makes a non-zero positive effect, so are so common.
The position costs money so it could have negative impact. It is interesting that every CDO I’ve met has been in an underrepresented population, most women, but not all. I doubt it’s a fair hiring system, but I still prefer it over the olden days when the diversity council was a bunch of old people all of the same race and gender. I know that shouldn’t matter as long as they delivered on increasing diversity, but it always seemed funny to me.
What should be concerning is that the environment has changed in such a way companies find it more profitable to forego these signals or maybe even use that change itself as a signal (e.g. consider Coinbase making a big show of becoming "apolitical" and gaining recognition from a certain audience for that).
CDOs exist so that the "right" people can have a high-power (even if only via public perception), well-paid C-suite role. These ethics teams exist so that a group of the "right" people can level up their resumes. Instead of being a developer at a bank in Omaha you can be on the AI ethics team at a startup. Instead of being a developer at a startup you can be on the AI ethics team at a FAANG.
If an AI product makes Microsoft look really bad publicly due to something understood by the public as an ethical concern, the ethics team will be back.
Definitely are positions invented by lawyers to provide a safe harbor to minimize corporate legal and PR liabilities while those PR liabilities are burning bright among the public. Practically to the minute that the public focus on DEI is distracted to whatever the “next thing” is you will see these positions start disappearing or absorbed into HR and some new position created to satisfy the optics and create a new safe harbor for the next thing.
The fact that you pre-emptively accept the idea that maximizing diversity to its limits is beneficial to corporations shows that people just take these things at heart because "the people in charge said so". It is not a coincidence that much of the research of the supposed benefits of diversity in businesses comes from business schools in prestigious institutions—HBR, Stanford Business School, etc. As these institutions are one of the most authoritative entities of the structures of power in the West. At some point this bubble must burst.
That’s a straw man argument. Diversity efforts are not about maximizing “to its limits”.
And most research about business come from the same prestigious business schools. Unless you think all research coming them are suspect, I don’t really understand your point.
I literally never said that, nor do I believe it, nor do I quite understand what "maximizing diversity to its limits" means.
My belief primarily comes from working in a range of companies and environments, some that were quite diverse, and some that were really lacking in diversity (and by "lacking in diversity" I mean cases where, in just one example, we had a sizable team with only a single woman), and I've seen that the environments that were lacking in diversity had very specific problems that were a detriment to the business (in addition to just generally sucking for some folks specifically due to that lack of diversity).
Similar challenge faced by CISOs everywhere. Yes, the CISO usually gets to own some little piece (depends on the company) but most of the job is about trying to sweet talk the other parts of the company to care about security without having actual ownership authority to make it happen.
I'd love to know what 'ethical AI' looks like when we exist in a system where the AI is going to replace human workers to the benefit of the employer class.
I have nothing against the technology, but it's a productivity tool that will only benefit a certain set of people.
It's like arguing about the amount of power drawn by electronics chargers in the context of global warming -- paying any attention to such details can suck the air out of the room we need to solve much more pressing issues where we can get much more traction.
It's the same as having "security" outside of the team, it becomes an us Vs them mentality.
How can we work around this new unannounced security/diversity requirement to get our product out of the door in the timescales that have been demanded of us?
I don't mean to start a flame war, but this is a reason why I think "Chief Diversity Officers" are such an ill-conceived notion. It's not that I don't think diversity in corporate environments is extremely valuable, but it's that a Chief Diversity Officer doesn't really "own" any sub-organization of the business. That is, the Chief Product Officer owns the product teams, the Chief Technical Officer owns the engineering teams, the Chief Marketing Officer owns the marketing teams. Even the Chief Human Resources Officer, while largely providing support to other teams, still has sizable staff of their own to get their job down. A Chief Diversity Officer's job largely comes down to telling other teams how they should change or structure their teams. I have yet to see that approach work.
Different teams can certainly work to a common corporate goal, even if that goal isn't the primary focus is a particular team. If you really want to ensure that diversity is valued, any initiatives should actually be led by the teams who also do the main "work of the business", and the goals should be championed by the executives of these teams.
So, I'm the vein of this article, I think independent "ethical AI teams" are a really bad idea. Better to define what ethical AI means at a high leadership level, and if teams need particular experts in these areas they should just hire them as part of that team, not off as a part of some dysfunctional "ombudsman" role.