News for hackers. Stuff that matters.
Re:Microsoft lays off an ethical AI team as it doubles down ... by dragonwriter on Monday March 13, 2023 @11:44PM (#35147183)
> AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion.

I don’t think that’s really true. Rather, the faction that sees alignment as a subordinate concern within ethics hates the faction that sees ethics as a subordinate concern within alignment.

> The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks.

Not really; the alignment-priority branch has no problem recognizing (certain subsets of) the bias and other issues that are the more central focus of the ethics branch as real and significant, but they view alignment as solving them and view narrow centralized control as a mitigation (for both “alignment” and “ethics” types of issues) until it is solved. They also view progress on AI as a general priority, for a mix of commercial, ideological, and other [0] reasons, and their own progress on AI in particular as essential (because they view AI from others – either because potentially unaligned, or for other reasons – as a critical and potentially existential threat.)

> The ethics branch (responsible for all the media attention about racist AIs/bias etc) view alignment people as pie in the sky researchers whose results are irrelevant to society.

No, they view them as an enormous threat, because they are working in a way that not only insufficiently mitigates in basic approach because of a lack of priority, but also magnifies by its focus on closely concentrated control, the issues of paramount concern to the ethics-focused.

> It seems increasingly that the concerns from the ethics branch have not come to pass at all.

They have in fact come to pass in deployments of AI in all kinds of socially-significant roles. E.g, the facial recognition in the CBP One app newly mandated for asylum seekers doesn’t work well for dark-skinned faces (racial bias in facial image recognition is literally one of the earliest publicly recognized AI bias problems that motivated the AI ethics movement.) They’ve also manifested in systems deployed in welfare management [1], hiring [2], and all kinds of other areas.

[0] e.g., Roko’s Basilisk

[1] https://www.wired.com/story/welfare-algorithms-discriminatio...

[2] https://www.reuters.com/article/us-amazon-com-jobs-automatio...

Re:Microsoft lays off an ethical AI team as it doubles down ... by FeepingCreature on Tuesday March 14, 2023 @05:55AM (#35149254)
Note that if you see someone bring up Roko, it's overwhelmingly somebody arguing against alignment researchers; its use is primarily as "look at this crazy thing these people believe."
Re:Microsoft lays off an ethical AI team as it doubles down ... by hgsgm on Tuesday March 14, 2023 @09:23AM (#35151051)
That's because the rationalist/alignment people's belief system leads to a logical conclusion that talking about Roko's Basilisk harms the person you tell, and even rationalists have a human heart interfering with their extreme ultilitarianism.
Re:Microsoft lays off an ethical AI team as it doubles down ... by FeepingCreature on Tuesday March 14, 2023 @09:50AM (#35151479)
I mean, it's also because no rationalists take Roko's Basilisk intellectually seriously nowadays. There was a period of maybe a few months in 2010 where people were genuinely worried about it, and then they gradually figured out the logical holes in it. Any spread after that has been from outsiders who were fascinated with the idea. Nowadays it mostly persists as a trap for people with schizoaffective disorders.

(Disclaimer: my view is of the forum itself; I don't know how long it persisted in SF circles.)