Re:Meta disbanded its Responsible AI team
by 0xDEAFBEAD on Sunday November 19, 2023 @07:02PM
(#38340107)
Read the mesa-optimization paper I linked elsewhere in this thread: https://arxiv.org/pdf/1906.01820.pdf Eliezer's point is that if AI researchers aren't looking for anomalous behavior that could indicate a potential danger, they won't find it.
[ Parent
| on HN ]
Re:Meta disbanded its Responsible AI team
by staticman2 on Sunday November 19, 2023 @09:23PM
(#38341404)
The issue isn't whether his point is correct. If I said people should safety test the space shuttle to make sure the phlogiston isn't going to overheat, I may be correct in my point that people should "safety test" the space shuttle but I'm still a crank because phlogeston isn't a real thing.
Re:Meta disbanded its Responsible AI team
by 0xDEAFBEAD on Sunday November 19, 2023 @10:28PM
(#38341924)
See my comment here: https://news.ycombinator.com/item?id=38336374
The reason AI alignment is challenging is because we're trying to make accurate predictions about unusual scenarios that we have essentially zero data about. No one can credibly claim expertise on what would constitute evidence of a worrisome anomaly. Jeremy Howard can't credibly say that a sudden drop in the loss function is certainly nothing to worry about, because the entire idea is to think about exotic situations that don't arise in the course of ordinary machine learning work. And the "loss" vs "loss function" thing is just silly gatekeeping, I worked in ML for years -- serious people generally don't care about minor terminology stuff like that.
Re:Meta disbanded its Responsible AI team
by staticman2 on Monday November 20, 2023 @01:22AM
(#38343330)
That's not what the conversation was about- you're just doing the thing Howard said where you squint and imagine he was saying something other than he did.