Re:Stanford Alpaca: An Instruction-following LLaMA model
by ben_w on Monday March 13, 2023 @03:48PM
(#35141414)
Paperclipping is just one example of one of many ways it can go wrong.
While I agree LLMs are unlikely to be the last word on AI, the fact we understand alignment so poorly that they spew random things, let alone any arguments about which words are acceptable[0], is a sign we have much foundational work to do.
Indeed, as I recall, one of the main researchers in this topic describes it as "pre paradigmatic" because we don't have a way to even compare the relative alignment of any two AI.
[0] personally, I suspect but cannot prove that tabooing certain words is a Potemkin village solution to the underlying social problems
While I agree LLMs are unlikely to be the last word on AI, the fact we understand alignment so poorly that they spew random things, let alone any arguments about which words are acceptable[0], is a sign we have much foundational work to do.
Indeed, as I recall, one of the main researchers in this topic describes it as "pre paradigmatic" because we don't have a way to even compare the relative alignment of any two AI.
[0] personally, I suspect but cannot prove that tabooing certain words is a Potemkin village solution to the underlying social problems