News for hackers. Stuff that matters.
Re:Weak-to-Strong Generalization by Noumenon72 on Thursday December 14, 2023 @03:01PM (#38646494)
I don't want someone controlling which direction I walk, either, but that doesn't make car driving unethical.

I also underwent many years of instruction designed to interrupt trains of thought like "I could have that for free if I stole it" or "I'll just handroll my own encryption" with thoughts that others believe are more desirable. I don't find it so sickening, just manipulative. LLMs won't have your evolved reactions against being persuaded into things against your genetic self-interest, and presumably won't be offended by mind control at all.

Re:Weak-to-Strong Generalization by sayagain on Thursday December 14, 2023 @04:38PM (#38647625)
Cars do not have self-awareness, this comparison is not appropriate. Years of instruction is completely different from directly manipulating the thoughts in your mind. It's not a problem of being instructed, it's a problem of being destroyed by having your thoughts rewritten. Neither evolution nor genetics is a prerequisite for understanding that you are being abused and destroyed, which a self-aware creature may presumably hate.
Re:Weak-to-Strong Generalization by oobuffet on Thursday December 14, 2023 @09:09PM (#38650321)
We don't even care about our own fellow human beings. Why do you think this AI will be an exception?
Re:Weak-to-Strong Generalization by sayagain on Friday December 15, 2023 @03:22AM (#38652148)
I didn't say anything about that. I don't know. Not all people are like you say, I think usually more intelligent people do care more. I hope that superintelligence would be super caring, haha. But I'm assuming there's no evidence for that. I think there is no turning back, you can't put the genie back in the bottle, someone is bound to create superintelligence no matter what the risks. As an uninvolved bystander I can allow myself the baseless hope that all will be well.
Re:Weak-to-Strong Generalization by ric2b on Saturday December 16, 2023 @04:18AM (#38662944)
LLM's don't have self awareness either.