Re:"The Traitors", a reality TV show, offers a useful econom...
by UniverseHacker on Sunday January 19, 2025 @04:03PM
(#42761787)
Yeah, I think it is literally provably 'optimal' if you can execute it correctly with informative data, don't forget omit any important considerations, and aren't just making up BS- all of which are almost always impossible for regular humans in real life no matter how much 'rationality training' they've had. It makes sense both for optimal behavior of some hypothetical superintelligent AI to realize its own goals efficiently, or for something like a government to weigh pros and cons of a difficult regulatory choice with well defined short term consequences - neither of which are anything like the everyday morality decisions humans make.
It’s the usual GIGO problem. These arguments almost always start with a bunch of completely made-up numbers. It doesn’t matter how good the math is, the results will be useless.
It can work. When a government regulator decides whether to mandate some new safety equipment and after rigorous technical analysis concludes that it would result in net lives lost and so doesn’t require it, that’s sensible. But thats not what happens here.
I occasionally see this problem acknowledged, but even then, the given error bars are way too small and then it’s just full steam ahead anyway.
It could be dangerous anyway, but this makes it even more so.