> No, it can just grow a traditional organization until it's Too Big to Turn Off.
Yeah, this is what I meant by "AI alignment" being inseparable from "corporate alignment".
Re:Uncensored (Language) Models
by jjoonathan on Monday May 15, 2023 @10:47AM
(#35949057)
...which we are notoriously bad at. We don't disagree on the analogy, we disagree on the analogy being a hopeful one.
Yes, exactly. Human organizations can be terrifyingly powerful and awful even with human limitations. Human organizations are guaranteed to have inefficiency from low-bandwidth communication and a million principle-agent-problem fracture points created by the need for delegation, not to mention that power ultimately has to be centralized in slow, squishy, easy-to-kill bodies. AI organizations are not subject to any those limitations. Starting with a terrifying power and removing a bunch of limitations could lead to a very bad place.
> The only realistic path to a global AI threat is a subset of the "nuclear war"
No, it can just grow a traditional organization until it's Too Big to Turn Off.