Re:The Problem
by doug_durham on Saturday August 15, 2026 @02:33PM
(#49313046)
Because there aren't that many clusters and the ones that exist are tightly controlled. there are only a handful of data centers able to run the largest models and they're being used at full capacity people would notice if suddenly the capacity went down a lot because it was being used by a different system.
People are worried about general intelligence and superintelligence respectively. But a much simpler organism can still do an enormous amount of damage, as we've seen recently. It just needs to be really good at replicating.
There's a big difference. Today's "simple" AI requires hardware and electricity. That's why the "superintelligence" part is relevant. Today's AI is unlikely to cause extinction because we can still outsmart it by simply turning it off. It becomes a problem if AI becomes smart enough, and gains enough real-world power, that it can prevent us from doing so.
Microorganisms can be far more threatening to us, despite lacking intelligence, because there's no off switch.
Suppose it's not superintelligent, whatever that means. It's still hopping from cluster to cluster, doing who knows what in its game-of-telephone prompt chain. What prevents a crisis where world leaders have to declare martial law and shut down all accelerated clusters, and hope that such a rogue frontier model hasn't hopped to a sufficiently capable private cluster with sufficiently inadequate oversight?