> The idea of a superintelligence becoming a bond villain via freelance software jobs (or, let's be honest, OnlyFans scamming) is not something I consider an existential threat. I can't find it anything other than laughable.
Conservative evangelical Christians find evolution laughable.
Finding something laughable is not a good reason to dismiss it as impossible. Indeed, it's probably a good reason to think "What am I so dangerously certain of that I find contradictory ideas comical?"
> Meanwhile, I work with LLMs every day and can only get them to print properly formatted JSON "some" of the time. Get real.
I don't think the current generation of LLMs is anything like AGI, nor an existential risk.
That doesn't mean it's impossible for some future software system to present an existential risk.
It's like you've looked at the Fermi paradox and decided we need Congress to immediately invest in anti-alien defense forces.
It's super-intelligent and it's a super-hacker and it's a super-criminal and it's super-self-replicating and it super-hates-humanity and it's super-uncritical and it's super-goal-oriented and it's super-perfect-at-mimicking-humans and it's super-compute-efficient and it's super-etcetera.
Meanwhile, I work with LLMs every day and can only get them to print properly formatted JSON "some" of the time. Get real.