News for hackers. Stuff that matters.
Re:The Darwinian argument for worrying about AI by noduerme on Friday June 30, 2023 @12:31PM (#36537939)
A lot of "dumb" systems we develop require unimaginable resource inputs just to produce a little extra output. Strip mining for coal, or using chemical fertilizer to grow corn to produce ethanol, for instance. There is no guarantee these days that a system will fail just because it requires huge energy inputs to produce marginal profit.

Evolutionary goals are not something that has to be aligned. Evolution isn't specific to organic life, it's an intrinsic rule of self-organizing systems. Viruses aren't alive, but they evolve. A clever stitch of self-writing code on a Pi attached to someone's TV may evolve without knowing or intending to.

What makes this dangerous now is the vast amount of energy input toward specific systems. Saying "there's surely not enough energy available for it to..." is false comfort. It's underrating the process of evolution.

As a poker analogy, if you just called the guy across from you because you think he couldn't possibly have more than two pair, you're wrong. You've bet into a full house.

[edit: Upon review, I think I've unintentionally gone 100% Jeff Goldblum Jurassic Park in this response LOL]

Re:The Darwinian argument for worrying about AI by roenxi on Friday June 30, 2023 @07:06PM (#36543781)
I don't think that completely addresses gp's argument, he wasn't talking about energy or resourcing; just complexity.

That being said, I still think gp's argument is flawed. We don't have examples of complexity that an AI can't overcome. Up until 2016 Go was a great example that was far to complex for an AI. Now there is no problem that an AI isn't expected, in principle, to outperform humans in with engineering effort. It is just a question of figuring out which problems it needs to outperform us in.

Engineering effort is not in shortage. There are many capable people on hand and modern AI is going to be a perfect force multiplier for engineering work. We're going to figure it out. Complexity is maybe a good argument that we've got a few generations to go, but life for a 2nd tier intelligence is not a well-resourced one. It could easily be like this world except with humans displacing orangutans in 2nd place to the smart species.

Re:The Darwinian argument for worrying about AI by ChatGTP on Friday June 30, 2023 @07:30PM (#36544058)
While I mostly agree with you , doesn’t AlphaGo does have problems with complexity which is how it was beaten quite easily in recent times ?
Re:The Darwinian argument for worrying about AI by roenxi on Friday June 30, 2023 @08:15PM (#36544542)
I think that might have been KataGo rather than AlphaGo, but either way I wouldn't count on that sort of attack being successful in the long term. We haven't even had superhuman Go AI for a decade; the researchers are done with the basic algorithms yet. The current approach is basically random play weighted by win probability and I don't think they've really finished bringing in novel search techniques to explore the game space (pun intended, there'll probably be something big to find from things like http://eplex.cs.ucf.edu/noveltysearch/).

The fact that AI today isn't superhuman in every aspect is no real comfort. Post-AlphaGo humans don't seem to have any advantage that can be pointed to apart from having existed for more time. And computers move forward quite quickly.

Re:The Darwinian argument for worrying about AI by thfuran on Friday June 30, 2023 @11:00PM (#36545982)
Brains have been around for half a billion years. I'd say AI is putting on a pretty good show the last decade or so, considering.
Re:The Darwinian argument for worrying about AI by ChatGTP on Friday June 30, 2023 @11:24PM (#36546135)
Is there supposed to be major difference between KataGo and AlphaGo? Seems you’re arguing semantics there.
Re:The Darwinian argument for worrying about AI by roenxi on Friday June 30, 2023 @11:34PM (#36546203)
No significant difference, for our purposes they are more or less the same thing. But KataGo is an open source project out of Jane Street Capital and probably a bit stronger than AlphaGo (which was a Google project that was never released AFAIK). KataGo is also quite a bit bigger than AlphaGo in terms of popularity because people can actually play around with it.

It is like the BSD Kernel vs Linux - in many ways the differences are minor but they are different projects.