News for hackers. Stuff that matters.
Re:Details emerge of surprise board coup that ousted CEO Sam... by T-A on Saturday November 18, 2023 @03:54PM (#38324297)
The scenario usually bandied about is AGI self-improving at an accelerating rate: once you cross the threshold to self-improvement, you quickly get superintelligence with God-like powers beyond human comprehension (a.k.a. the Singularity) as AGI v1 creates a faster AGI v2 which creates a faster AGI v3 etc.

Any AI researchers still plodding along at mere human speed are then doomed: they won't be able to catch up even if they manage to reproduce the original breakthrough, since the head start enjoyed by AGI #1 guarantees that its latest iteration is always further along the exponential self-improvement curve and therefore superior to any would-be competitor. Being rational(ists), they give up and welcome their new AI overlord.

And if not, the AI god will surely make them see the error of their ways.

Re:Details emerge of surprise board coup that ousted CEO Sam... by anonymouskimmer on Saturday November 18, 2023 @07:53PM (#38327005)
It seems to me that non-General AI would typically outcompete AGI, all else held equal. In such a scenario even a first-past-the-post AGI would have trouble becoming an overload if non-Generalized AIs were marshaled against it.
Re:Details emerge of surprise board coup that ousted CEO Sam... by Aerbil313 on Sunday November 19, 2023 @08:50AM (#38332608)
This makes no sense at all.
Re:Details emerge of surprise board coup that ousted CEO Sam... by anonymouskimmer on Sunday November 19, 2023 @12:28PM (#38335017)
Re:Details emerge of surprise board coup that ousted CEO Sam... by veidr on Sunday November 19, 2023 @11:55AM (#38334566)
uhm, wat?
Re:Details emerge of surprise board coup that ousted CEO Sam... by anonymouskimmer on Sunday November 19, 2023 @12:27PM (#38335012)
This is just the thesis that paperclip optimizers win over general intelligence, because they optimize.
Re:Details emerge of surprise board coup that ousted CEO Sam... by true_religion on Saturday November 18, 2023 @08:42PM (#38327397)
What if AI self improvement is not exponential?

We assume a self improving AI will lead to some runaway intelligence improvement but if it grows at 1% per year or even per month that’s something we can adapt to.

Re:Details emerge of surprise board coup that ousted CEO Sam... by devbent on Sunday November 19, 2023 @01:58AM (#38329944)
Assume the AGI has access to a credit card and it goes ahead and reserves itself every GPU cycle in existence so it's 1 month is turned into a day, and now we're back to being fucked.

Maybe an ongoing GPU shortage is the only thing that'll save us!

Re:Details emerge of surprise board coup that ousted CEO Sam... by sicariusnoctis on Sunday November 19, 2023 @03:45AM (#38330569)
How would an AGI gain access to an unlimited credit card that immediately gives it remote access to all GPUs in the world?
Re:Details emerge of surprise board coup that ousted CEO Sam... by JetSpiegel on Sunday November 19, 2023 @11:30AM (#38334182)
It could hack into NVIDIA and AMD, compromise their firmware build machines silently, then publish a GPU vulnerability that required firmware updates.

After a couple months, turn on the backdoor.

Re:Details emerge of surprise board coup that ousted CEO Sam... by veidr on Sunday November 19, 2023 @11:52AM (#38334526)
E.g. by convincing 35% of this website's users to "subscribe" to its "service"?

¯\_(ಠ_ಠ)_/¯