News for hackers. Stuff that matters.
Re:John Carmack on AI by fnordpiglet on Tuesday October 03, 2023 @07:25PM (#37758927)
I think the most likely way to achieve “AI alignment” is the peasants with crossbows. Everyone having a powerful AI at their fingertips provides a balancing factor that is only overcome if those tools are in the hands of very few. Trusting Google and various governments to “do the right thing” is absurd - even assuming good behavior and intention on their part, they just aren’t nearly as competent as people seem to assume. Everyone having access everywhere all at once provides the transparency and space to innovate, but also to counterbalance the use of AI in dark ways.

I fundamentally don’t like the crossbow and guns and second amendment argument. AI isn’t a tool to murder with, having no other function than the death of your neighbor. I feel like it’s more like the “should you be required to license your seeds from a megacorp” type thing. Food is existentially important, and it could be weaponized. But it’s not fundamentally a weapon, it’s fundamentally a thing of utility.

Re:John Carmack on AI by coffeebeqn on Tuesday October 03, 2023 @07:57PM (#37759156)
A finite resource fairly allocated to all? Which civilization are we talking about again? Certainly not humans.

Also a billion or trillion or infinite ants can’t outsmart a single human. So unless some hard cap is enforced (how?) then mini AIs will probably not do anything

Re:John Carmack on AI by czl on Wednesday October 04, 2023 @03:38AM (#37762056)
> Also a billion or trillion or infinite ants can’t outsmart a single human.

Cells are simpler than ants so can trillion or infinite cells outsmart a single human? Why not? How many cells is a single human?

Re:John Carmack on AI by kig on Wednesday October 04, 2023 @06:38AM (#37763235)
Around 37 trillion cells, of which 25 trillion are red blood cells. Plus a 100 trillion gut bacteria.

The brain is about 0.1 trillion cells, of which 0.02 trillion cells are the cerebral cortex, which is occasionally consulted by the rest of the brain on matters such as what words to enter into this textarea.

Re:John Carmack on AI by trident5000 on Tuesday October 03, 2023 @08:16PM (#37759344)
We definitely got a small taste of what people do with their time when they are not at work during Covid...and it wasnt pretty.
Re:John Carmack on AI by Cthulhu_ on Wednesday October 04, 2023 @04:38AM (#37762498)
Not pretty for who? Can you explain?
Re:John Carmack on AI by trident5000 on Wednesday October 04, 2023 @09:56PM (#37774086)
The summer riots that were fueled by people not needing to be anywhere.
Re:John Carmack on AI by TheRoque on Tuesday October 03, 2023 @08:28PM (#37759433)
Well somehow I trust google more on having my data, than anyone in the world. I prefer the knowledge to be in a known identifiable entity that can be accounted for, than run in the wild and be used by anyone with bad intention (once again, regarding my data). So this is not about data, but AI, but I think it still applies.
Re:John Carmack on AI by hackernewds on Wednesday October 04, 2023 @03:29AM (#37761981)
this is how I schedule my daily life. it is inevitable that I use software services. I'd like to keep them limited to one megalith, and that is the Google ecosystem. at least their biz model is not predatory, where using my individual data exceeds the risk of doing so.
Re:John Carmack on AI by zelphirkalt on Wednesday October 04, 2023 @04:33AM (#37762464)
> I prefer the knowledge to be in a known identifiable entity that can be accounted for […]

Google can be accounted for? As in "I know the entity that has the data." or "can be held accountable" or "will be held accountable"?

Re:John Carmack on AI by hackernewds on Wednesday October 04, 2023 @03:27AM (#37761968)
If you replace "peasants with arrows", with "good guy with a gun" I think the whole argument rings a different tone..
Re:John Carmack on AI by iudqnolq on Wednesday October 04, 2023 @03:52AM (#37762149)
It's less "good guy with a gun" and more "some random country with a small to medium army". Access to violence doesn't democratize on the individual level, but as the parent comment alludes to weapons changes allowed peasant armies to defeat trained knights, changing the structure of society.

You personally might not be able to exploit open AI fully, but there are lots of countries and orgs that could. Remember millions of dollars is loose change in the defense context.

Re:John Carmack on AI by otabdeveloper4 on Wednesday October 04, 2023 @04:07AM (#37762243)
> powerful AI

AI is not powerful. AI doesn't increase information complexity or create new information. It actually does the opposite - it averages out the complexity peaks.

In effect, AI isn't intelligence, it dumbs everything down and reduces information to a common-denominator baseline.

(In the future the real difference between the "haves" and the "have nots" will be those who can understand and deal with information complexity and those who can't. Creating information complexity is the only thing humans are naturally good at, it's our biological imperative.)

Re:John Carmack on AI by kris_lujanovic on Wednesday October 04, 2023 @05:13AM (#37762728)
AI is not here to dumb down anything but automate tasks and find and reveal complexity esp hidden complexity. Humans are limited by their short and long term memory and bandwidth esp in modern digital environment.

I would still rate opportunity gap as higher priority over understanding complexity. its still funny when people try to be digital nomads just to discover geopolitics or time zones

Re:John Carmack on AI by throwbmw on Wednesday October 04, 2023 @07:53AM (#37763845)
Any further reading recommendations on this topic?
Re:John Carmack on AI by marcus_holmes on Wednesday October 04, 2023 @04:54AM (#37762608)
I would prefer a "peasants and longbows" analogy; any country that trained its people in AI would become super effective but would have to keep its peasants very happy.
Re:John Carmack on AI by hollerith on Friday October 06, 2023 @02:39PM (#37794515)
>I think the most likely way to achieve “AI alignment” is the peasants with crossbows. Everyone having a powerful AI at their fingertips provides a balancing factor that is only overcome if those tools are in the hands of very few.

I think neither elite actors like Sam Altman nor the peasants will manage to get their AI to do what they want it to do once the AI becomes knowledgeable enough and good enough at planning to make plans resistant to determined human opposition.

I.e., I think inequality is a distraction from the main problem with frontier AI research.