News for hackers. Stuff that matters.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ianceicys on Friday August 19, 2022 @05:16PM (#32525996)
Artificial general intelligence (AGI) — in other words, systems that could successfully perform any intellectual task that a human can.

Not in my lifetime, not in this millennium. Possibly in the year 2,300.

Weird way to blow $20 million.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by diputsmonro on Friday August 19, 2022 @05:21PM (#32526051)
A society grows great when old men plant trees whose shade they never expect to sit in. Not everything requires an immediate profit incentive to be a good idea.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by tengbretson on Friday August 19, 2022 @05:29PM (#32526153)
A society does not grow great when an old man collects $20 million dollars for the fruit of a tree that he has no capability of planting in the first place.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by icelancer on Friday August 19, 2022 @05:47PM (#32526358)
So sure are you that Carmack can't make inroads here, I wonder where you get the confidence from?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by paxys on Friday August 19, 2022 @05:28PM (#32526150)
$20 million is pretty much nothing when split among a handful of billionaires and the biggest VC firm in the world. Regardless of the project itself it is worth it to spend that money just to have Carmack's name attached to it and buy some future goodwill.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by Ekaros on Friday August 19, 2022 @06:21PM (#32526696)
Never underestimate the greater fool theory. Specially in current tech landscape. It just needs him to produce some results and you could end up selling company to FAANG or some big fund for profit.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by willio58 on Friday August 19, 2022 @05:23PM (#32526081)
It’s not blowing 20 million if it results in meaningful progress in this area. We have something like 2700 billionaires on this planet. This isn’t even a drop in the bucket for someone like that interested in furthering this research.

AGI could quite literally shift any job to automation. This is human-experience changing stuff.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by blibble on Friday August 19, 2022 @05:25PM (#32526102)
> This is human-experience changing stuff.

that's one way of putting it

it will remove the need for the vast majority of the population, which will end extremely badly

Re:John Carmack's new AGI company, Keen Technologies, has ra... by gizajob on Friday August 19, 2022 @05:27PM (#32526126)
But by the same token, there's no need for billions of humans now. AGI isn't really going to change that except for making work even more superfluous than it already is.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by Jensson on Friday August 19, 2022 @05:35PM (#32526225)
Currently the life of leaders gets better the more people they can control, since it creates a larger tax base. That means leaders tries to encourage population increase, they want more immigration, encourages people to multiply and sees population reduction as harmful.

With AGI that is no longer true, they can just replace most people with computers and automated combat drones while they keep a small number of personal servants to look after them. Currently most jobs either is there to support other humans or can be replaced by a computer, remove the need for humans and all of those jobs just disappear and leaders no longer care about having lots of people around.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by ge96 on Friday August 19, 2022 @05:30PM (#32526165)
I wonder about this, if you had great/true automation, free energy from the sun, is there any need to do anything. As in value of money.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by Arcuru on Friday August 19, 2022 @05:37PM (#32526258)
If you want to look into it more, that situation is usually called a post-scarcity economy[1]. It's talked about and depicted in a few fictionalized places, including Star Trek.

[1] - https://en.wikipedia.org/wiki/Post-scarcity_economy

Re:John Carmack's new AGI company, Keen Technologies, has ra... by blibble on Friday August 19, 2022 @06:00PM (#32526485)
in Star Trek: the Federation has unlimited energy and the ability to replicate most forms of matter

but human(oid) intelligence is still scarce, and they don't have AGI (other than Data)

there is however a society that has no need for humanoid intelligence, and that's the Dominion

and I suspect that is what our society would turn into if AGI is invented (and not the Federation)

Re:John Carmack's new AGI company, Keen Technologies, has ra... by SketchySeaBeast on Friday August 19, 2022 @05:50PM (#32526386)
But who would own the automatons and power generators, and what would be their impetus to share their power? Unless the means of (energy) production moved out of the hands of the few it seems like it wouldn't make the rest of our lives any more idyllic.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ge96 on Friday August 19, 2022 @06:18PM (#32526664)
Yeah it's true. When I donate/help I always feel this "mine". I believe in merit, you know, effort in effort out. It's nice to help people but there are also too many... and bad actors. So idk if it'll ever happen or just for select few anyway.

I almost regret being at this phase of life where we are aware of what's possible but we most likely not see it in our lifetime. This AGI talk, colonization of space, etc... but can strive towards it/have fun trying in the meantime.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by omg_stoppit on Friday August 19, 2022 @05:29PM (#32526158)
And as societies progress, they must either realize why basic necessities like Universal Basic Income exist, or just allow for large swathes of their population to die off.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by barrysteve on Friday August 19, 2022 @11:13PM (#32528689)
Whatever you automate and is truly useful becomes the next hammer, circular saw or vice grips. The human experience which is being changed, is creating better tools for the next generation.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by hervature on Friday August 19, 2022 @05:29PM (#32526155)
2,300 is in this millennium?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by freediver on Friday August 19, 2022 @05:19PM (#32526032)
You are probably right, but if anyone can make a dent Carmak is the person.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by vlunkr on Friday August 19, 2022 @05:21PM (#32526055)
Do game dev skills transfer to AGI? I know he's a smart guy, but I don't think that's a given.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by sowbug on Friday August 19, 2022 @10:26PM (#32528427)
Read Michael Abrash's Graphics Programming Black Book for the story of how the original Quake came to life. You'll get an appreciation for John Carmack's ability to thoroughly research widely varying solutions to a problem, quickly create production-quality implementations of the promising ones, and even more quickly abandon the dead ends. The result is this almost boring, seemingly linear progression toward a final product that seems obvious in hindsight, yet it represents a leap forward the way Quake did in the mid-1990s compared to other FPSes at the time. I don't know of other public stories of individual engineers who can span both the very cutting edge of research and the practicalities of shipping real commercial software.

https://github.com/jagregory/abrash-black-book

Re:John Carmack's new AGI company, Keen Technologies, has ra... by zaptrem on Friday August 19, 2022 @05:25PM (#32526114)
He's not just a game dev, he is one of the most legendary graphics programmers (and just programmers) alive. Similar to how GPUs transferred well from gaming to ML, it seems like much of the math and parallel/efficiency-focused thinking of graphics programing is useful in ML.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by gizajob on Friday August 19, 2022 @05:27PM (#32526130)
Worked for Demis Hassabis
Re:John Carmack's new AGI company, Keen Technologies, has ra... by 5d8767c68926 on Friday August 19, 2022 @05:26PM (#32526118)
If he succeeds, his skillet becomes the Platonic ideal of an AGI developer.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by mda on Friday August 19, 2022 @06:10PM (#32526582)
skillet? well I for one welcome our new kitchen utensil overlords.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ggm on Friday August 19, 2022 @05:29PM (#32526154)
He'll make it back on small increments with high value. If he can shave 30% LOC on a vision system for small BoM in some context like self driving cars, 10x stake is coming his way.

Basically, they could completely fail to advance AGI (and I think this is what will happen btw, like you) and make gigabucks.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by jacquesm on Friday August 19, 2022 @05:38PM (#32526269)
The year 2300 is definitely in this millennium.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by dmoy on Friday August 19, 2022 @05:43PM (#32526318)
Only if you don't count the second dark age of 1200 years that fit between 2093 and 2094
Re:John Carmack's new AGI company, Keen Technologies, has ra... by gfodor on Friday August 19, 2022 @05:20PM (#32526045)
You have some catching up to do. Consensus is dropping to this lifetime for sure, if not this decade.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by efficax on Friday August 19, 2022 @05:21PM (#32526050)
what consensus? i think most researchers remain skeptical
Re:John Carmack's new AGI company, Keen Technologies, has ra... by Isinlor on Friday August 19, 2022 @05:34PM (#32526219)
The survey [0], fielded in late 2019 (before GPT-3, Chinchilla, Flamingo, PaLM, Codex, Dall-E, Minerva etc.), elicited forecasts for near-term AI development milestones and high- or human-level machine intelligence, defined as when machines are able to accomplish every or almost every task humans are able to do currently. They sample 296 researchers who presented at two important AI/ML conferences ICML and NeurIPS. Results from their 2019 survey show that, in aggregate, AI/ML researchers surveyed placed a 50% likelihood of human-level machine intelligence being achieved by 2060. The results show researchers newly contacted in 2019 expressed similar beliefs about the progress of advanced AI as respondents in the Grace et al. (2018) survey.

[0] https://arxiv.org/abs/2206.04132

Re:John Carmack's new AGI company, Keen Technologies, has ra... by semi-extrinsic on Friday August 19, 2022 @05:26PM (#32526116)
Yeah, I don't think there is even any agreement about what criteria a "minimal AGI" would need to meet. If we can't even define what the thing is, saying we'll have it within ten years is pure hubris.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by bpodgursky on Friday August 19, 2022 @05:22PM (#32526065)
Uh... no. Most researchers have moved their timelines to somewhere between 2030 and 2040.

You can argue they're wrong, but there is absolutely a general consensus that AGI is going to be this generation.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by _delirium on Friday August 19, 2022 @05:36PM (#32526243)
Who do you have in mind? In my corner of AI it's pretty uncommon for researchers to even predict "timelines". Predictions have a bad track record in the field and most researchers know it, so don't like to go on record making them. The only prominent AI researcher I know who has made a bunch of predictions with dates is Rodney Brooks [1], and he puts even dog-level general intelligence as "not earlier than 2048". I imagine folks like LeCun or Hinton are more optimistic, but as far as I'm aware they haven't wanted to make specific predictions with dates like that (and LeCun doesn't like the term "AGI", because he doesn't think "general intelligence" exists even for humans).

[1] https://rodneybrooks.com/my-dated-predictions/

Re:John Carmack's new AGI company, Keen Technologies, has ra... by gizajob on Friday August 19, 2022 @05:28PM (#32526143)
AGI has been 20-30 years away for some 70 years now...
Re:John Carmack's new AGI company, Keen Technologies, has ra... by Isinlor on Friday August 19, 2022 @05:47PM (#32526351)
Kurzweil in 2002 made $20,000 bet that a difficult, well defined 2h version of Turing test will by passed by 2029.

https://longbets.org/1/

Given development in language models in the last 2 years he may have a decent chance at winning that bet.

People give him 65% chance [0] and by now there are only 7 years left.

[0] https://www.metaculus.com/questions/3648/computer-passes-tur...

Re:John Carmack's new AGI company, Keen Technologies, has ra... by TaupeRanger on Friday August 19, 2022 @06:10PM (#32526591)
Sure...just like there was during the last episode of AI hype a generation ago.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by xen2xen1 on Friday August 19, 2022 @05:27PM (#32526132)
And consensus is never wrong!
Re:John Carmack's new AGI company, Keen Technologies, has ra... by SketchySeaBeast on Friday August 19, 2022 @05:41PM (#32526302)
Especially assertions of consensus provided without evidence of said consensus.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by gfodor on Friday August 19, 2022 @08:45PM (#32527866)
I don’t know that the consensus is right but the OP’s confidence of it being far off made me think they were unaware of the recent shifts (not just in consensus, but the upstream capabilities.)
Re:John Carmack's new AGI company, Keen Technologies, has ra... by makeee on Friday August 19, 2022 @05:18PM (#32526022)
Doesn't imply any task, just a wide variety of tasks. 10 years at most.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by jtwaleson on Friday August 19, 2022 @05:29PM (#32526159)
My takeaway from the Lex Fridman interview is of someone that’s machine-like in his approach. AGI suddenly seemed simpler and within reach. Skipping consciousness and qualia. It’s inhumane, but machine-like and effective. Curious what will become of it.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by bsenftner on Friday August 19, 2022 @05:36PM (#32526245)
I believe AGI is the threshold where generalized artificial comprehension is achieved and the model can understand any task. Once the understanding part is composable the building portion is following the understanding. I'm using understanding rather than model because our models we make today are not these kinds of comprehensions, understandings are more intelligent.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by arkitaip on Friday August 19, 2022 @05:24PM (#32526093)
By definition it has to be any task otherwise it wouldn't be general. What tasks wouldn't an AGI be able to perform and still be an AGI?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by nomel on Friday August 19, 2022 @05:27PM (#32526129)
It sounds like you may be demanding more from AGI that we do of humans. AGI is a mushy concept, not a hard requirement. "Any task" is definitely not required for a low functioning AGI, just as it's not a requirement for a low functioning human, who still easily fits the definition of an intelligent being.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by yeellow on Friday August 19, 2022 @05:30PM (#32526167)
For each human being having GI there are many tasks that person won't be able to perform. For example proving math theorems, doing research in physics, writing a poem, etc. Specyfic AGI could have its limitations as well.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by dymk on Friday August 19, 2022 @05:26PM (#32526124)
Reliably trick a humans into thinking it's a human. That's it.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by mod on Friday August 19, 2022 @06:10PM (#32526584)
I believe that's the Turing Test, not necessarily a definition (or requirement) for AGI.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by dymk on Friday August 19, 2022 @07:19PM (#32527274)
Where’s AGI defined? I’ve only seen it used in the context of “can pass the Turing test”
Re:John Carmack's new AGI company, Keen Technologies, has ra... by dinosaurdynasty on Friday August 19, 2022 @08:51PM (#32527916)
AGI isn't really explicitly defined (it wouldn't be a problem if it was) but it's essentially an artificial version of the "general" intelligence that humans have, aka we can do and learn many things we were not programmed or trained by evolution to do
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ianceicys on Friday August 19, 2022 @05:23PM (#32526079)
Then it's NOT generalized. ANY means ANY.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by dymk on Friday August 19, 2022 @05:25PM (#32526109)
Can you do any task asked of you, which could be asked of a human being? ANY task.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ianceicys on Friday August 19, 2022 @05:30PM (#32526172)
I may not be able to ANY task sufficiently well (ex Calculus, Poetry, Emotion), but by the very definition of being a Human I can do *any* Human task.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by dymk on Friday August 19, 2022 @05:50PM (#32526384)
With specific training, sure. Why are we holding an AI to a higher standard?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by kmnc on Friday August 19, 2022 @05:41PM (#32526299)
If the task is possible… then why not?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by dymk on Friday August 19, 2022 @05:49PM (#32526380)
What if you don't know how to complete the task?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by Ekaros on Friday August 19, 2022 @06:20PM (#32526675)
20 million doesn't actually sound in anyway stupid investment with name like Carmack involved. Just have the company produce something and then flip it to next idiot...
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ZephyrBlu on Friday August 19, 2022 @05:23PM (#32526072)
If I'm remembering right, Carmack believes AGI will be a thing by 2030. He said this in his recent interview with Lex Fridman.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by fancy_pantser on Friday August 19, 2022 @05:32PM (#32526195)
It's a long interview, here's just the bit focused on AGI: https://www.youtube.com/watch?v=xLi83prR5fg
Re:John Carmack's new AGI company, Keen Technologies, has ra... by mupuff1234 on Friday August 19, 2022 @05:40PM (#32526295)
But I think something of the level of a 6 year old, not so much a super being.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by nomel on Friday August 19, 2022 @05:25PM (#32526105)
From what I remember, his definition of AGI didn't include an average IQ, which it shouldn't.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by systemvoltage on Friday August 19, 2022 @05:23PM (#32526085)
Weestern civilization would be dead if it weren't for eccentric people like this. Let them blow $20M, there are worse ways.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by TrainedMonkey on Friday August 19, 2022 @06:28PM (#32526785)
Do you have a rationale for that? I get a feeling progress in both machine learning and understanding biological intelligence is fairly rapid and has been accelerating. I believe two primary contributing factors are cheaper compute and vast amount of investment poured into machine learning, see https://venturebeat.com/ai/report-ai-investments-see-largest...

Now, the question of whether we are going to have AGI is incredibly broad. So I am going to split it into two smaller ones: - Are we going to have enough compute by year X to implement AGI. Note that we are not talking about super intelligence or singularity here. This AGI might be below human intelligence and incredibly uneconomical to run. - Assuming we have enough compute, will we a way to get AGI working.

The compute advancements scale with new Chip Fabs linearly and tech node improvements exponentially. I think it is reasonable for compute to get cheaper and more accessible throughout at least 2030. I expect this because TSMC is starting 3nm node production, Intel is decoupling fabing and chip design (aka TSMC model), and the strategic investments into into chap manufacturing driven by supply chain disruptions. See https://www.tomshardware.com/news/tsmc-initiates-3nm-chips-p...

How much compute do we need? This is hard to estimate, but amount of human connections in human brain is estimated at 100 trillion, that is 1e14. Current largest model has 530B parameters, that is 5.3e11: https://developer.nvidia.com/blog/using-deepspeed-and-megatr... . That is factor of 500 or 9 doublings off. To get there by 2040 we would need a doubling every 2 years. This is slower that recent progress, but past performance does not predict future results. Still, I believe getting models with 1e14 parameters by 2040 is possible for tech giants. I believe it is likely that a model with 1e14 parameters is sufficient for AGI if we know how structure and train it.

Will we know how to structure and train it? I think is mostly driven by investment into the AI field. More money means more people and given the venture beat link above the investment seems to be accelerating. A lot of that investment will be unprofitable, but we are not looking to make a profit - we are looking for breakthroughs and larger model sizes. Self-driving, stock trading, and voice controls are machine learning applications which are currently deployed in the real world. At the very least it is reasonable to expect continuous investment to improve those applications.

Based on the above I believe we would need to mess things up royally to not get AGI by 2100. Remember this could be below human and super uneconomical AGI. I am rather optimistic, so my personal prediction is that we have 50% chance to get AGI by 2040 and 5-10% chance of getting there by 2030.