News for hackers. Stuff that matters.
Re:Greg (2017) by shrimpx on Saturday November 18, 2023 @05:12PM (#38325135)
Near-100% certainty that Altman and Brockman cofound a new AI company in the coming days. The question is will they be able to recruit a team that can actually build competitive models? Ilya Sutksevers don't grow on trees. Maybe they'll just get a team good enough to specialize Llama2, since Altman/Brockman seem to think what's lacking in this space is glitzy products, app stores, b2b integrations, etc. Maybe OpenAI starts to open source everything and Altman/Brockman can have their cake and eat it, too.
Re:Greg (2017) by drexlspivey on Saturday November 18, 2023 @05:33PM (#38325381)
It will be very ironic when their new startup gets dragged down by regulations due to them not having an AI license that sama pushed heavily in congress.
Re:Greg (2017) by willsmith72 on Saturday November 18, 2023 @05:52PM (#38325562)
> Maybe OpenAI starts to open source everything

I really doubt it

Re:Greg (2017) by healthyusa on Saturday November 18, 2023 @06:00PM (#38325648)
Sure. They just call Sundar.

Yo, bro what up. Want to grab lunch?

Re:Greg (2017) by thatsadude on Saturday November 18, 2023 @06:45PM (#38326230)
Microsoft would be horrified.
Re:Greg (2017) by shrimpx on Saturday November 18, 2023 @08:07PM (#38327126)
Would they tho? Altman and Brockman are not AI people, they’re business guys.
Re:Greg (2017) by peanuty1 on Saturday November 18, 2023 @11:42PM (#38329053)
Brockman is also a software engineer. My understanding is that he spends most of his time on software problems.
Re:Greg (2017) by borski on Sunday November 19, 2023 @02:52AM (#38330239)
gdb is easily one of the most intelligent engineers I've ever had the pleasure of interacting with.
Re:Greg (2017) by loveparade on Saturday November 18, 2023 @10:17PM (#38328350)
You're massively overestimating the skills needed to build "competitive models". Researchers like Ilya were essential several years ago when OpenAI got started, because at that time nobody even knew what to build or how to go about it. It was all research, and OpenAI was solely focused on RL. But these days, LLM code is 95%+ data and infrastructure engineering and very little novel research. Even things like LLM speedups via new attention mechanisms are more engineering than they are research.

Above all else, what you need to build competitive models is a huge amount of money and access to compute. Money alone doesn't solve that either due to the global hardware shortage. You are competing for a globally limited pool of hardware with all other companies.

Re:Greg (2017) by p1esk on Sunday November 19, 2023 @02:46AM (#38330195)
Google and Meta have more money and more compute than OpenAI. They have more data, better infrastructure, and more AI researchers. And they have been trying hard to catch up - so far unsuccessfully.
Re:Greg (2017) by fidotron on Sunday November 19, 2023 @04:38AM (#38330918)
But they have the key problem of all established companies: they built the infrastructure for what is now the wrong thing.

This means to do the right thing at Google would require fighting battles on all sorts of fronts as established mini empires within attempt to latch on to this new perceived opportunity to expand.

Re:Greg (2017) by p1esk on Sunday November 19, 2023 @03:18PM (#38337372)
to do the right thing at Google would require fighting battles

Determining what the right thing is the hard part. Unfortunately that’s also the case right now at OpenAI.

Re:Greg (2017) by loveparade on Sunday November 19, 2023 @07:37AM (#38332108)
Google and Meta have less compute infrastructure purpose-built for training and testing LLMs than OpenAI does. It's something that OpenAI has been investing in for several years, while Google/Meta/etc have just started playing catch up very recently.
Re:Greg (2017) by varjag on Sunday November 19, 2023 @05:41AM (#38331341)
Researchers like Ilya were essential several years ago when OpenAI got started, because at that time nobody even knew what to build or how to go about it.

Nah Sam will just use GPT4 to code his vision. That's how it works now, we've been told.

Re:Greg (2017) by parentheses on Saturday November 18, 2023 @11:00PM (#38328742)
Nor do Andrej Karpathys (grow on trees).

OAI still have people who are true leaders in this deeply technical space - I don't see research wins going to a competitor unless they cultivate similar talent at the top.

It's also possible that Altman was privy to a breakthrough which makes it possible for him to execute from scratch on the AGI goal without a strong research org.

It's all speculation :)