News for hackers. Stuff that matters.
Re:OpenAI's Stargate project struggling to get off the groun... by saubeidl on Tuesday May 13, 2025 @12:37PM (#43974795)
I can't help but think this fixation on ever-bigger data centers with ever-more compute is a very American approach.

It's the same line of thought that gave us jumbo refrigerators, muscle cars and supersize meals.

In the American mind, bigger = better.

That that's not necessarily the best way to better results has been demonstrated by the Deepseek shock.

I do wonder if more foundational research would be a better investment.

Re:OpenAI's Stargate project struggling to get off the groun... by Closi on Tuesday May 13, 2025 @12:45PM (#43974903)
It’s not an either/or - it can be both!

If you have the best foundational research, but then find yourself blocked by compute, that’s a bad place to be in.

At the moment compute is a constraint too (at least for the smaller shops, maybe not OpenAI!)

We don’t necessarily know that the next AI breakthrough will need less compute either! A breakthrough that gives 5x performance for 20x training compute and 2x inference cost would still be a giant breakthrough and need a lot of hardware! Although this sort of breakthrough is unlikely, and it’s likely to be smaller steps forward - but I think it might be the case that models continue to get more computationally expensive even if there are breakthroughs that make models more efficient because we will want higher capability.

Re:OpenAI's Stargate project struggling to get off the groun... by saubeidl on Tuesday May 13, 2025 @12:55PM (#43975039)
I think it's often been shown that constraints are what brings out true breakthroughs, so I'm skeptical of scaling up throw brute force - to me it seems like a lazy way out that will prevent you from challenging your base assumptions and figuring out better ways of doing things.
Re:OpenAI's Stargate project struggling to get off the groun... by Closi on Wednesday May 14, 2025 @07:56AM (#43983426)
I think there is a world where you have both - a portion of research sitting under constraints, and also massive computing capability to help build the next big model.

Put another way - I doubt OpenAI would have built GPT-3 faster if they had less compute.

Re:OpenAI's Stargate project struggling to get off the groun... by klipt on Tuesday May 13, 2025 @12:48PM (#43974940)
> more foundational research

So you're saying a bigger amount of research is better?

Re:OpenAI's Stargate project struggling to get off the groun... by saubeidl on Tuesday May 13, 2025 @12:52PM (#43974995)
I do. I'm saying more thinking, less throwing raw material at a problem.
Re:OpenAI's Stargate project struggling to get off the groun... by courseofaction on Tuesday May 13, 2025 @12:42PM (#43974862)
But those damnable universities are just left wing brainwashing facilities, very little room for profit there. Cut their funding, educated people don't vote right /s