News for hackers. Stuff that matters.
Re:The Collapse of GPT by lacker on Friday May 16, 2025 @08:44PM (#44011128)
Unfortunately, I don't really know if I can trust academics to analyze the development of large language models. No academic team has built an LLM. So... do people working at Stanford or Oxford really have good insight how LLMs are developed?

If people at OpenAI, Anthropic, or Google said this, that would be interesting. But I don't think it makes sense any more to treat academic computer scientists as relevant experts here.

Re:The Collapse of GPT by _heimdall on Friday May 16, 2025 @08:46PM (#44011137)
My understanding is that those building them don't really know how they work. Research into interoperability has fallen way behind as funding went towards features and scale.

Any understanding of how they work is largely theoretical, that seems like a reasonable place for academics to lean in and join the conversation.

Re:The Collapse of GPT by jsheard on Friday May 16, 2025 @08:47PM (#44011148)
It doesn't really make sense to trust what OpenAI and friends say about this either, when admitting to any kind of scaling limits would go against the narrative propping up their multi-hundred-billion-dollar valuations. I guess we're just flying blind for now.
Re:The Collapse of GPT by pphysch on Friday May 16, 2025 @08:46PM (#44011140)
Why would Big AI kill their golden goose like that?