News for hackers. Stuff that matters.
Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by upghost on Thursday November 30, 2023 @09:12AM (#38473733)
Technical foul. Doing science based on closed-source model. 10 yard penalty for humanity.

GPT-4 almost disappeared last week. Aside from the obvious AI bus factor and "continuity of science" concerns, I find it absurd that so many papers and open source ML/LLM frameworks and libraries exist that simply would not work at all if not for GPT-4. Have we simply given up?

I thought this was hacker news.

Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by Philpax on Thursday November 30, 2023 @09:39AM (#38474054)
I'm all in favour of doing reproducible work, especially against open-source ML (I'm the maintainer of a library for LLM inference), but what is the alternative here?

GPT-4 is still the flagship model "of humanity"; it is still the best model that is publicly available. The intent of the paper is to determine whether the best general model can compete with specialized models - how do you do that without using the best general model?

Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by tensor on Thursday November 30, 2023 @08:30PM (#38481815)
This is not answering that question. It is answering whether a specific large model with unknown training set can compete with specific small models on known smaller training sets.
Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by pixl97 on Thursday November 30, 2023 @11:37AM (#38475574)
Almost all modern science is gated by needing incredibly massive amounts of money.

'Closed source' science happens in every industry. While you may not think of what DOW is doing at science, it is and huge companies like that progress by their internal workings.

What you're stating is something else, that maybe we should fund sciences, such as AI more, but we really did that in the past and it had it's own fits and starts. Then transformers came out of Google and private industry has been leading the way in AI. If you want to keep up with cutting edge and the 10s of millions needed to train them, you'll need to use these private models.

Hackers hack stuff from private companies all the time. Hackers use proprietary software. There is no purity test we have to pass.

Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by rafram on Thursday November 30, 2023 @09:59AM (#38474293)
Half of science comes down to poking at black boxes to see how they react. The universe is a closed-source model!
Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by ZiiS on Thursday November 30, 2023 @10:16AM (#38474484)
The other half is publishing results to fix that.
Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by quickthrower2 on Thursday November 30, 2023 @06:51PM (#38480960)
The black hole here is political though! The algorithm in question is here on earth.
Re:Can Generalist Foundation Models Beat Special-Purpose Tun... by lambda_garden on Thursday November 30, 2023 @10:34AM (#38474746)
Llamafile is (rightfully) the top of HN right now.

I have high hopes that models will end up a bit like Linux servers, where everyone is building on open foundations.