News for hackers. Stuff that matters.
Re:Clef: Open-source decision models, and new RL fine-tuning... by seizethecheese on Thursday October 01, 2026 @01:35PM (#49924640)
Name a few of these low hanging fruit left around.
Re:Clef: Open-source decision models, and new RL fine-tuning... by TeMPOraL on Thursday October 01, 2026 @01:48PM (#49924806)
Jev is one.

Diffusion transformers are not "easy" but underfunded.

Random one in terms of applications: getting GPT-4-level[0] LLMs to operate at hundreds of tokens per second on edge hardware - opens up so many possibilities I'm probably unable to imagine half of them.

E.g. Imagine spellcheck/predictive text (or code autocomplete) where the model is able to process a whole paragraph + surrounding application/system context in between keystrokes. Or an OS being able to reliably guess what you're doing in real-time, in between your UI interactions, and offer actually helpful contextual reactions.

Or imagine finally funding some decent studies into exploring the models as computational artifacts - studying their latent spaces, how they form and how they model reality internally.

Or imagine automated sliding doors that don't suck.

--

[0] - Or anything substantially better than BERT-level models used in Jev or that demo from the company doing inference ASICs, that has a chatbot online that does 14 kilotokens per second.

Re:Clef: Open-source decision models, and new RL fine-tuning... by mdp2021 on Thursday October 01, 2026 @03:42PM (#49926190)
There are "low hanging fruits" - easier to achieve goals -, and there are super-fruits, milestone-fruits.

Among the most important ones:

-- the long-known Problem of Transparency, applied to the apparent emergent intelligence in NNs. Why does it happen - in detail?

-- then, a Theory of Apparent Intelligence through NNs. Transforming the results achieved into a Science. Which allows to do what we are doing - but in a lean and targeted way.

-- then, a General Theory of Intelligence, that includes the above to go beyond current architectures and get those features of Intelligence we expect and still not have.

The long-term direction we got into must lead to this.

(You note a ponderant detail of the above when you note the importance of explaining the emergence of a World Model from a Language Model.)

Re:Clef: Open-source decision models, and new RL fine-tuning... by TeMPOraL on Thursday October 01, 2026 @04:11PM (#49926507)
Those are the absolutely fascinating parts, and I sincerely hope AI won't get out of control before we're able to tackle some of these.
Re:Clef: Open-source decision models, and new RL fine-tuning... by patcon on Thursday October 01, 2026 @04:50PM (#49926938)
If anyone is interested, following Dr Michael Levin's Thoughtforms.life podcast is the cutting edge of where all this previously fuzzy stuff is becoming more concrete. So long as you can tolerate distinguished scientists flailing about as they discuss consciousness and life and developmental biology (and other less-obviously living things, like algorithms) as involving "free lunches" and "ingressing patterns from the platonic realm" :)
Re:Clef: Open-source decision models, and new RL fine-tuning... by blurbleblurble on Thursday October 01, 2026 @02:11PM (#49925080)
Diffusion models combined with these new looping techniques are gonna change the whole conversation about efficiency. Imagine control net but in one or more conceptual latent spaces.

But also harnesses and more generally new insights on "the control flow problem" could end up squeezing a ton of performance out of small models.

Re:Clef: Open-source decision models, and new RL fine-tuning... by seizethecheese on Thursday October 01, 2026 @03:32PM (#49926060)
I commend you for actually answering, independent of what I think of the answers.
Re:Clef: Open-source decision models, and new RL fine-tuning... by dominotw on Thursday October 01, 2026 @04:28PM (#49926685)
not a very good answer though
Re:Clef: Open-source decision models, and new RL fine-tuning... by tomrod on Friday October 02, 2026 @12:06AM (#49929731)
Yes yes, most comments and responses fail to contribute to conversation (though I disagree with your particular assessment); it thus remains important to continue to share our own thoughts as we battle through thesis and antithesis to arrive at synthesis.
Re:Clef: Open-source decision models, and new RL fine-tuning... by flipping_beacon on Thursday October 01, 2026 @02:06PM (#49925012)
Definitely agree with edge computation, although inference extensively researched and funded if SOTA LLMs hit a dead end tomorrow,there is still a lot to explore and research in inference and edge computation
Re:Clef: Open-source decision models, and new RL fine-tuning... by Amekedl on Thursday October 01, 2026 @02:44PM (#49925482)
yeah your reply, nobody can predict the future.

Enough stuff can happen, software use itself might change, and that could really cause anything. "What will we do with all the gpus" might become a question if for a magnitude of tech and reasons leaked-opus-9 runs on a macbook m6 or 7

Re:Clef: Open-source decision models, and new RL fine-tuning... by aeve890 on Thursday October 01, 2026 @02:04PM (#49924992)
>Random one in terms of applications: getting GPT-4-level[0] LLMs to operate at hundreds of tokens per second on edge hardware

That's low hanging for you?

Re:Clef: Open-source decision models, and new RL fine-tuning... by mdp2021 on Thursday October 01, 2026 @03:49PM (#49926246)
> That's low hanging for you

An important part of the industry is studying that: it is built-up effort. Sooner or later, the fruits will be harvested. The targeted preparation has been there for years now.

Re:Clef: Open-source decision models, and new RL fine-tuning... by guyomes on Thursday October 01, 2026 @02:36PM (#49925388)
If we throw in hardware dedicated to a specific LLM, it seems to be a rather low hanging fruit. Especially considering that this is already happening for vision models [1].

[1]: "FPGA-based CNN Acceleration using Pattern-Aware Pruning" https://inria.hal.science/hal-04689673/document

Re:Clef: Open-source decision models, and new RL fine-tuning... by mdp2021 on Thursday October 01, 2026 @04:33PM (#49926742)
> hardware dedicated to a specific LLM

That wording screams "Taalas". Which, importantly, is not the only player trying to abate the distance between data and arithmetics...

Re:Clef: Open-source decision models, and new RL fine-tuning... by msdz on Thursday October 01, 2026 @02:12PM (#49925097)
Maybe they meant in the sense of “untapped potential”, because so far a lot of the focus has been on increasing model capabilities, not necessarily performance/power budget.
Re:Clef: Open-source decision models, and new RL fine-tuning... by TeMPOraL on Thursday October 01, 2026 @02:54PM (#49925610)
Yes. Point is, it's untapped only because everyone is running in the race (even if out of curiosity), and there's just not enough people with means to tap into these side threads. For the past few years, there's been many interesting papers that circulated the industry, got recognized as worthwhile pursuits, and then dropped because running behind the Big Vendors had massively better ROI.
Re:Clef: Open-source decision models, and new RL fine-tuning... by blurbleblurble on Thursday October 01, 2026 @02:12PM (#49925095)
It's likely quite close. There are so many papers proving concepts that would bring this, they just haven't been combined in production.
Re:Clef: Open-source decision models, and new RL fine-tuning... by TeMPOraL on Thursday October 01, 2026 @02:51PM (#49925571)
Yes. It's well within realm of possibility, but so far wasn't pursued because the Big Vendors went all-in into capability growth (rightfully testing "the bitter lesson" to its limits) and got themselves stuck in an arms race, while everyone else is barely keeping up and/or starstruck with fascination, exploring what these models can do.

This got everyone racing forward and right now there is not enough human attention left in the world to productionize this, or any of the other "side threads". When the race slows down, people will catch up, branch out, and loop back.

Re:Clef: Open-source decision models, and new RL fine-tuning... by blurbleblurble on Thursday October 01, 2026 @02:56PM (#49925654)
Just like renewable energy and so many other things. Hyperconcentration of capital is really tragic. I hope things turn around.
Re:Clef: Open-source decision models, and new RL fine-tuning... by TeMPOraL on Thursday October 01, 2026 @03:06PM (#49925769)
They will. That's the fallacy of the "S-curve" everyone likes to commit these days actually giving a positive outlook.

Assuming it won't get to full RSI, the current approach will burn out - most likely economically. The race slows down, people branch out, look back, start picking up the "untapped potential"/low-hanging fruits, and you have new S-curves launching in place of the one that just tapered off (hence a fallacy - a stack of S-curves adds up to continuing exponential growth).

In other words: it comes and goes. Hyperconcentrated capital will eventually deconcentrate.

Re:Clef: Open-source decision models, and new RL fine-tuning... by ekabod on Thursday October 01, 2026 @02:17PM (#49925158)
That's a high hanging fruit, not low.
Re:Clef: Open-source decision models, and new RL fine-tuning... by Twirrim on Thursday October 01, 2026 @05:36PM (#49927347)
We already have examples of LLMs running 16k+ tokens a second using custom ASICs.

It's down at the moment (Not sure if it'll return?) but Chat Jimmy[0] produced by Taalas[1] was powered by an ASIC running Llama 3.1 8B, and hitting 17,000 tokens/sec. It was amazing to use, you'd no sooner have hit enter than you had a full response back. I actually found its speed to be a problem for interactive stuff, as every answer got several paragraphs I'd then wade through, vs a model populating text closer to my reading speed.

I appreciate, there are differences between an 8 billion parameter model and something GPT-4-ish, but we're currently in the middle of a race between a half dozen or so companies to produce the next best frontier model, which requires their infrastructure to be dynamic.

We really don't always need newer better faster stronger models, there's quite a lot of room for "good enough" where getting 17kt/s at significantly lower power would be amazing.

[0] https://chatjimmy.ai/ [1] https://taalas.com/

Re:Clef: Open-source decision models, and new RL fine-tuning... by AshamedBadger56 on Thursday October 01, 2026 @05:52PM (#49927500)
>I actually found its speed to be a problem for interactive stuff, as every answer got several paragraphs I'd then wade through, vs a model populating text closer to my reading speed.

It would be interesting to pair the super fast model with a normal speed model. Have the super fast one do all the background research, code writing, etc. The normal model would just relay the needed info to you at a more reasonable pace.

Re:Clef: Open-source decision models, and new RL fine-tuning... by drewstiff on Friday October 02, 2026 @06:38AM (#49931983)
Why do you even need a second model for that? There are many simpler ways to slow down the rate that the near-immediate response is pseudo-typed onto the screen.
Re:Clef: Open-source decision models, and new RL fine-tuning... by Twirrim on Tuesday October 06, 2026 @01:47AM (#49974680)
Agreed, could be something as dumb as a simple javascript snippet running in the browser. Would be nuts to use something like an LLM just for that.
Re:Clef: Open-source decision models, and new RL fine-tuning... by sarkarghya on Thursday October 01, 2026 @03:10PM (#49925814)
I can imagine advancements on making smaller models work together better instead of a generalized core. Imagine a community or city having a https://pirateface.co/ so that the shard of the model that you need can be streamed in with minimal latency with your box only holding the minimal version (say deepseek v4 flash as orchestrator) of the model that you use on day to day basis.

We have overcome split brain problems before so this wont be our first

Re:Clef: Open-source decision models, and new RL fine-tuning... by esseph on Thursday October 01, 2026 @03:27PM (#49926002)
You're putting a lot of trust in uncorrupted, untrusted, unknown models (potentially).
Re:Clef: Open-source decision models, and new RL fine-tuning... by hobofan on Thursday October 01, 2026 @01:54PM (#49924893)
Closely connected to decision models: A good library to do ranking based on pairwise ranking on multiple attributes. By using a decision model (especially one that can make decisions on multiple fields at the same time) this becomes a lot faster and more powerful. Could make for a pretty nice search reranker as well as prioritizer for many problems.

Of course you can also do ranking one-off with a decision model, but this likely less stable, and by doing pairwise ranking you can also relatively quickly do incremental inserts to the list.

Re:Clef: Open-source decision models, and new RL fine-tuning... by murkt on Thursday October 01, 2026 @01:37PM (#49924675)
Easy to reach doesn’t automatically mean “easy to see”.
Re:Clef: Open-source decision models, and new RL fine-tuning... by CamperBob2 on Thursday October 01, 2026 @03:43PM (#49926199)
An example I like to use is: compare the quality and scope of games released with a brand-new console to the ones released for that console towards the end of its life, when everyone has learned how to take advantage of whatever weird, wacky hardware Sony invented for that console generation.

There is still a lot we don't know about how to get the most out of existing LLM components from a speed or cognitive-performance perspective. People could easily spend the next decade studying and refining what's been built so far, even if no new, original approaches ever arrive.

Re:Clef: Open-source decision models, and new RL fine-tuning... by randomNumber7 on Friday October 02, 2026 @08:22AM (#49932687)
I never enjoyed playing ps4 when it is constantly louder than my vacuum cleaner.
Re:Clef: Open-source decision models, and new RL fine-tuning... by sroussey on Thursday October 01, 2026 @05:53PM (#49927503)
Just look at all the model type on hugging face. LLMs are a small percentage.