News for hackers. Stuff that matters.
Re:Mistral Large 4 by baxtr on Tuesday October 06, 2026 @10:58AM (#49979495)
I think it might have accelerated things but on a much more basic level, there seems to be no real moat in synthesizing the world’s knowledge into LLMs.
Re:Mistral Large 4 by rpozarickij on Tuesday October 06, 2026 @12:10PM (#49980569)
There's no question that training leading LLMs requires some serious expertise and know-how, but surely already having advanced LLMs/agents must be helping tremendously not only for software engineers but also for those working on LLMs themselves.
Re:Mistral Large 4 by joe_the_user on Tuesday October 06, 2026 @01:03PM (#49981216)
I think one could describe LLM optimization as "hard but not a moat". Years ago, optimizing neural nets was described "graduate student descent" - it's tricky but throw enough conventionally smart people at it and it will happen. It's like tuning a hot rod and finding a reproducible bug in a large code base. It's hard and there are tricks but not absolute hurdles, no problems waiting for a conceptual breakthrough (and at today's scales, are there any problems waiting for an Einstein to solve? That's an open (AI) question).
Re:Mistral Large 4 by cyanydeez on Tuesday October 06, 2026 @11:34AM (#49980050)
I also think we're seeing the sigmoid approaching.
Re:Mistral Large 4 by spwa4 on Tuesday October 06, 2026 @11:48AM (#49980269)
What is really going on: all the AI labs are doing panicked model releases (and panicked training of new ones) because Qwen4 is rumored to come out end of October and is rumored to be very nice. Question is: is it another "Deepseek-moment" nice? Or just nice?

Btw: with Qwen4 I mean the next large Qwen model that is based on the Qwen4 architecture (Qwen 3.8 flash next was "almost" based on the new arch but obviously was a small model)

Re:Mistral Large 4 by ejeje12a on Tuesday October 06, 2026 @12:00PM (#49980447)
It doesn’t matter.

What matters more is if firm’s start using a bundle of American and Chinese models and when they find their feet - how large is the market for frontier?

Frontier has to displace labour one for one at some point or it’s over.

Re:Mistral Large 4 by christkv on Tuesday October 06, 2026 @12:06PM (#49980521)
Awesome 3.8 next runs great on my Framework Desktop so I'm loving more local models.
Re:Mistral Large 4 by hypercube33 on Tuesday October 06, 2026 @01:07PM (#49981265)
I have similar hardware - what specific version of the 3.8 Next model are you running and how many tokens/s are you seeing? 3.6 35B A3B gets about 68t/s for me so I've been sticking with that model for the mean time.
Re:Mistral Large 4 by throwa356262 on Tuesday October 06, 2026 @01:53PM (#49981829)
The thing with 3.8 next is that it uses a variant of ngrams. Part of the network is replaced by a lookup table you can store on a fast ssd.

In practice, you will be able to run models a bit bigger than 35B.

https://unsloth.ai/docs/models/qwen3.8-next

Re:Mistral Large 4 by cyanydeez on Tuesday October 06, 2026 @03:12PM (#49982709)
https://github.com/peonist-ai/halogen-server is beating the pants off of anything I've tried.

38GB of vram resident. more tk/s, more prefill.

Re:Mistral Large 4 by christkv on Thursday October 08, 2026 @02:54PM (#50010323)
I have been using Q4 and running it with https://github.com/gufo-org/gufo which seems to be working pretty well. But I have to get off bazzite as my distro. I love the fedora atomic in principle but it's a pain for normal installs.
Re:Mistral Large 4 by verdverm on Tuesday October 06, 2026 @12:29PM (#49980784)
I wondered if there would be a Qwen 4 or we would go straight to 5, re: tetraphobia, but perhaps it's more like an uno reverse card in this case

https://en.wikipedia.org/wiki/Tetraphobia

Re:Mistral Large 4 by satvikpendem on Tuesday October 06, 2026 @01:07PM (#49981280)
DeepSeek being Chinese also has 4 so I don't think it's a big deal for model makers.
Re:Mistral Large 4 by verdverm on Tuesday October 06, 2026 @01:32PM (#49981601)
GLM moved through their 4-series without consequence

I wonder if the next DS models will also graduate to 5.x, I think I saw they are training up a 10T model, and just raised $12B too

Re:Mistral Large 4 by disgruntledphd2 on Wednesday October 07, 2026 @04:53AM (#49990039)
> and just raised $12B too

It's important to realise that Deepseek are a very unconventional model lab, as they were originally funded by the founder's work in quantitative trading.

They mostly don't need that much money, the reason they took funding is to provide employees with equity, as lots of their good people were getting poached by other Chinese model startups.

(I have no inside info, just read the FT articles about them).

Re:Mistral Large 4 by cyanydeez on Tuesday October 06, 2026 @03:11PM (#49982695)
I'm pretty sure the point of Qwen3.8-Flash-Next was to get the open source engines to integrate the qwen4 architecture.

The fact that it basically broke open the local model supremacy was just a nice side effect.

I'm running: https://github.com/peonist-ai/halogen-server with a quant4, PLE offloaded, and it's resident VRAM is 36GB at 265k context.

Shave 10 more GB off and the TAM openai and anthropic are targeting is a lost cause. Local models are what 90% of people will need.

If the world governments can get a handle on the memory cartel, then there's no more moat for most normal humans.

Re:Mistral Large 4 by ben_w on Tuesday October 06, 2026 @03:55PM (#49983225)
I want that to be true (assuming you mean specifically the second half of the sigmoid) just to give me room to adapt to the changes we've already seen; but I've seen comments saying things are slowing down since around when GPT-4 came out.
Re:Mistral Large 4 by scotty79 on Tuesday October 06, 2026 @11:28AM (#49979976)
I think the moat is going to be compute. So far compute needed to push the frontier is still extremely cheap so the capital can afford to spread its bets. But when further improvement is going to cost in trillions, capital will have to pick a winner and bet only on him. It won't be a matter of finding the best bet, it will be a matter of survival.

This will cause the picked winner to get massively ahead with sheer compute alone used both for training and inference dedicated to recursive self improvement.

Re:Mistral Large 4 by pennomi on Tuesday October 06, 2026 @11:32AM (#49980034)
Surely there is a point where algorithmic improvements will be more cost effective than buying more hardware.
Re:Mistral Large 4 by scotty79 on Tuesday October 06, 2026 @11:47AM (#49980259)
I'm afraid it might be the other way around. RSI might pick all of the low hanging fruit soon. There must be a physical limit of how much intelligence you can squeeze out of some amount of parameters and compute.

There are going to still be worthwhile improvements but they are going to be more like not how to make transformers 10x cheaper but how to make next training run cost 9 trillions instead of 10 with a very particular optimization designed at the cost of hundreds of millions for this one specific run.

Re:Mistral Large 4 by LPisGood on Tuesday October 06, 2026 @12:01PM (#49980450)
I think we’re no where near a physical information theoretic limit.
Re:Mistral Large 4 by thfuran on Tuesday October 06, 2026 @12:52PM (#49981066)
The hardware also is, so there ought to be a whole lot more room for improvement.
Re:Mistral Large 4 by ejeje12a on Tuesday October 06, 2026 @12:02PM (#49980467)
“ RSI might pick all of the low hanging fruit soon”

Scam Altman is that you?

Re:Mistral Large 4 by bee_rider on Tuesday October 06, 2026 @01:41PM (#49981696)
It is just occurring to me that “RSI” expands to recursive self improvement. Thought people were talking about repetitive stress injuries; either in regards to programmers writing too much code/not having to write code anymore, or the frontier AI companies and their tendency to applaud themselves.
Re:Mistral Large 4 by Zigurd on Tuesday October 06, 2026 @12:45PM (#49980986)
If you're actually applying LLMs, all of the things around the LLM that adapt it to coding, for example, that enable it to use existing validation tools for code, and enable it to diagnose and fix tool chain issues that aren't directly coding problems, are what makes the difference between a model that that scores a little higher on a coding benchmark and a model that's useful in a particular code base on a particular platform.

Are there any use cases that have enabled one customer of a frontier LLM to outperform a competitor using a different frontier LLM? Or is this why we are seeing confected points of comparison like solving challenge problems in mathematics?

Re:Mistral Large 4 by bushbaba on Tuesday October 06, 2026 @11:35AM (#49980080)
Not just compute but energy. Most of Europe has no access to the cost effective power generation needed
Re:Mistral Large 4 by pyrale on Tuesday October 06, 2026 @12:23PM (#49980712)
Europe has lots of zero-cost windows for electricity, and areas with cheap prices. The real issue is access to oil and gas.
Re:Mistral Large 4 by randomNumber7 on Tuesday October 06, 2026 @12:52PM (#49981077)
Maybe for training big models one can wait for times when the wind is blowing.
Re:Mistral Large 4 by flir on Tuesday October 06, 2026 @01:44PM (#49981737)
There are worse ideas.

I could imagine a belt of data centres around the equator, that hand off their computational loads as the sun sets. Good scifi-esque premise.

Re:Mistral Large 4 by kergonath on Tuesday October 06, 2026 @02:19PM (#49982118)
That does not really sound practical to let datacenters costing billions idle half the time.
Re:Mistral Large 4 by Danox on Tuesday October 06, 2026 @12:17PM (#49980653)
Build Nuclear, Build Thorium the Chinese are building whatever they can. They’re not locked in by special interest. Is that because they have lots of engineers on the job in government?
Re:Mistral Large 4 by oakesm9 on Tuesday October 06, 2026 @12:03PM (#49980481)
France is actually pretty cheap in Europe. About 15% more than average USA electric prices (but I know that varies a lot across the states so still likely much more than the cheaper areas)
Re:Mistral Large 4 by jsw97 on Tuesday October 06, 2026 @11:54AM (#49980352)
Training location is flexible. Iceland?
Re:Mistral Large 4 by jayd16 on Tuesday October 06, 2026 @11:38AM (#49980119)
But the old models still exist at trivial marginal cost. The frontier models would need to dominate every price point to really take all and so far they haven't been.
Re:Mistral Large 4 by ambicapter on Tuesday October 06, 2026 @12:01PM (#49980455)
Don't worry, AI boosters will be in here soon denigrating anyone that uses anything but the latest and greatest models as irrelevant.
Re:Mistral Large 4 by flir on Tuesday October 06, 2026 @12:28PM (#49980763)
I guess all predictions age like milk, but here's one:

There's a law of diminishing returns at play here, and doubling the energy cost of training to wring 2% more performance out of the technology isn't going to be very useful, because most of the problems it is capable of solving will be solvable with the previous-gen 98%-as-good model.

("there's a law of diminishing returns at play here" is an article of faith. But then, so is the belief that these models will keep getting better).

Re:Mistral Large 4 by londons_explore on Tuesday October 06, 2026 @12:39PM (#49980905)
As soon as you can demonstrate decent financial returns (ie. the AI can run a company better than humans can), suddenly it makes sense to put a lot more $$$ in even if returns are diminishing - since whoever runs companies the best gets control of a big chunk of the world economy.
Re:Mistral Large 4 by ejeje12a on Tuesday October 06, 2026 @12:46PM (#49980999)
“ ie. the AI can run a company better than humans can”

lol You can always tell who has never ran a business before with comments like this

Re:Mistral Large 4 by satvikpendem on Tuesday October 06, 2026 @01:08PM (#49981290)
I understood their point, if we truly get AGI then no reason to think AI would be worse than a human.
Re:Mistral Large 4 by asplake on Tuesday October 06, 2026 @02:57PM (#49982543)
AGI with people skills?
Re:Mistral Large 4 by scotty79 on Tuesday October 06, 2026 @11:14PM (#49987657)
Do you doubt? Already plenty of people prefer people skills of AI chatbots to those of other people.
Re:Mistral Large 4 by satvikpendem on Wednesday October 07, 2026 @12:52AM (#49988357)
Chatbots are more considerate than many people I know. They at least explain things in a calm way.
[dead]
Re:Mistral Large 4 by cheevly on Tuesday October 06, 2026 @02:09PM (#49982000)
You are literally an AI spambot account, that's the hilarious irony here.
Re:Mistral Large 4 by satvikpendem on Tuesday October 06, 2026 @01:20PM (#49981452)
Well yes that's the worry isn't it? That people will soon all be unemployed? Just because it sounds farfetched doesn't mean you should stick your head in the sand, seeing the pace of development these days, that is "reasoning properly" and it just seems you are trying to block out what seems inconvenient to hear.
Re:Mistral Large 4 by flir on Tuesday October 06, 2026 @12:56PM (#49981132)
If that works... why not just jump straight to a planned economy run by LLM? Skip the whole messy "free market" thing altogether?

(I don't think it will work).

Re:Mistral Large 4 by allendoerfer on Tuesday October 06, 2026 @01:11PM (#49981319)
What do you have against multi-agent reinforcement learning systems and why do you think they are not AI?
Re:Mistral Large 4 by flir on Tuesday October 06, 2026 @01:39PM (#49981682)
londons_explore is arguing for a winner-takes-all scenario, with an early advantage locking everyone and everything else out.