latentsea

> You can ask them only to consider the 5 facts in front of them, but they won't do that, and cannot do that.

Has training them to do it been tried and proven to fail?

Sorry, I need to ask Claude how I feel about this.

4 bit has always been accepted as the quality floor that retains most of the models capability. Going beyond that brings problems. It doesn't make models totally unusable, but my contention is the quant providers are intentionally not showing an accurate picture, and then everyone is glazing 3-bit quants. They are what they are, but it'd be nice if we had a more accurate shared picture of it rather than just hype and vaguery.

Tbh I think it's mostly a matter of time. I use SOTA at work, but only local at home, and the performance improvements out of local models in the last few months have been substantial, and more are on the way. To add to that, people are starting get better at harness engineering (which is itself improving with things like Harness2 etc) and also now we see people doing things like fine-tuning a Qwen 4B model to do 3D modelling with some success etc. The latent capability for local models to do what you're talking about is there, but the in-the-trenches reality is people are still working through unlocking the full potential.

edit: I should point out that it's also starting to accelerate now in that you can get your agents to help you with making them better at these tasks. So, it matters significantly less if you yourself don't have the skills to progress things, so long as you can figure out just enough to get the models to do it themselves. I suspect we'll see this dynamic ramp up quite a bit in 2027 with Qwen4.

you're probably better off trying to do this on Qwen3.8-Flash-Next on something like an R9700 or a pair of 5060 Tis, since you can run that at like 70 t/s and it's better. Then you just run multiple slots, assuming you can fit the KV-cache, which maybe you can't. You just need like 128GB system ram too.

I need to try this... it's on my list anyway.

Well if you can run Qwen3.8-Flash-Next at decent speeds single stream, and you can then run it slowly but on multiple streams, and if multiple streams can act as an agent swarm that can compete with current SOTA, then... yes, in theory it's possible today. I think R9700 + vLLM-radiance is the play?

Very much so. Probably a lot of compute is being wasted on using SOTA models for tasks that can be done on way, way weaker models.

Yeah, they're surprisingly usable eventually. What I found running the Qwen3.8-Flash-Next IQ3_XXS on CUDA kernel development was, sure it generated faster t/s than 27B dense, but it generated so many more bugs, and when the quantization is lower the reasoning traces get longer. It was more or less always able to figure out what bugs it had made and fix them, but it took quite a lot of time to do. I switched back to Q6 on the dense model and that just went away for the most part. I recently upgraded to 128GB system RAM so that I can up to Q5 on Flash Next. Really excited for Qwen4-Flash when it comes out. Gonna be a real beast.

They aren't "Just as accurate". What you see with these quants is the providers only ever show performance on already saturated benchmarks. You'll see labs put out 1-bit quants Deepseek v4.1 claiming only 1% degradation because they're only showing the benchmarks which tell that story. Then you look at the performance on those same benchmarks for significantly weaker models like Qwen3.8-27B even and it's virtually the same because that benchmark is saturated.

I like these quants as much as the next guy when they're all I can run but I have both R9700 with 32GB VRAM and a 5060 Ti with only 16GB VRAM and so I run both Q6 Unlsoth quants and ISTA IQ3_XXS quants and yeah there is an absolute world of difference in realworld performance.

IMHO I think it's pretty scummy behavior from the quant providers to frame these quants this way. Just be honest about where the degradation is really occurring on the non-saturated benchmarks.

No one is going to look to you in a world where anything you can do a machine can do better.

Does that mean the laptop was compromised or the DevOps engineer was?

Wake up comrade, new model just dropped!

Do androids do that?

A future in which nobody needs you.

The future is not set.

Is it like mathematician slang for 'yak shaving'?

>But maybe that’s still just making the proof easier for humans to understand? No! For one, this usually makes the proof shorter, and sometimes dramatically shorter.

Hmm, so is the way to train this to take all the long complicated proofs it generates in Lean and then use autoresearch to minimize the length of the proof?

My priors have been so thoroughly updated by all the recent progress in AI that it really seems like there isn't anything that fundamentally cannot be learnt. So, is this how we train this kind of taste?

I'm not a mathematician, nor an AI engineer/researcher. Just watching from the sidelines and spitballing ideas.

What a time to be alive?

Fucking yikes. As someone that relies on Proton, I'm devastated to hear this. They have just completely destroyed their brand.

Jev for fast actions + LLMs for more strategic planning?

Why 3 to 5 years??

I dunno. Aren't we somewhat similar? Why do we use software?

Twice the length from one end to the middle.

I think it doesn't matter what the truth is. There will always be people that want to believe their preferred narrative.

There is no rethinking the path we're on. It's unsteerable. I say so because I think you can't convince people to steer it away from a potentially bad outcome faster than that outcome is arriving.

Cuts in pay it is.

Well that's all there will be to do.

> I hope AI educates people and doesn't merely steer them down deeper into their echo-chambers.

Honestly, it feels like there is exceptionally little incentive to learn anything deeply anymore because the payoff might never be there.

I know, I know, do it because you love it or you find it interesting. Sure, I will to the degree that I love it or find it interesting. But I find even the incentives on that have changed weirdly enough.

> What will we produce in a post AI world?

Sports games. Plays. Concerts. Sex work.

Take your pick.

How are their API prices compared to others?

If I suspect stuff is LLM output I often respond in kind to give them a taste of their own medicine. I can imagine that's what this is.

When software engineers get put out of work, they'll move to other industries. They'll take their engineering mentality/skills and their affinity for Claude and they'll look at their menial, boring work they're handed and they'll say... I know... I can automate that!