a11r

A negative datapoint - We have been evaluated prior Deepseek Flash models (both the original 4.1 flash and the smaller 4.0 flash) and they have been overthinkers. Our workload is well defined but moderately difficult coding tasks and among open weight models GLM5.3 flash performs best with Qwen3.8 flash next a close second. DeepSeek 4.1 flash fails at lower thinking levels and spends a huge amount of tokens (an order of magnitude more) at the highest thinking levels where it is able to successfully complete the tasks. I will evaluate this newest release since they specifically claim "the model reduces task execution time and output length", but keeping my hopes low.

Artificial analysis shows this: underperforming Qwen3.8 Flash Next and GLM 5.3 Flash, both of which you can run on a single B300 . I think Meta has a bit of catching up to do in the open-weight space.

As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases. I would really like them to start providing comparisons at low and medium reasoning levels across these models because those tend to be the ones that actually get used in production.

I seriously doubt their ability to double API prices on Dec 1. Token prices are in a clear downward trend and Google should maximize revenue by competing for volume. Currently OpenAI is the only US frontier lab trying to seriously compete with open-weight models that you can run yourself for an order of magnitude lower than what frontier labs want to charge

I am renting spot VMs from Nebius. I've tried a variety of other providers like Vast and Spheron. Vast worked well for renting 5090s but I like the large memory and pricing I'm getting at Nebius for RTX Pro 6000. The extra RAM really matters because I need PLE to offload the ngram to RAM.

Yes, you can probably get the whole thing up and running in about an hour the first time. If you pause and restart, it takes about 15 minutes to load the models from disk into GPU memory, so budget for cold startup time.

I'm a little skeptical of going below 4-bit quants due to the potential for significant degradation in quality. I'm running 4-bit quants on an RTX Pro 6000 rented for approximately $1/hour and getting about 1.2 million tokens out and 40 million tokens in per hour with caching. The quality of 4-bit quant is good enough for difficult but well-scoped coding tasks. Here is the inference stack I am using: https://www.reddit.com/r/BlackwellPerformance/s/FrKwk3GoDK

We recently moved from 27B to Flash Next. The quality is superior for coding. Our workload is primarily well-defined coding tasks that need to be attempted a few times before the model gets it just right. FlashNext is also better at finding issues in generated code than Gemini 3.8 Flash.

First they ignore you. Then they ridicule you. Then they attack you and want to burn you. And then they build monuments to you. (Nicholas Klein)

Looks like Anthropic is moving on from ridiculing Open Weight to wanting to Burn them.

GLM-5.3 flash has been great for us and I am going to now invest serious effort in evaluating the full fat GLM-5.3 given this ringing endorsement.

Interestingly Z.ai does not train on user prompts, unlike Anthropic. (source: https://openrouter.ai/z-ai/glm-5.3#providers )

One key factor not directly mentioned in the article is the simple alignment of financial incentives when electricity distribution was privatized. Before privatization, the agencies responsible for collecting electricity bills had little incentive to actually perform the collection. When coupled with rampant corruption, you get a situation where a lot of bills are either getting unpaid or not even being generated and just being written off as transmission and distribution losses. Switching to private distribution immediately aligns the incentives of the distributor to actually bill their customers for the electricity they are using. None of this is meant to take away from the value of the technical and policy work that led to improvements, and I'm sure all of the factors called out in the article also contributed to this change.

That whitepaper puts the cost of a 40 MW cluster at $167m on land and $8.2m in space, inclusive of launch costs.

Pretty impressive to see the amount of performance they can squeeze out of the same hardware. I suspect the same process will play out for all combinations of LLMs, inference providers and hardware stacks. This should bring down the cost of inference for the providers by an order of magnitude in the next year and lead to fantastic margins for inference providers.

Back in 2008 Fujitsu has one of the best performing 10Gbps Switches. We were building 40 Gbps packet sniffers at Google (4x 10 Gbps NICs) and needed switches that could do things like mirror traffic across ports at line rate. Fujitsu was way ahead of the pack. I always wondered what held them back from building a meaningful networking business in the US.

It is great to see another player introduce a fully open stack. Nvidia's Nemotron is the only other prominent one I know of.

All that said, the headline claims do not match the self-reported performance. For example, the dense 32B model is significantly behind Qwen3.8 27B (chart towards the bottom of https://ifm.ai/blog/k2). Gemma4 31B is not in the comparison set. This is the most important sweet spot for self hosted open-weight models today and real competition here will be very welcome.

Looks like the strategy of regular updates with incremental improvements is working out well. Interestingly, the biggest jump in Artificial Analysis Intelligence Index score is for reasoning level Medium ( 3.7 was 51, 53, 57 for Low, Medium and High, 3.8 is 52,57, 59 respectively). I think scores at lower reasoning levels are more indicative of model capability since higher reasoning levels are focussed on benchmaxxing. We use the lowest reasoning level in production with good results.

Quoting RGFusion from Reddit: LLMs run into an issue where the further you train a model, the more it overwrites facts with generalized concepts. You need the model to be able to do both. Intelligence arises from generalization, but without accurate information the model will hallucinate.

The engram table allows for a low-computational method of fact-recall. You can think of it like a better form of RAG, where the data doesn't take up any of your context window and it's injected deeper into the model's layers, freeing the lower layers to carry out abstraction. This results in better "focus" for the model, both in regards to its intelligence and context recall.

Basically, they've separated the specificity-critical portions of the models memory into a parameter space that doesn't need fast compute (you can run it on system RAM) and allows the model to be trained on higher volumes of data without ruining its knowledge-base.

https://www.reddit.com/r/LocalLLaMA/comments/1vy6smx/comment...

There is a difference between needing frontier capability because one is solving a truly open ended problem, and needing a reliable workhorse model to do something well understood. Local models (like Qwen 3.8 27B) have gotten so good that they can do all routine tasks at a fraction of the cost of frontier models.

Even a 4-bit quant of Qwen3.8 27b is indistinguishable from Gemini 3.7 flash in our internal tests. With an RTX5090 card and ninfer, you can get ~800 TPS token generation (c=8) and ~140 Tokens per second single stream.

M3 Pro 36GB. I am getting 17 tps with MTPLX.

Amit Patel was employee #7 at Google and built many things including the little geo display in the lobby that would show live search queries as dots color coded by language.

The pricing is 40% of Kimi K3 at the same Model size. It will be interesting to see benchmarks tomorrow and to see how the pricing evolves across this size class.

Seminanlysis is estimating sub $1 cost per MT for ~2Trillion models. The numbers change based on throughput and quant, but it is conceivable that provider costs at scale are low enough that even $2.42 per MT on GLM 5.2 (current best price) is margin positive by a wide margin.

This resonates with my experience. At Morph we use gemini for well specified point coding tasks, and it does very well across millions of lines of code every day. We also use claude code as an engineering tool for our own codebase and it does better at being adaptive and for working on open ended issues.

Understood. I am looking for a side-by-side comparison focused on your feature set, not Github's. You answered it partially by calling out your focus areas. Github API reliability has been iffy for us, so it would be good to quantify the difference we can expect to get with you.

It would be nice to see a side-by-side comparison with Github on pricing and features. We are using github and creating hundreds of repos everyday without any issues (except for the occassional API outages that Github has). Curious to see your take on where Pierre is better.

This is very nicely done. We have seen the same issue at a higher level of getting separators right when generating multiple files in a single inference call.

Nice ! Curious what your prompt was.

These are large coal seams with significant exposure to the atmosphere. See https://en.wikipedia.org/wiki/Jharia_coalfield for an example. That excavator in the picture is not trying to put out the fire, it is just mining coal that happens to be burning. Spray some water, put out the fire and ship it off to customers.

The system prompt in this experiment limits the solution to always spell out the concrete moves verbally. A human solving the Tower of Hanoi gives up around N=4 and goes off to invent a recursive solution instead. Prompted differently, the LLM would solve these puzzles just fine.

Here is my complete review/analysis of the paper: https://www.linkedin.com/pulse/art-abstraction-human-advanta...

edit: fixed typo

Gary Taubes has written multiple scholarly books on the subject. He has won the Science in Society Journalism Award of the National Association of Science Writers three times. I think a training in one scientific field does qualify a person to go spend their time digging into other fields. How well they do is a matter of personal ability and effort and one would have to read their output and judge for oneself. I have read many of Taubes's book and many of the papers cited in his books and have concluded that he is a well qualified expert in the field.