Ok buddy

Is this a derived from flash next? That's the impression I get from reading over everything. Using Strata + Qwen 3.8 Flash Next now and I am beyond impressed. It's basically as capable as 5.6 Luna in my hands so far. If this is an upgrade I can't wait to try it out.

I assume so - Qwen/Qwen3.8-Flash-Next is marked as the "Base Model" on the huggingface page and states it may be subject to "Qwen's separate authorization or attribution requirements"

And I assume the metrics comparing qwen are using a quantized version - from the blog they compare memory use against qwen's "standard 4-bit deployment". Which might explain the improved performance over that base model they show - it might lose less from quantization.

https://huggingface.co/Edge0/Edge0-145B-NVFP4

It's interesting. It does seem to be based on QFN, but they've modified the architecture. 10B active versus QFN's 6B, plus also QFN is 180B parameters, not 145B. But it has the same 51B n-gram table.

If you look closely there is also a 25B, tho you question still stands.

Their GitHub also references that:

https://github.com/Edge0-AI/Edge0/blob/main/licenses/edge0-m...

3.8F/N is amazing. Truly useful, runnable local AI on machines that are not ridiculously expensive. (I’m running it on 2x 3090 and 128gb of ram, on the quantum roughly equivalent to 5-K-M. fast, but you can run it on much less

This looks like it would actually work on my 64mb M1 MBP. That would be fantastic.

We have different definitions of "ridiculously expensive." To me, more than $3k is ridiculously expensive. What you described sounds a lot more expensive than that.

When GPT4 ? 3.5? was all the rage, I started buying up AI gear because I saw memory starting to rise, so I bought 4x 3090s and 256gb of RAM, a couple mobos and some ryzen chips. I’m glad I did. It’s ridiculous that I could buy a decent used car for the price of this stuff now lol.

If you're only interested in coding tasks, this version of QFN will fit in 64GB RAM:

https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RC...

As a software professional, spending $8k on an RTX 6000 Blackwell was a reasonable investment in tools. I've learned so much, and now I have "free" Qwen3.8 Flash Next at NVFP4. I'm still using my M1 Macbook Max which was $5k. I don't think is unreasonable money for a professional. Would I pay the current price of $16k for the knowledge I've gained? I think I would. Luckily I got in early.

Hobbyist, sure, too expensive for most? OTOH as a kid, my parents sweated to accumulate $5k worth of computer (and 5.25" drives!) in today's dollars, so that investment paid off.

> Using Strata + Qwen 3.8 Flash Next now and I am beyond impressed.

The inference engine they use is called edge0-strata because it's a strata fork.

This project looks like someone fine-tuned Qwen3.8 Flash Next, made some tweaks to the model, forked strata to run it, and then put a lot of marketing and branding on top of it

This is not open source, this is open weights

It is not open weight either. The license is too restrictive to meet the standard definition of that team.

I see Apache 2.0 license, is that too restrictive?

Interesting, if 3rd party testing and benchmarking hold the published values this is very exciting. I am a bit skeptical given the model size, but I hope it turns out to be true!

I'm really impressed with the Blender examples. It's amazing how far "tool use" has come, when it now means using a tool as complicated as Blender to perform work that would take a human days to complete. I'm a little skeptical the actual models are workable, in the sense that a human could come in after and manually refine and apply better textures, etc. Historically, agent-made designs are messy as hell, and by the time you clean them up, you haven't saved any time. But, that's begun to get better; the best models are now very good at making clean shape-based SVGs, for example, that are no messier than what a human would make in Inkscape or Illustrator. But, I'll definitely give it a try.

Do they say how they interact with Blender? Launched within a python interpreter everything in Blender is accessible as a method call. I hope they didn’t do anything more complex than that.

My guess is MCP - there is an official blender MCP service

I just gave Fable 5.2 a shot at placing and routing a couple of circuit boards. I'm actually sure it would work fine for simpler/sparser designs but when the component density gets high enough, Fable spends too long struggling in the layer of the layout it's in rather than doing what a human would do, and intuiting that placement, or a component footprint, or different part choice altogether would enable the work to continue.

At first the work looked awesome, and once I started checking it out, I realized I would have to significantly alter the placements and manually draw most of the schematic to get it up to my standards (and hit the performance criteria of the board).

Yeah, I still expect to do a lot of manual work with this kind of thing. By "this kind of thing", I mean any kind of complex visual thing in SVG. I haven't tried any kind of 3D or circuit layout, yet, as it's just not something I do often, but if models could do it, I probably would do more of it...I have a 3D printer, but too slow at 3D modeling to ever want to do anything custom, and I do electronics projects occasionally, but have never had my own boards printed for the same reason. But, I would expect the capability of the models for 3D and circuits to trail SVG, simply because there's certainly less training data and 3D can be wildly more complicated than the average SVG.

It's surprisingly good but you have to have a bit of "theory of mind" to get what and why it can do graphical manipulations. Somehow Claude can draw 3D object wireframes but struggles to know where to draw the leader for a label in a diagram.

Also funny you should say you're too slow at 3D modeling but interested in using LLMs to do it. I'm on the other end. Not speed demon, but generally it's a lot faster to generate the geometry myself than to try to describe it. There's a lot of "how you got there" stuff that matters a ton for parametric modeling. If you want a CAD model that can easily be worked with you have got to be a bit smart about it, and I am not sure I see LLMs having that kind of spatial intelligence + deep physical world context soon. For one thing, it's a lot harder to do RL on.

My gut says that both (pcb and svg) would be much further along if current models could use tooling designed around how AI works

For basic 3D models (think project boxes, adapters of one shape to another etc.), give OpenSCAD a try. Add "BOSL2" library into it if you need threads or some extra niceties.

It's not a CAD in the traditional sense, but since it's basically a scripting language for 3D primitives, it works surprisingly well in LLMs. And since it's just text, there's not even any MCP or agents necessary (though it can save some time).

Basic test : open a chatbot of your choice (I've tried Gemini 3.8), ask it for some basic box for your model of Arduino, or give it some napkin drawing with dimensions and ask it to give you an OpenSCAD code for it. Then paste to OpenSCAD, render, take a screenshot and mark on it what you don't like and prompt again. Usually it's pretty usable within 2-3 such iterations.

Try the new Qwen 4 pro, it seems to beat fable 5.2 in many tasks.

Where is Qwen 4 Pro available?

Qwen app, qwen studio, qwencloud etc https://www.qwencloud.com/benefits/tokenplan

Yesterday I got an email from Alibaba about Qwen 4, but seemingly "Max Preview", not "Pro", wonder if they confused the two or there is something else? I'm not super familiar with the hosted Qwen models so maybe I'm the one who is confused here.

> Qwen4-Max Preview is now live on Alibaba Token Plan at a limited-time discounted price.

No I think I'm confused. qwen4-max-preview is the correct name.

TerminalBench scores often correspond to real world performance for coding agents that rely heavily on bash.

This score is roughly on par with 5.6 Terra max, which in my direct experience is perfectly capable of driving a cloud agent harness that can do pretty complex knowledge work.

Definitely worth an evaluation!

Are there any iOS apps that support the Edge0-35B model? I'm curious to try that out on-device.

Thy have shipped this as NVFP4 so its Nvidia blackwell only, presumably their inference engine doesn't support anything outside CUDA 13 specified either. Will be interesting to see if this gains some other quants and other inference engines pick it up and we get some standard GGUF files, its a bit of weird release at the moment rather than GGUF file downloads the variants link on HF is leading to a directory of stuff which contains the model spread across a variety of files.

[delayed]

And the app is "Windows only"... Such a strange release.

Why so little information about the company on the corporate website?

I'm really glad they provided us with the graph of "Model Parameters over time", fucking lmao

> Sharing matrices substantially reduces the weights that must be stored and moved while keeping the expert count unchanged. Each group of 64 experts shares two large matrices, while each expert retains a small set of parameters that captures its differences.

145B parameters with only peak 29GB memory usage - really impressive, truly something you can actually run on a decent spec MBP

The blog post says:

> For Edge0-145B, day-one support covers macOS, Windows, and Linux, including Apple M-series chips and NVIDIA platforms such as RTX Spark, RTX 4090, and RTX 5090. The release includes one-click deployment tools and hardware guides.

But... there's no guidance on how to run on a mac at all. Weird.

For some reason the engine that can run the 120B/145B models (edge0-strata) only seems to support CUDA for now:

https://github.com/Edge0-AI/Edge0/tree/main#platforms

I am not in the material, so perhaps someone else can comment on the quality of the outcome and technique? After skimming a page I am not really sure that the whole model fits inside 29 GB RAM. They seem to use some interesting techniques.

ModelTokenMemoryUsage [Bytes] = Parameters * (BitsPerParameter/8)

For a model with 145B parameters, this means:

    Precision       GB       GiB
    2-bit         36.25     33.76
    3-bit         54.38     50.64
    4-bit         72.50     67.52
    6-bit        108.75    101.28
    8-bit        145.00    135.04
    16-bit       290.00    270.08
    32-bit       580.00    540.17

You have roughly a factor 8 in memory usage from 32 bit down to 4 bit precision. That is quite some room already, but not new.

It seems that they radically reduce the model information required per token via the sparse expert approach - 145B->10B parameters - and then use some other approaches I do not understand at that point to make effective use of loaded information.

Did anyone try the model?

In the "Under the Hood of Edge0" section it mentions that it needs another 40 GB of system memory for experts plus 48 GB of SSD storage for vocabulary (engrams?). The 29 GB headline seems to just be a flat lie (/the LLM writing this article getting confused about what memory usage it's reading).

Oh that’s annoying, from the hype I was hoping my 48GB MBP would be able to run this happily, but if we’re talking about -70GB combined RAM footprint then that’s not going to cut it.

Still very interesting though.

The image under “Competitive capability with a smaller memory footprint” explicitly claims:

> Edge0-145B scores above comparable models with less than half the memory (29 GB vs ~70 GB) and runs on a 48GB laptop

So either they’re lying or confused or I’m confused.

Most likely a huge part of the magic is keeping the ngram table on disk, as only one row from it is read per token, so there's next to no performance hit.

Qwen 3.8 Flash Next utilizes this, with 40% of its parameters being offloaded to disk, making the effective memory requirements much lower.

My daily driver is this:

https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RC...

It takes things a step further by simply chopping off that half of experts which isn't (or rarely is) activated in coding tasks.

End result is a somewhat competent [0] coding model which runs on a 64GB machine.

[0] it still occasionally does weird stuff like replacing an entire file with just one line from its answer when you ask it a question if you don't explicitly tell it to not change anything or loop itself if the KV Cache is quantised at below Q8_0.

I tried it - it doesn't work with my configuration and setup on a DGX Spark.

I won't post my prompt here (I don't want it to get benchmaxxed), but I use a python programming question that used to be a common interview question 10-20 years ago to test models. It spent 8,000+ tokens at default thinking and never produced a coherent answer.

This is essentially a very clever concept that isn't really ready yet. Or perhaps the custom inference engine just doesn't play nice on my hardware yet. Either way, it's not something I'll be able to actually use in it's current state.

Their edge0-strata engine is just a fork of the strata project, which has been doing this for a long time: https://github.com/Niko1221/Strata

They're not hiding that fact once you get to the repo. I mean it's called edge0-strata and the README says it's a fork of strata.

It's weird to see this marketing web page imply that it was all new and novel work that they did, though.

And their model is just derived from Qwen3.8-next-flash which you can see when going to their HF page. Not much details to be found about the company apart from the founder who "was named in MIT Technology Review's Innovators Under 35".

Not much innovation here though it seems. More like a marketing stunt to collect VC.

Qwen3.8-next-flash is impressive though, especially with Strata or Basalt as inference engines. On a 5090 laptop with 64GB RAM one can run NVFP4 quants at 120-140t/s decode and 3000-4000t/s prefill.

On Mac it is much slower and the Kapitsa engine probably the best bet. Despite the claim of 29GB RAM for Edge0, in order to be fast you need to have at least a 128GB Mac.

How about Splash on Apple silicon? I've heard it's pretty fast.

https://github.com/incoai/splash

it is fast, 2x faster prefill and decode than similar quants than oMLX. At least in my limited, and unscientific, 36 hour+ tests

https://github.com/JamesFranc/refactored-parakeet/blob/main/...

Don't forget the memory requirements for context.

[delayed]

> Edge0-145B requires a compatible laptop with...

Damn, guess I can't run this on my desktop. /s

This smells off.

So did Strata Vs Llama.cpp, yet Strata pulls 2.5 to 3x the performance of the Llama.cpp codebase it was based on. If they've improved the architecture further by sharing experts, then this might be plausible.

You are not wrong, but it depends on the available hardware. I can run llama.cpp with Qwen 35B-A3B with usable speed (30ish t/s tg) on my laptop, but strata doesn't run at all, because my VRAM doesn't suffice.

Is the era of one seat per person AI coming?

Harness improvements like context pruning, malformed tool call healing also very helpful to boost smaller local models. Doing alot of work in this area lately. Qwen 27B in Terminal Bench 2.1 73->85 for instance.

Do tell more! I haven’t updated my harness in a while and ideas are welcome.

They may be referring to Swival https://swival.dev/pages/open-models.html

What's the point of the 120B model?

To fit in my 48GB MacBook.

The table in OP suggests the memory footprint is the same.

Model Mean across six benchmarks Memory footprint \(measured or estimated\) Edge0-145B 72.8 29GB Edge0-120B 68.9 29GB

For those of us with slow SSDs but a lot of RAM (relative to this model's file size) would this benefit from the extra memory, loading all in, or would I need to set up an old school RAM disk to prevent the swapping?

I'd guess the normal filesystem caching should automatically get you the benefits you describe.

Possibly not - the file shards may be too large to cache?

This just a quantized version of 3.8 next flash on strata that’s branded?

100% my exact impression of this project as well

And HIP support (amd) broken.

I'm not saying it's not, but if this is the case, how do we think it beats 3.8FN in benchmarks?

> I'm not saying it's not,

Their GitHub repo and HuggingFace pages directly state that it's based on Qwen3.8 Flash Next. Their inference engine is called edge0-strata because it's a strata fork.

They fine tuned it and tweaked the architecture.

The real question is if the model performs better on general purpose tasks, or if it was just fine tuned to increase the benchmark scores.

Fair enough. I was just responding to the "just a quantized version of 3.8 next flash" claim. Benchmaxxing via fine-tuning is a very plausible possibility.

Similar to a Linux distribution then?

I think the major difference is they have a set of profile data they use to manage the placement of expert weights.

"Hot" commonly used experts are kept on the GPU, and "cold" less used experts are bumped to system ram and calculated on the CPU. So it can run larger than vram sized models without completely crippling performance.

I'm not exactly sure what the changes they made to the architecture are for, perhaps they're aimed at extending the difference between the "hot" and "cold" expert usage?

I’m admittedly not an expert, I’ve not had the time or energy to try out many of these new American open weight models.

I’ll just say this, we obviously all know they (American open weight models) are playing catch-up to the competition. So if your reaction is “why would I use this over Qwen this, or Deepseek that” you’re really missing the mark.

Yeah, they are all going to be behind on benchmarks for now. Nobody realistically expects some American open weight model to come out of nowhere and beat Qwen/ Deepseek/ et cetera. But there has been a flurry of new releases from this segment, and it’s interesting to me to watch how it develops.

There an irony to your statement...since this model seems to be quantized version of Qwen Flash Next + some sort of post quantize training.

I am once again impressed by the ability of language models to generate huge amounts of relevant-looking text (and diagrams now!) while actually communicating as little as possible.

It's amazing the level of optimizations we are discovering. At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Over the past few weeks, I've been putting together a small, 6.5M param model. It is conversational, does tool calling, RAG, mathematical modeling/calculations, basic physical simulations, and multi-step chained reasoning. It's still very much _not_ a frontier model. But it already does more than I could get a qwen 2.5 0.5b param model to do with finetuning just 1 year ago

Most of the improvements come from being able to direct claude to apply different ideas from recent research that keep coming out. AI labs seem to be focused on 1B+ models, so not much comes out about small models, but many of the things being discovered in research with billion+ model, are adaptable to small ones as well

> At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Been reading this for years now. Seems both, bigger models optimized for scale AND smaller models optimized for single machines, keep on being released. I don't think it's changing, different markets.

I guess the question is less about what kind of model is viable and more about what kind of application fits any given model. I'm highly skeptical that my employer is willing to pay long-term costs of close to double what I make an hour for token usage, high precision or otherwise. I'm also highly skeptical of the return in some cases. I have to put my codebases in straitjackets tighter than ever to keep them on the rails, and this requires yet more tokens to do so. For premium work, okay, maybe? But it doesn't always excel at that work. For typical work, I don't see it being worth it for the big models.

The GitHub repo has more actual details: https://github.com/Edge0-AI/Edge0

Based on Qwen3.8-Flash-Next and Strata. They made some interesting tweaks and did some fine tuning on the model, but it's interesting how much marketing speak was layered on their website to hide this fact.

It's not at all unusual for people to fine tune models, tweak them, and fork inference engines. It's highly encouraged and appreciated in the community. What is unusual is seeing someone wrap this all up in extensive branding and publish it online as a company.

As with all fine-tunes, I'd be a little skeptical of the improved benchmark numbers until people actually use it more. It's really easy to take off the shelf models and fine tune them to get slightly higher benchmark scores without really improving them for normal use. HuggingFace is full of fine tunes that advertise amazing results on specific benchmarks but are worse than the base model when you use them for real work.

Has anyone figured out the macOS situation with this? The blog post mentions Mac repeatedly, saying things like "day-one support covers macOS". But the released stuff so far seems to be CUDA only. I see no info on when and if the actual Mac support is coming.

This weird approach is frustrating, because the actual promise sounds very intriguing, so I'd very much like to test this out.

https://chatpanel.net/blog/edge0-35b-macos-metal i have it working but not the Edge0-145B. just the 35B

> Runs on a compatible personal laptop, such as a MacBook Pro

I'm curious, do normal people (so not HN professional developers or entrepreneurs) have that, as actually personal device (not from their tech company)?

I imagine some might have but I feel, and that's me so I'm asking the broader community here, that the average PC user has at best a gaming rig, but that's already a minority, and a lot more often a low end laptop with 16GB of normal RAM, not unified or VRAM. Am I out of touch?

I allways use my own devices. Most universities allow to run/use your own devices. I run a m2 with 96gb.

no.

It was SO HARD to convince my sister to spend that 100€ more for 16 GB of ram.

My personal laptop is a fairly decent one, still with 16GB of soldered ram.

Just got approved a new laptop for work. 4k for 64GB of RAM and high end CPU. 1.5y ago that would have been 2.5k, only macs would have been that expensive

My dev machine is of the shelf Acer aspire laptop that I bought used, it comes with a 4GB RAM, I also added 4GB of swap memory. I use it to develop fullstack applications for servers, browser and android. My stack is Rust, Javascript, Dart/Flutter, Postgresql and running Debian 13.

It works great, I mainly use 2 applications , Firefox and urxvt, using emacs -nw to write code, no LSP or code completion because they use too much RAM. I also use i3wm which very lightweight.

How much did it cost you?

$47

I am not who you're replying to, but my daily driver is a Dell Precision 7550 with 32gb RAM that I bought refurbished two years ago for $400.

(Now I wish I'd bought two.)

> do normal people (so not HN professional developers or entrepreneurs) have that, as actually personal device (not from their tech company)?

I guess I'm far from the norm, as I have a Macbook Pro privately. I personally don't have 30+ GB of ram, though I'm sure the next one will have that.

However, I doubt that 'normal people' are even the target market for this. 'Normal people' don't care about their models running in the cloud. They want their voice assistant to be somewhat useful when there's no internet and we don't need 145B models for that.

So I think that I and people like me, often caller prosumers, are the target for these type of things. Some people buy €5k camera lenses for their hobby, others buy €5k laptops.

I think that only for showoff.

Those weighting value/price for the use case they are having the Pro is an overkill for most in the population. I have a good but still medium performance laptop personally as I use it for learning experimentation (and fun, which is not independent from the two former), but I did not push such for my wife. For her even the entry level Mac is an overkill. Be both recognise that. For those thinking like us the MacBook Neo was a good move from Apple.

Said that, this was before personal AI assistants. Now, for privacy reasons, I might push for a more capable laptop for her too (if I was sure I could set up a reliable config locally. I did not dig into that enough to make sure, but I'd do if it was an issue today).

I think the trajectory is more important than the snapshot in time. A year or two ago if you wanted to run things locally you were either running toy models, or somehow had a super-computing cluster. Now it's just high end 'normal' hardware. At current trends, we'll be running frontiers on a plain old computer within a few years.

All the snapshot does is set a pricepoint. Most 'normal' people I've talked to about LLMs have expressed some concerns about privacy/control from LLM cloud companies. So this is the cost it takes to opt out - actually substantially less since a 4090 is also enough to run Qwen 3.8. MSRP there is $1600. Retail price is a bit higher but trending down quickly. Once this bubble pops, this sort of hardware will probably be available on the cheap.

Anyhow, so yeah - a 'normal' person isn't going to have a $15k laptop or a $1600 video card (that retails upwards of $2k realistically) because they had no reason to. Now they do. Of course in a year or two, those prices will likely have plummeted precipitously, so perhaps they still have no reason unless they're just particularly impatient.

Frontier AI capabilities also include practical token rates. My current workload would require around 300 tokens/s continuously looking at the last 30 days of token spend. More realistic would be 1500-2000/s for human in the loop activities. I can't babysit the thing 24/7.

How far are we from this kind of throughput with local models?

Isn't this pretty much backward? If the thing spit out codes so fast, you'd have to babysit it. Slower throughput let you do something else while you wait.

Also throughput pretty much depends on compute capabilities and active weight. So if someone build a A0.1B then maybe you can look at 1500t/s of pure uselessness.

What it is spitting out is mostly not code anymore. I think much of the bubble narrative around HN is borne out of this perception that everything anyone would ever do with AI is related to code.

testing is believing! I'll see if I can report back this night.

5800X3D + 32 DDR4 GB RAM + AMD 7800 XT 16 GB.

Specially interested in the blender usage.

edit: ah, just read a bit more. seems it will be a waste of my time given the reqs?

The page is extremely verbose and its difficult for me to scan the page to find the kernels of specificity im looking for.

Slop article? Saying it works on mac but nothing on the repo to run it without CUDA.

Just ask Fable 5.2 for a version that works on Mac. Magically happens... There are just a few tweaks and voila! Getting about 50 tok/s on an m5 max at nvfp4.

This looks promising, but the comparison to Opus 4.6 on old benchmarks is outdated

> The constraint | What Edge0-145B makes possible

Whatever happened to "Problem | Solution"

Can anyone answer these two questions? I looked on the web page and the repo readmes, but couldn't find out:

- Is there a minimum VRAM requirement? I see the requirement for 29GB RAM (in total, as there are benchmark results with 16GB VRAM) and the requirement for CUDA13, but what is the VRAM requirement? Does it work with 4/6/8GB?

- Where is that company located? Didn't find it in the about us, careers and contact page.

Its not frontier even if it performs like a previous frontier model 12 month ago.

[delayed]

This is a PRC startup doing a remarket/optimization of Qwen, a Chinese model… May not matter to some people, but important to know.

> Frontier AI Capabilities on Your Own Computer

Hyperbole? No way that's true. Frontier will be generally more capable than local for the foreseeable future.

> Some Frontier AI Capabilities on Your Own Computer

FTFY