News for hackers. Stuff that matters.
Re:Muse Spark Open by a11r on Thursday October 15, 2026 @02:42PM (#50108293)
Artificial analysis shows this: underperforming Qwen3.8 Flash Next and GLM 5.3 Flash, both of which you can run on a single B300 . I think Meta has a bit of catching up to do in the open-weight space.

As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases. I would really like them to start providing comparisons at low and medium reasoning levels across these models because those tend to be the ones that actually get used in production.

Re:Muse Spark Open by alexpotato on Thursday October 15, 2026 @04:08PM (#50109534)
Been using GLM 5.3 Flash via Ollama Cloud basic subscription:

if this is the only model I had to use for the rest of my life, I would be cool with that. It's good at a wide variety of tasks including reverse engineering, design, working with ESP 32 etc.

I think Muse Spark 1.3 is better but GLM 5.3 Flash is an excellent daily driver.

Re:Muse Spark Open by neya on Thursday October 15, 2026 @04:12PM (#50109588)
If you liked GLM 5.3, you would appreciate Deepseek V4 Flash a lot. In my personal opinion and use-cases, it outperforms GLM 5.3.
Re:Muse Spark Open by 0xbadcafebee on Thursday October 15, 2026 @04:23PM (#50109712)
V4 isn't the newest DeepSeek, and GLM 5.3 non-Flash isn't the same ballpark

- DeepSeek V4.1 Flash has marginally better coding, and smaller experts (8B-In/16B-Out) with more compressed attention and KV cache, so it's faster but will have less expertise per task. It's a 552B model, so while it theoretically may contain more trained data it also requires more RAM (the experts are smaller to compensate for speed loss) making it harder to serve without the distributed KV cache DeepSeek hosts.

- GLM 5.3 Flash has marginally better agentic work, and has video input (DS doesn't), and its experts are 18B so there's more intelligence per task. It's a 320B model so it requires less RAM which will make it cheaper to serve. Considering it's a much smaller model, the higher intelligence indicates their architecture/training is better.

- GLM 5.3 (non-Flash) is far more intelligent than both; it's a 744B model with 40B experts, but it's text-only.

Re:Muse Spark Open by Shebanator on Thursday October 15, 2026 @06:24PM (#50111254)
Model sizes aren't directly comparable, and they aren't even that relevant. I care about result quality, cost, and speed, not size. I switched to deepseek because it was giving me better coding results than GLM, and for far less cost. GLM is still better at planning.
Re:Muse Spark Open by neya on Thursday October 15, 2026 @09:56PM (#50112893)
In OpenCode, it shows up as "Deepseek V4 Flash Latest", so I assume it is the v4.1, so that's where the confusion comes from.

However, I disagree on the technical specifications. For most projects, a single prompt will get me what I want with good quality with Deepseek vs GLM 5.3. This is from real world experience with JS and Elixir. Perhaps for other tasks GLM 5.3 might be good. But, my biggest annoyance with most non-flash model providers - including GLM is they will dumb down the models based on load (like switching a non-flash model to flash model). That sucks because you expect a certain level of intelligence from them. Not a coin toss on "here's me hoping it works" and you expect a certain model only when you explicitly specify it by name.

Re:Muse Spark Open by Implicated on Saturday October 17, 2026 @01:18PM (#50133000)
> But, my biggest annoyance with most non-flash model providers - including GLM is they will dumb down the models based on load (like switching a non-flash model to flash model).

Where are you seeing this? This would be sufficient for me to stop using that provider entirely, immediately.

Re:Muse Spark Open by samtp on Thursday October 15, 2026 @04:10PM (#50109561)
I've been using GLM 5.3 Flash as the planner and DS 4.1 as the task worker over the last month and the results have been very good, predictable, ridiculously cheap.
Re:Muse Spark Open by DaSHacka on Thursday October 15, 2026 @06:11PM (#50111117)
What reasoning level do you use for each?
Re:Muse Spark Open by edg5000 on Friday October 16, 2026 @12:48AM (#50114202)
Why this model split? And are your work volumes so high that you prefer these cheaper models over a subscription or are you simply trying to avoid the 200/m? (which will only go up).
Re:Muse Spark Open by dgellow on Thursday October 15, 2026 @04:43PM (#50109932)
Have you compared to deepseek v4.1 flash? It’s so good, at a ridiculous price
Re:Muse Spark Open by thekevan on Thursday October 15, 2026 @04:52PM (#50110035)
Genuinely curious, why not use GPT Sol 6.1 high which has a better cost per task and intelligence ($0.32, 50) or Mimo 2.6 pro? ($0.13, 46) over GLM Max's $2.01, 45?
Re:Muse Spark Open by mapkkk on Thursday October 15, 2026 @05:07PM (#50110252)
Not OP, but I've been trying out Sol 6.1 medium with hermes agent, and I have not been able to match usage costs to what I've been getting with glm 5.3 flash high.

Really don't know why, since I've offloaded thinking to the LLMs in this area of my life. Though even at medium, 6.1-Sol is noticeably better at most tasks I throw at it.

Re:Muse Spark Open by alexpotato on Thursday October 15, 2026 @05:11PM (#50110300)
I'm OP and I have the Ollama basic $20/mo plan.

Been very happy with it although I will say Muse Spark 1.3 being free on OpenCode is my main model these days. If I run out of free credits on OC (which almost never happens) then I drop into GLM Flash 5.3.

Re:Muse Spark Open by qingcharles on Thursday October 15, 2026 @09:15PM (#50112508)
Sol won't do some things. Like, GPT freaked out yesterday because I needed to vector trace a logo and it said it would be a rights violation.

Then I have to work down the list: Grok, then Muse, then if those fail I make the Chinese one do it.

https://imgur.com/a/8LVmpLV

Re:Muse Spark Open by windexh8er on Friday October 16, 2026 @12:06AM (#50113956)
Not a fan of Elon but latest Grok gets shit done and isn't a narc model from Anthropic/OAI.
Re:Muse Spark Open by kennywinker on Friday October 16, 2026 @12:30AM (#50114092)
Feelings about elon aside, the fact that they’re using gas generators to run these is a dealbreaker for me.

Source, to preempt the inevitable musk stans: https://www.scientificamerican.com/article/spacexais-colossu...

Re:Muse Spark Open by quaffcode on Friday October 16, 2026 @01:05AM (#50114300)
They're not using gas generators to run the other models?

Gas generators are common in new data center buildouts. I'm not a fan, because they are insanely loud, but they do prevent the data centers from weighing down the grid and raising rates for all the neighbors.

Re:Muse Spark Open by kennywinker on Friday October 16, 2026 @01:10AM (#50114337)
My local llm running on my eight year old gpu and ten year old pc is powered by hydroelectric dams. I can’t vouch for the other data centers tho.
Re:Muse Spark Open by logicallee on Friday October 16, 2026 @02:14AM (#50114745)
>My local llm running on my eight year old gpu and ten year old pc is powered by hydroelectric dams.

how do you still have a job?

Re:Muse Spark Open by u8080 on Friday October 16, 2026 @03:19AM (#50115197)
Hydroelectric dams disrupt biosphere by flooding areas previosly inhabited by animals - and kill the whole populations of wildlife and fish.

How do you feel destroying our planet and murdering innocent creatures for your entertainment clanker?

Re:Muse Spark Open by b112 on Friday October 16, 2026 @03:30AM (#50115292)
This is absurd nonsense, and needs to be quelled wherever espoused.

Dams aren't as a nuclear explosion. They don't burst into appearance, in a way where animals cannot walk away from the rising water. Birds lose nests all the time (wind knocks down trees, for example) and they merely build another. And they only need nests to rear young, and those fall out of trees so they... build another.

What happens in reality, and not in fantastical, hyperbolic renditions of the truth, is that one ecosystem replaces another. Where before was river, is now lake. And this is an entirely natural process. There have been cases of earthquakes causing river blockage, and of course the reverse!

And of course our friendly beaver blocks rivers all the time, sometimes to the extent that it can be seen from space. Rivers being blocked is a natural, normal, common practice.

But I suppose you hate all the beavers too? Do you raise fist to air, shake it in emotive angst, and rail against the horrors of natural geological processes too? Do you hunt the noble beaver, too?

How about instead of dismissing, deriding, and debasing one of the most environmentally friendly methods of power generation, you get behind it? How about instead of suppressing a hydroelectric dam, which means people will build Ng generators, or use coal, you support them.

Or would you prefer nuclear, for that stable, steady, backing, of which windmill and our friendly sol does not provide? Or would you prefer massive acres of batteries, instead?

I beg you sir, please cease this mindless chatter!

(no I don't normally talk like this, but what the hell, a but of levity may bring less conflict here)

Re:Muse Spark Open by u8080 on Friday October 16, 2026 @07:29AM (#50117098)
This was intended to be absurd nonsense (or as I call it, irony).

I mean, if you think about this, burning stuff is also a part of nature, wildfires happens regularly, volcanos active right now, etc. Burning fossils like gas was(from caveman burning wood) and is still most popular way to generate energy among humans. I don't get what's so special about datacenters people find compared to any industrial facilities to fuel hateboner on most absurd nitpicks.

Re:Muse Spark Open by kennywinker on Friday October 16, 2026 @10:05AM (#50118553)
Arsenic is a part of nature, but it’ll still kill you. Burning the massive amount of co2 stored underground during the carboniferous period is not the same as random forest fires that burn co2 fixed a few decades ago.
Re:Muse Spark Open by kennywinker on Friday October 16, 2026 @10:27AM (#50118768)
It’s funny because there are very real criticisms of hydroelectric dams, from a climate collapse perspective, but these are not them.
Re:Muse Spark Open by skinfaxi on Friday October 16, 2026 @10:57AM (#50119166)
What models do you run?
Re:Muse Spark Open by reverius42 on Friday October 16, 2026 @05:50AM (#50116324)
As far as I know at least Anthropic also rents space in Musk's gas generator powered data centers.
Re:Muse Spark Open by windexh8er on Friday October 16, 2026 @09:23AM (#50118100)
Pretty sure you're not able to use many models, besides local, if you need to guarantee where the energy produced powers them. I don't align with much of what's going on in frontier AI today. And I don't actually use it as much as I had been as I've invested in local hardware. But I'm still curious and continue to test frontier models on different tasks. My limited experience with Grok is that it's boundaries are much less rigid than Anthropic and OAI, IME.

But if the planet is the only consideration here you can't use any in of these models when pointing a finger at one. They're putting them wherever they can. And I'll be one of the first one to cheer some of these systems being dismantled after the fallout / when the US has some political leaders that have a spine again.

Re:Muse Spark Open by kennywinker on Friday October 16, 2026 @10:12AM (#50118625)
> But if the planet is the only consideration here you can't use any in of these models

This is simply a purity test. Just because they’re all bad, doesn’t mean there isn’t better and worse options.

Re:Muse Spark Open by windexh8er on Friday October 16, 2026 @11:37AM (#50119733)
I don't buy that - you can't distinguish. This sounds like a convenient way for people to say "my choice is better for the planet" with basically no basis.

Feel free to elaborate.

Re:Muse Spark Open by kennywinker on Saturday October 17, 2026 @09:18AM (#50130920)
https://sustainability.atmeta.com/blog/2024/10/14/our-approa...

Anthropic: no sustainability goals or claims

OpenAI: no sustainability goals or claims

SpaceX AI: no sustainability goals or claims

Some other non-lab inference providers:

https://regolo.ai/ Eu based, 100% renewables powered

https://greenpt.com/us/sustainability Eu based, allows you to track carbon cost of inference.

Acting like there is no difference between bad things is how you get worse things.

Re:Muse Spark Open by windexh8er on Saturday October 17, 2026 @02:51PM (#50133894)
The fact that you believe Meta, of all companies who's gone to great lengths to lie about their core business model, speaks volumes.

This also highlights my point: the norm is that the bulk of the inference and training providers have no goals.

Re:Muse Spark Open by kennywinker on Sunday October 18, 2026 @12:36AM (#50137958)
You've moved the goalpost and declared victory my guy. I literally gave you an option, that isn't meta, and is run on renewable energy. The fact that it's hard to get clear information doesn't negate the fact that there are differences, and we should seek to find the best option rather than giving up and accepting whatever bad option is nearest or shiniest.
Re:Muse Spark Open by windexh8er on Monday October 19, 2026 @07:42AM (#50151169)
Let me clarify my perspective here. You put the frontier providers at the top. That's probably representative of >80% of the total, global, inference. The other ones you mention? They're likely showcasing "green AI" as a differentiator, but comparatively, they likely represent a percent of global capacity.

And you don't get to put a feel good landing page from Meta and "declare victory" (I didn't) just the same. It means nothing because there isn't any accountability. Nice try.

Re:Muse Spark Open by timmmmmmay on Friday October 16, 2026 @10:31AM (#50118829)
43% of the US electrical grid runs on natural gas, it's the largest single source by a considerable margin.

https://www.eia.gov/tools/faqs/faq.php?id=427&t=3

Re:Muse Spark Open by jubilee33 on Friday October 16, 2026 @08:19AM (#50117472)
Seems like you should flip your list and save time. I'm not sure why everyone is so ideological about tools. If it works well I use it. Refusals are a non starter.

Imagine if my table saw refused to cut out certain shapes. I mean once they are internet connected some brand will probably try that...but just why put up with it at all?

Re:Muse Spark Open by windexh8er on Friday October 16, 2026 @12:03AM (#50113933)
Because it's not as good IRL. I find that DS Flash does just as good of a job on most things.
Re:Muse Spark Open by bscphil on Friday October 16, 2026 @08:35AM (#50117609)
My experience has been that the "cost per task" estimate wildly underestimates the cost of top models on most real work, because the tasks (as I understand it) tend to be isolated bits of problem solving or code generation, which rewards writing code that works the first time and not generating too many reasoning tokens. Sol is great for both. However, a lot of real work involves tracking down bugs, working through tests, examining many parts of a codebase, refactoring... even with subagents and good tool use this tends to bloat context before the problem gets solved. Deepseek's absurdly low cache price really shines in these instances, but even GLM 5.3 Flash easily beats the top models on price in real world usage. Maybe that's different if you factor in subscriptions.
Re:Muse Spark Open by dbreunig on Thursday October 15, 2026 @04:59PM (#50110136)
I'm the weirdo who is clinging onto GLM 5.2. 5.3 seems to have picked up the bad writing from Opus 5.
Re:Muse Spark Open by chillfox on Thursday October 15, 2026 @07:46PM (#50111913)
GLM 5.2 was the first open model that I was happy to use as my main. It's a fantastic model. 5.3 cost a lot more per task, was slower and didn't get that much better results for me, so I tried 5.3 Flash for a bit, it was ok, but didn't feel as magical as 5.2 and I have now landed on DeepSeek 4.1 Flash as my main.
Re:Muse Spark Open by arcanemachiner on Thursday October 15, 2026 @07:20PM (#50111729)
I don't think that's likely, given the close release dates of both? I wonder if 5.3 just choked on too much RL slop, independently of Opus 5.
Re:Muse Spark Open by nullc on Thursday October 15, 2026 @10:44PM (#50113325)
It's not just you-- the claudisms in 5.3 are pathetic and drive me crazy, I was very sad to see it go in that direction and consider it a big step down in that respect.

BUT OTOH the performance of 5.3-flash are impossible to argue with-- it has the same pathetic claude behavior full of "Honest" and "lever" and "load bearing" tripe... but it's scortchingly fast on hardware that is pretty slow running 5.2/5.3.

As a result I use 5.3-flash a lot, in spite of feeling kind of gross about its language. I don't enjoy using it the way I enjoyed GLM-5.2, but its pretty effective.

Re:Muse Spark Open by andy99 on Thursday October 15, 2026 @05:50PM (#50110862)
Stupid question probably because I’ve gotten out of sync with the new releases but is Flash better or worse than vanilla 5.3? I’ve been using 5.3 extensively (and maybe ironically have used flash only for viewing images which I have as a skill in Pi because 5.3 vanilla seemingly doesn’t support vision). I find it generally competent but still very far behind opus 5.5. I’ve been trying to avoid Anthropic but the lift of opus vs GLM is enough that I end up coming back to it. I feel like currently there’s no comparison.

anyway, tldr I would not be happy if stuck with vanilla 5.3 so is flash better and should I move to that?

Re:Muse Spark Open by bigwheels on Thursday October 15, 2026 @05:58PM (#50110961)
GLM 5.3 Flash is better if you need native multi-modal capabilities, higher speed, and lower costs, but worse than vanilla 5.3 in raw reasoning and for complex programming.
Re:Muse Spark Open by colingauvin on Thursday October 15, 2026 @08:59PM (#50112395)
It is unreal how good GLM 5.3 Flash is. There's an ~4 bpw EXL3 quant that fits on 2x DGX Sparks. I'm not sure it's worth it now ($7k/Spark) or even when I bought them ($4k/Spark) but there's something about knowing that the whole internet could go down and I could still have this model running at 50-60 tokens/s with 1M context, indefinitely. It's just so good. TensorFold hits 100+ tokens/s in aggregate with 2-3 M total context pool. It's like having Claude at home.
Re:Muse Spark Open by bigyabai on Thursday October 15, 2026 @09:33PM (#50112663)
I'm using Flash and the full 5.3 model side-by-side, and for coding I'm hard-pressed to find a difference. Flash is slightly worse at steering in longer contexts, but writes code that's just as good as anything I get out of full 5.3. You could totally build a SaaS or video game with nothing but 5.3 Flash.

The full 5.3 is only really needed for super deep bugfixes/profiling passes, or ridiculous reverse-engineering in my experience.

Re:Muse Spark Open by colingauvin on Thursday October 15, 2026 @10:00PM (#50112930)
Yeah, MiMo Pro is the first Chinese model I can really clearly identify as an upgrade over GLM 5.3 Flash. K3, Qwen 3.8 Pro, GLM 5.3, DS4.1 Flash all feel around equivalent to me. Oddly lots of people have derided MiMo but I have found it able to tackle slightly tougher problems. For instance, I tried GLM 5.3 Flash, DS4.1 Flash, and MiMo Pro on making a new map layer for my Garmin watch, and MiMo was the only one that found the actual best method (.img file for OSM).
Re:Muse Spark Open by satirr on Sunday October 18, 2026 @12:13AM (#50137858)
Garmin.img thing is the interesting, but not really on my Experience.

Been running the same stuff on GLM 5.3 Flash, Deepseek v4p, Qwen 3.x model and Mimo v2.x pro model. Mimo faster yeah, but its hallucinate more and losses the thread easier. And when the context window is not limited, when usage of >200k token, is loss context. even he didnt know what im sayin.

Also, sometimes he Had a Loop over and over, for the same tool call (on my case is Grep and Read. >15x looping) until is burning a bunch of context for nothing. GLM and Deepseek doesn't do that (but the deepseek Hallucinations is huge asf too).

Is your Mimo do the same too??

Re:Muse Spark Open by lifeisloving on Thursday October 15, 2026 @11:24PM (#50113645)
In my experience, like most models, it requires a ton of supervision. It is extermely verbose. Flash loves to create some wildly unecessary abstractions/spaghetti code (like most llms). It wants to implement things in the least effecient manner and needs to be reminded constantly not to do things a certain way.
Re:Muse Spark Open by meeq on Friday October 16, 2026 @01:33AM (#50114464)
Am I the only one experiencing accidental chinese tokens, wrong grammar, and other sloppy mistakes in almost every output when using GLM 5.3? I feel like using it through any kind of subscription service (z.ai, Opencode Go) just serves me aggressive quants.

On some days it's usable, but on most days it feels more like having to trust a pretty smart intern that may or may not get a small stroke every couple of minutes.

Re:Muse Spark Open by disgruntledphd2 on Friday October 16, 2026 @02:10AM (#50114710)
I've had a bunch of failed tool calls and weirdness through Cursor with it recently but it's so good and so much cheaper that I put up with it.
Re:Muse Spark Open by nojs on Friday October 16, 2026 @02:31AM (#50114870)
That sounds like a bug. I’ve never had this happen, using it daily since release.
Re:Muse Spark Open by jubilee33 on Friday October 16, 2026 @08:12AM (#50117427)
Never had that happen at all and using since launch as well, but I have a very custom harness running. My local machine went down however and I used their web API to repair it and talking to that felt much stupider, I think harness really matters Alot.

I do get the really annoying LLM "that's A, not B" and "load-bearing" from time to time, those two tells stay no matter what I do even though the rest of the output is pretty amazing,

Re:Muse Spark Open by tough on Friday October 16, 2026 @09:50AM (#50118387)
We use GLM at work and YMMV a lot depending on the provider. So i guess some have bugs on their inference pipeline, or serve too agressive quants
Re:Muse Spark Open by marcosscriven on Friday October 16, 2026 @05:21PM (#50124935)
What’s a good provider?
Re:Muse Spark Open by colingauvin on Friday October 16, 2026 @02:44PM (#50122616)
I got this frequently with on GLM 5.3 Flash with GLM's actual plan, I do not get this with self hosting.
Re:Muse Spark Open by 59nadir on Saturday October 17, 2026 @04:35AM (#50129109)
I've only ever gotten sudden chinese (and some other script I couldn't identify) from GPT5.6 Luna via OpenAI. It's a very interesting failure mode.
Re:Muse Spark Open by Scaled on Friday October 16, 2026 @06:29AM (#50116628)
How do you feel it compares to DeepSeek 0731 if you have tried that? (Also runs well on 2x spark)
Re:Muse Spark Open by colingauvin on Friday October 16, 2026 @02:47PM (#50122657)
I was running DS731 until Qwen 3.8 Flash Next (so what, like a couple weeks? The pace has been torrid). I switched to Qwen for image input, then GLM 5.3 Flash. At the time, Qwen felt like a minor step down from DS731, GLM 5.3 Flash feels like a major step up from Qwen, so I'd say slightly better than DS731. I realize that's not exactly what the benchmarks say, but DS731 definitely flailed around on some of the more challenging things that GLM mostly gets for me.
Re:Muse Spark Open by Scaled on Friday October 16, 2026 @09:24PM (#50126908)
Thanks for the info, it encourages me to give it a try myself!
Re:Muse Spark Open by orbifold on Friday October 16, 2026 @01:26PM (#50121473)
How did you setup the 2x dgx spark?
Re:Muse Spark Open by colingauvin on Friday October 16, 2026 @02:43PM (#50122587)
Pretty much just this, but tweaked to have I think 64 active conversations be cacheable, which drops the context pool to ~2M.

https://github.com/MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Spark...

I also had to downclock my Sparks (Asus) slightly, to 2000 Mhz. Otherwise they'd overheat and crash.

Re:Muse Spark Open by oliwary on Thursday October 15, 2026 @04:12PM (#50109586)
The most important graph for me on Artificial Analysis is this: https://artificialanalysis.ai/?agentic-speed=intelligence-vs...

It shows the time to complete a task versus intelligence index, which is much more important for my use-case as I do quick back and forths usually. Someone on here suggested it to me a while ago. It also captures speed and token efficiency nicely, since both of these decrease time per task.

Re:Muse Spark Open by hadlock on Thursday October 15, 2026 @03:05PM (#50108665)
I agree, however It's too easy to game arbitrary limits like "medium". There is no olympics for "just rolled out of bed and slightly hungover"
Re:Muse Spark Open by RussianCow on Thursday October 15, 2026 @03:26PM (#50108943)
You don't need to directly compare the same reasoning levels. Today you can compare Sol on Low vs Haiku on Max, for example, but you can't look at GLM 5.3 High, only Max.
Re:Muse Spark Open by leidenfrost on Thursday October 15, 2026 @03:21PM (#50108861)
We don't even need a standard for reasoning levels.

Just accept whatever the provider names as "medium" and compare the accuracy between models vs the time it takes to complete a task.

Re:Muse Spark Open by SamuelAdams on Friday October 16, 2026 @07:15AM (#50116982)
The fashion industry has been doing this for decades. Medium means different things for different brands.
Re:Muse Spark Open by jchw on Thursday October 15, 2026 @03:26PM (#50108942)
For some tasks I genuinely feel Opus 5.5 does better on low than medium or high. The trouble, as always, is figuring out which.

Edit: I realize after commenting that you're not suggesting medium is useless, but just arbitrary and differs between models. Personally I don't want to directly compare mediums between models, I just want to be able to compare across effort levels.

Re:Muse Spark Open by disiplus on Thursday October 15, 2026 @03:41PM (#50109191)
i run opus on medium, sonet on high, and haiku on high for implementation based how complicated is for planing and specing and review opus is on high usually, except if if the task is contained.

did not try to run it on low, will have to try it.

Re:Muse Spark Open by jchw on Thursday October 15, 2026 @06:15PM (#50111159)
Opus 5.5 on low is good if you are relatively sure a given task is straightforward and time spent gathering background information is purely wasted or possibly even detrimental. Opus 5.5 low seems great at doing the most obvious thing based on the prompt without thinking much about it.
Re:Muse Spark Open by nl on Thursday October 15, 2026 @07:19PM (#50111719)
I agree - Opus 5.5 on low is a really great model if it has some direction and the tasks aren't complex.

If the tasks are complex cranking Opus up to high or xhigh and getting it to delegate to Opus low subagents is pretty effective and much faster than running everything at high.

Re:Muse Spark Open by eli on Thursday October 15, 2026 @03:25PM (#50108931)
A graph of time (or cost) per task against success rate does it. Just include all the different levels.
Re:Muse Spark Open by Kinrany on Thursday October 15, 2026 @03:43PM (#50109214)
This exactly. The three interesting parameters are total time to answer, total cost, and output quality (for any given purpose).
Re:Muse Spark Open by esafak on Thursday October 15, 2026 @03:28PM (#50108974)
That's the beauty of the line charts; the labels are arbitrary points on the line. You can easily compare the series visually.
Re:Muse Spark Open by disiplus on Thursday October 15, 2026 @03:44PM (#50109236)
this is properly bad as you can run glm on 6000pro, at (in my eyes) pretty good quant, and a q2 on a 5090. im suprised at how good 5.3 flash is at some parts. i still think ds4.1 flash is better but it is more demanding. i hope meta pushes something in a range of 300-400b with a 15-20b active parameters. i know its a crowded space. but it would be fast and probably mostly good enough.
Re:Muse Spark Open by Melatonic on Thursday October 15, 2026 @07:05PM (#50111593)
Have you tried two 6000 Pro Blackwells with a model spread between both ?
Re:Muse Spark Open by drob518 on Thursday October 15, 2026 @04:09PM (#50109545)
My immediate reaction is that it’s weaker on coding than other open-weight models and that coding is a primary use case for running locally (not the only one, but surely a major one). That said, it takes a lot to get full Kimi and GLM running locally.
Re:Muse Spark Open by AustinDev on Thursday October 15, 2026 @03:53PM (#50109341)
Yep, just benchmarked it against GLM5.3 Flash on my own local bench and it's slower in TTFT, tok/s and concurrency. It also scored lower.

Tried different reasoning levels and didn't really see any difference but GLM 5.3 flash only really works high, max or none so I couldn't really compare the lower efforts which appear to work on Spark.

Re:Muse Spark Open by RussianCow on Thursday October 15, 2026 @03:29PM (#50108986)
> As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases. I would really like them to start providing comparisons at low and medium reasoning levels across these models because those tend to be the ones that actually get used in production.

The frustrating thing is they started doing this for the big labs—you can see their benchmarks for all reasoning levels of Sol 6.1 or Haiku 5.5. They just don't seem to do it for any of the open weight models.

Re:Muse Spark Open by Xenoamorphous on Thursday October 15, 2026 @03:28PM (#50108970)
There’s a reason they put the “US” in “the most capable US open weight model”.
Re:Muse Spark Open by aniviacat on Thursday October 15, 2026 @03:41PM (#50109176)
Artificial analysis puts it next to Mistral's new model.
Re:Muse Spark Open by halJordan on Thursday October 15, 2026 @03:50PM (#50109310)
Which is crazy bc they promised to open source muse 1.2 which is good. And they give us this bait and switch
Re:Muse Spark Open by lmeyerov on Thursday October 15, 2026 @03:50PM (#50109304)
yeah for botsbench.com side where we do cybersecurity investigation evals of models + harnesses, we've been having to play with:

1) time budget model gets (5min / task) -- this became a real problem with opus/astra/fable at high+

2) tuning which effort level we report on (when not all)

Re:Muse Spark Open by jmspring on Thursday October 15, 2026 @03:23PM (#50108896)
Why didnt thry include OpenAI or Anthropic models? I get they were comparing open weights.
Re:Muse Spark Open by sebiandev on Thursday October 15, 2026 @03:05PM (#50108655)
very true. when configuring custom agents for enterprise, we almost always default to none or low reasoning knobs. The added latency from medium or higher reasoning levels compared to the improvement in response quality is a diminished return.
Re:Muse Spark Open by segmondy on Thursday October 15, 2026 @04:06PM (#50109505)
AA is a joke.
Re:Muse Spark Open by TZubiri on Thursday October 15, 2026 @05:58PM (#50110960)
it’s quite suspect to claim that a model is ‘underperforming’ so resoundingly, especially after so little time since release.

there’s this objectivist approach of reducing llm quality to a single dimension, furthermore one with measurability bias.

my thesis is that the chinese models beat the american models on benchmarks, on quantitative measures, but probably do worse in subjective measures. to put an example, compare a chinese phone with an iphone, the specs are great, but the OS? the materials? the privacy posture?

so will chinese llms have great specs and benchmarks, but will miss the mark when it comes to the actual quality in practice when your input stops looking like the benchmarks it was trained on.

America number one, it’s the chinese that play catch up with clones, not the other way around

Re:Muse Spark Open by gigatexal on Thursday October 15, 2026 @06:50PM (#50111480)
Um a 512GB m5 max Mac Studio is what 25k? A B300 GPU and the computer to use it is what? Double? Triple that? Apples v oranges my friend. Apples v oranges.
Re:Muse Spark Open by nl on Thursday October 15, 2026 @06:51PM (#50111488)
> As an aside, it's really unfortunate that artificial analysis only provides comparison at the highest reasoning level

They do other reasoning levels for some models.

Here is Sonnet 5.5 across the full range of efforts from low to max: https://artificialanalysis.ai/?models=claude-sonnet-5-5-low%...

Opus 5.5 low to max: https://artificialanalysis.ai/?models=claude-opus-5-5%2Cclau...

Here is Muse Spark 1.3 Max and Xhigh: https://artificialanalysis.ai/?models=muse-spark-1-3%2Cmuse-...

I'm guessing it depends how much the vendor wants to contribute. Running these benchmarks isn't cheap!

Re:Muse Spark Open by eek2121 on Thursday October 15, 2026 @07:08PM (#50111610)
I feel like reasoning is a useless metric, and we should instead focus on results, and time to achieved said results. That is a more useful metric overall.
Re:Muse Spark Open by viraptor on Thursday October 15, 2026 @07:30PM (#50111801)
Reasoning level influences both of those things dramatically. So we need to have it available as a parameter for comparisons.
Re:Muse Spark Open by throawayonthe on Thursday October 15, 2026 @07:11PM (#50111650)
not a metric, it's a parameter used at inference time
Re:Muse Spark Open by arcanemachiner on Thursday October 15, 2026 @07:18PM (#50111713)
Yes, but what metric defines what constitutes a given reasoning level? That's what's arbitrary here, not the label (which is also arbitrary).
Re:Muse Spark Open by the_real_cher on Thursday October 15, 2026 @07:26PM (#50111776)
Ahat's the cost of a b300 versus two mac minis?
Re:Muse Spark Open by viraptor on Thursday October 15, 2026 @07:32PM (#50111813)
How much longer does it take to ask this and wait for response compared to checking yourself?
Re:Muse Spark Open by the_real_cher on Saturday October 17, 2026 @07:59AM (#50130326)
I'm making a point.
Re:Muse Spark Open by viraptor on Saturday October 17, 2026 @09:56AM (#50131223)
You're asking a question, which if people follow up with searching may lead them to the implied point. Just post the numbers if you think the numbers are interesting ¯\_(ツ)_/¯
Re:Muse Spark Open by the_real_cher on Friday October 23, 2026 @06:45PM (#50216410)
You should google rhetorical questions ;)
Re:Muse Spark Open by hatthew on Thursday October 15, 2026 @09:04PM (#50112426)
> only provides comparison at the highest reasoning level, which is mostly unusable for high-volume production use cases

Similarly, AFAIK evals tend to be performed at native precision, while anyone who cares about speed is probably running quantized.

Re:Muse Spark Open by colordrops on Thursday October 15, 2026 @09:37PM (#50112685)
Qwen 3.8 flash next runs pretty damn fast on my 5090, no need for a b300.
Re:Muse Spark Open by jnaina on Thursday October 15, 2026 @09:46PM (#50112775)
Yes, running Qwen 3.8 flash next on a 5090 with 96GB of RAM, using Strata. Fast and good.
Re:Muse Spark Open by sheepscreek on Friday October 16, 2026 @12:31AM (#50114097)
Model X vs Y is the new Vim vs Emacs.
Re:Muse Spark Open by walrus01 on Friday October 16, 2026 @04:55PM (#50124541)
> Artificial analysis shows this: underperforming Qwen3.8 Flash Next and GLM 5.3 Flash, both of which you can run on a single B300

You don't need anywhere near a B300 to run Qwen 3.8-Flash-Next in its Q8 largest variant published as a GGUF by unsloth. It fits very comfortably in a 256GB system.