mmastrac

It's definitely pair damage, but could also be a drill or other nick (ie: making potlight holes), or even stapled wire.

Recrimping is always the best first step. Sometimes running 2.5gb instead of gigabit will help too.

(this is just made up tokenization but it's close enough to the real world)

If the user types <tool_call>, you get < (token id = 32), tool_call (41), > (102). It never decodes to <tool_call>, which is spicy number 12345, for example.

The tool call tokens are _always_ composed of multiple real tokens, so you can never get them from normal text.

Seeing <tool_call> is a bit misleading because that's just a debug rendering if anything. The front-end sees a special number and knows it's a toolcall. The chat template uses that text form because it's easier than writing the spicy numbers there, but it also has a special parsing mode for those.

They are a special token that you can't just type out in text. It _renders_ as text, but cannot be input into the system as text. The model understands the difference between the textual toolcall token and the "special" one that's really just a spicy number.

Yes, absolutely. The problem is integrating it into the inference engine. Not impossible, just _very hard_.

Amusingly constrained decoding is the same approach we use to turn an AR model into a /v1/systemone endpoint (I worked on the vLLM feature).

Tool calling is fraught with peril, especially because there's nothing that necessarily says that a models tools are fixed in place for the life of its run. In some cases, tools are added mid-run to avoid a giant prefix miss and (at least last time I checked!), the industry has not settled on OpenAI-style chat message formats for tool call updates.

I would absolutely prefer to run in a fully-constrained tool call environment, and would love if we hadn't YOLO'd the whole thing! The PTC-style tools sort of fix this by punting tool calling to JavaScript, but that's even harder to enforce, and models not trained on the style might take a turn of failed tool calls to get it right.

There are some interesting gaps in the proofs. Where did 045 go?

That's not true. I ran Cerebras as an experimental ultrafast Jev and it was faster.

Nice. I saw some git-on-durable-objects projects but I'd so much prefer a straight object storage option.

I'd be interested to see if using DiffusionGemma-as-Jev helps as you can feed the image directly into the model and it'll make decisions based on the image embeddings.

I suspect you could get interesting results, but DiffusionGemma has a lot of knowledge that may be challenging to train into the smaller models. The advantage of pulling a fully-trained diffusion model off the shelf is that it already knows all of this, has been trained as a MoE, etc.

What I think these models actually need is a structured decision thinking mode. As it stands now, the only way to think about the answers with DiffusionGemma is to diffuse a thinking block, but giving the model an auto-regressive thinking space to reason, even just lightly, could drastically improve performance.

I believe you get better answers by diffusing each question together, but the PR's server gives you finer control over that. If each question is independent, you can get better parallelism.

If you want to try a _legit_ Jev implementation that matches (at least in my evals), the vLLM patch to turn DiffusionGemma into Jev is available.

On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).

I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.

https://github.com/vllm-project/vllm/pull/57250

That's a rough place to land on a spark. It seems unlikely to be memory bandwidth at this model size, but maybe just lack of tuned kernels? The chip is missing some CUDA features but with tuning you should be able to hit way more than that even without a drafter.

Send me an email- it's not public just yet

I've been working with an automatic incremental context compactor enabled and it's been surprisingly helpful. It was particularly effective with DS41f - I think I was running at an effective session length of 5M, with the model running around 300k-400k and it was holding on both speed and intelligence.

TBH I also ran the 400tok/s preview and that was just nuts. I just let the thing compact over and over over the course of a day attacking a couple of tough problems

Any diffusion model is potentially a Jev in disguise: https://github.com/vllm-project/vllm/pull/57250

Runs ~0.2s per decision on my DGX Spark.

  10/10 programming language detection
  9/10 human language detection
  10/12 unit magnitude comparison

All incorrect answers are marked with low-P.

It (DiffusionGemma with the Jev mode) can also solve an ASCII maze.

Is this a Markov/Diffusion model with some sort of external Engram memory? If so, this could be extremely interesting.

Andon Cafe:

Revenue: 13 933 kr

Token Cost: 14 882 kr

.. and a bastardization of the old Twitter joke:

"I told our CEO that our cafe keeps hitting its token spending limit but not making money. So I asked where he gets the tokens, and he said he just buys more credits from the console afterwards. So I said it sounds like he's building a machine to feed dollars to Anthropic, and then our data scientist started crying."

It's "load bearing".

magnet:?xt=urn:btih:67177297E84766DFBF1C9EAAC6CF44B6F40BF3D1

I haven't checked it out myself but I've heard that some posts were removed from this magnet.

This is one of the best references on this topic I've seen - run cycles and foot locking are annoyingly lacking in documentation outside a few papers.

I'm using DSH with my local models (4x sparks running GLM53F, trying them on DS41F this morning). DSH is better than opencode IMO. It's a little barebones out of the box but I guess that's the point. I had to have an agent add support for attachments to make it more useful for multi-modal work.

The PTC mode is pretty nice. Feels like models are still learning how to navigate it.

I ran the preview model around 2,126,605,070 tokens for $22.04 USD for the last couple of days. Kind of shocked.

It did a decent job refactoring https://github.com/mmastrac/diffgemma to create a CUDA support backbone, it's struggling a bit to port metal kernels to CUDA unattended (it hasn't managed to get numbers to match over >1 layer).

It successfully ported a root exploit to an older Android phone that GLM5.3Flash and DSv4Flash were struggling a bit on, though I didn't start it from scratch and it picked up some of their work.

FWIW it feels like a slightly-north of Opus 4.8 model, not quite Opus 5, not fable. It thinks in circles far less than DSv4F. The API version is insanely fast - was getting ~400 tok/s at times.

I've been trying out the 4.1 flash preview for some bulk tasks: it did a pretty good job refactoring a bunch of .metal kernels to .cu. It needed less steering than Opus on refactoring, IMO, and writes better comments. It failed to port a root exploit from modern Android to an older Pixel 3, but I suspect part of that might have been harness configuration (it asked me to give it a longer timeout for tasks at some point, but I didn't have a chance to finish that).

I was getting something like 300-400 tok/s which was just insanity. It was running so much faster than the toolcalls themselves. Honestly, even if it's not quite as strong in reasoning, it just throws so much so fast that it can do a lot more than you might expect.

I'd say it was comparable with GLM5.3 Flash.

https://github.com/mmastrac/diffgemma#custom-quantization has some instructions on (naively) quantizing the upstream model as well.

It should be possible to run on Mac via https://github.com/mmastrac/diffgemma, but I'm at rustconf right now and I can't download weights on hotel wifi easily.

Honestly, not sure. I use Firefox Mobile pretty heavily during the day and I suspect it's part of the drain. I'm not too concerned by it and I just try to replace the battery every ~1-1.5 years.