msp26

Fortunes weave doesn't even run at 60 frames. The hardware could be better.

These are hilarious thanks for the rundown

> Our friends at Hugging Face

Hahahahhahahaha

Claude Code Remote control only works with telemetry enabled too.

This Luna pricing is obscene man. 5.6 was good enough for so many use cases (data analysis, structured extraction etc).

Incredible.

Got it, I'll test more deployment params when I benchmark models for vision tasks.

Do you have any tips for Gemma 4 31B in particular? I quite like the model but I feel like I'm underutilizing my rented GPU hard due to skill issues. Throughput should be way higher than this, right?

---

Gemma 4 31B NVFP4, vLLM 0.29, single B200 (modal), FlashInfer, fp8 KV, prefix caching, 32k ctx

Workload: ~6k-token shared prefix (~98% cache hit) + short input, ~250 tokens out, ~250 seqs running. Get ~5-6k output tok/s, flat from 128 to 384 concurrency.

Spec decode (DFlash, n-gram) didn't give a good boost which is annoying because I feel like this task should be easy for a draft model to predict.

Individual request latency doesn't matter I just need as much throughput as possible. Its only active for a few hours when I need it.

Do you have any advice on this front? I use vllm for a project but only for text tasks at the moment.

Relatable. Every time I finish working on something, I spot so many more new things that can be built.

It's completely mental that HF ran into cyber safety blocks trying to use OpenAI models to help defend against the attack. They could only rely on a local hosted chinese model in the end.

Not sure how to fully fix this but I remember a session last week where I got so fed up mid way though reading a response that I used the following:

"give me this again without jargon invented this session at high density

and with a couple (maybe more or less) simple useful ascii diagrams underneath each design"

The context is that I was discussing an experimental new idea for my video game review analysis product.

Designs 1,2, and 3 were horrible: the model even suggested a rejection after the word soup so it would have been pointless to waste my fleeting time on Earth reading it.

Otherwise, I generally really enjoyed using fable for bouncing ideas. It was an absolute joy to have this thing provide useful criticism, analyse sample data, and create prototypes so that I could elevate my understanding of the problem without stepping down from a pure intuition/design headspace.

But I don't consider the purely model written code usable for a feature this important. I'll probably scrap it entirely and start from scratch with newfound understanding.

No the models are just ass at communication without being directed.

Try asking them to make useful diagrams for some stuff in a codebase, out of the box without excessive hand holding they don't make good choices about what's worth communicating and how to do it.

You see this in their pointless frontend copy all the time too.

yep matches my experience completely

But even fable has the annoying tendency to invent new jargon and produce an incomprehensible soup of text.

the hand drawn diagrams and highlights are charming

Asking fable to read it's own model card triggers this btw. Or asking if mitochondria is the powerhouse of the cell.

Last I heard, the feature was in beta so avoided it. But I'll definitely give it a go if it's mature now!

I have been using the --watch flag to let my agent play with the notebook as I use it already.

I do that a decent chunk of the time yeah especially for learning. I also have a bunch of marimo notebooks that double as clis and they're lovely.

But sometimes I want to do something too specific or high fidelity and it's just easier to get the clanker to write typescript and make a webpage/components.

I fucking love marimo for exploring data.

However my use of it has decreased a little with how easily I can conjure disposable frontends with agents to explore one off things.

hello, please fix needing to reauth every day (sometimes with email verification). This started happening this month. It's tedious and makes switching very tempting.

I'm using the VS Code extension over SSH.

Deeply unserious company, flip flopping on policy every week with ludicrous, sometimes invisible, guard rails on their top model. Imagine trying to make business decisions about AI use with this.

I love Fable for many things including coding but I cannot justify basing any internal AI tools or LLM powered products on top of Anthropic's offerings.

> Summarized thinking provides the full intelligence benefits of extended thinking, while preventing misuse.

> preventing misuse.

Imagine not being able to read the tokens you are paying for.

Yep agreed completely. I couldn't imagine torturing myself with a small model for local coding. But Gemma 4 31B is so fucking good for a variety of language modelling tasks.

It triggered for me when I asked "Web search for your own model card (released today) and pick out your favourite highlights from the pdf"

>Pricing for both models is $10 per million input tokens and $50 per million output tokens.

hell will freeze over before anthropic release anything meaningful to the public

Interesting, I might try that, thanks!

Google is singlehandedly carrying western open source models. Gemma 4 31B is fantastic.

However, it is a little painful to try to fit the best possible version into 24GB vram with vision + this drafter soon. My build doesn't support any more GPUs and I believe I would want another 4090 (overpriced) for best performance or otherwise just replace it altogether.

I like starting most of my projects on marimo notebooks now and slowly moving parts of it to the main codebase + db.

By the end of it I might remove the notebook entirely but usually I keep it for some visualisation + running stuff as a cli tool.

session usage limits this week feel like ass. Even when being careful to not break prefix caching.

Not necessarily with speculative decoding. Whitespace would be trivial to predict and they would petty much keep using the same amount of compute as before.

I don't think that's their primary motive for doing this but it is a side effect.

They don't have the compute to make Mythos generally available: that's all there is to it. The exclusivity is also nice from a marketing pov.