jbellis

Not sure why you're getting downvoted. I haven't resolved conflicts by hand in almost a year.

I spent several weeks working on trying to get swarms of cheap models to match the coding ability of a single expensive one. My take: it's remarkably easy to build a harness that improves a single eval, and remarkably difficult to build one that generalizes.

They walked that part back within days. Still made me permanently leery off relying on anthropic for anything important.

If you like Herdr, you might also appreciate https://github.com/BrokkAi/mjolnir/, which is similar in its multiplexing features (although built on ACP) but offers cross-provider subagents, memory and session log sync, live migration of sessions across profiles and providers, and load balancing across multiple subscriptions.

Yes. And this is the right tradeoff vs the ossification you saw previously in Java (and we're starting to see now in Python, see the recent ejection of the CPython JIT), especially now when you can say "GPT, source dive the JDK and figure out which cutting edge is causing the bleeding."

It's not that hard to create your own uncontaminated eval for the problems you care about, and compare.

Here's mine for coding in Rust, and I definitely see the same pattern of "most chinese models do much worse than on public evals that have been around long enough to train on" that AA v5 surfaces: https://slopcop.com/power-ranking

My iPad Pro is six years old now, the last generation before they started putting M chips in them. Still not interested in buying another until I can run a Linux VM on it.

As someone maintaining a close to 3M life Rust codebase, the downsides are real, especially rustc speed. I contribute to the mbx cache project but it only helps so much. It's only a small exaggeration to say that I fantasize daily about pursuing to Go.

On mobile, but I linked one of the relevant issues in my writeup here: https://blog.brokk.ai/infrastructure-for-agentic-rust/

In general, it really does reduce the prevalence of something to ban it or nearly so. To take examples that both the American left and right hate to think about: - Banning abortion really does make it harder to get one. - Banning handgun sales really do make those harder to get as well.

(And yes, I am using the word "ban" as shorthand for "extremely tight/onerous regulations in red and blue states, respectively".)

Legalizing sports gambling was the biggest unforced error in United States public policy in living memory. I expect that almost nobody reading Hacker News has a gambling problem, but people at the other end of the economic spectrum are being absolutely ripped up by it. Unfortunately, it's probably not going to change soon. Too much money to be made now.

Reasonable overview here and it's only gotten worse since 2024. https://thezvi.substack.com/p/the-online-sports-gambling-exp...

this looks like a bug, but even normal happy-path codex creates many GB of logs, openai apparently considers this fine bc it's been going on for months

two things help:

1. fs-level compression, I use ZFS bc Btrfs has massive contention problems with concurrent Rust builds

2. Archive your old sessions using something like sessionwiki (https://github.com/youdie006/sessionwiki), I've integrated this into my control plane (https://github.com/BrokkAi/mjolnir/)

I built http://github.com/brokkai/mjolnir/ to do exactly this cross-harness over ACP. As a bonus it can let Opus drive Luna as a subagent, which is just ridiculously cost effective. (5x clst multiplier means Haiku is much more expensive for realistic task sizes.)

I built mjolnir in large part so I could have Opus manage DeepSeek Flash subagents. It's phenomenal and extremely light on the Claude tokens. https://github.com/BrokkAi/mjolnir/

And yes, Opus is enough smarter than DSF that it's worth the extra steps. This ranking is from live tickets, no contamination: https://slopcop.com/power-ranking

A couple recs for sccache, which is great, but I upgraded to mbx (https://github.com/jdx/mr-boxington/) which goes beyond sccache by wrapping cargo to give you better rust-specific management, including incremental compilation.

My harness control plane will autoconfigure mbx across all your machines, if you like. https://github.com/BrokkAi/mjolnir/

Yes. And on subscription pricing it's even cheaper per task solved than DeepSeek.

It is. TY!

There's a bunch of skepticism in the replies but I ran over 100 tasks against DeepSeek 4.1 Flash and Sol (among others) and I can confirm, it is in fact a little smarter than Sol and a little more expensive than Luna. https://slopcop.com/power-ranking?pricing=api

I also spent $280 on DeepSeek doing the tests (direct to DS, not OpenRouter). I suggest that if you can't conceive of anyone spending $200 on DeepSeek, you're not being ambitious enough!

The whole point is "We baked the best defaults in for most people and most use cases, here are some cases where you might not like the defaults and how to change its behavior effectively."

Because vanilla pi doesn't do a lot of important things out of the box, and they didn't want to open the "which extensions should we add" can of worms.

Anthropic knows that the benchmarks showing Opus 5 better than Fable 5.1 are measuring something that's less than entirely useful.

For the mj adversarial review we use a configurable coordinator (I recommend Fable) with six subagents (I recommend Luna) for specific specialties like code complexity. Narrowing down what a smaller model like Luna needs to look for / care about helps them do better work; then the larger model synthesizes and fills in any gaps.

https://blog.brokk.ai/mjolnir-automated-cross-vendor-adversa...

Oh, cool. I used tailscale to provide a private web app for my cross-harness orchestrator, too. (https://github.com/BrokkAi/mjolnir/)

While I'm generally sympathetic to the idea that the public coding benchmarks are inadequate, to the point that I've written my own in the past and will likely do so again, the complaint here is that "these tasks don't match what I do in my day job" which is ~always going to be the case. The hope with benchmarks is that you can capture properties that generalize, from examining performance against small set of tasks, and I do think that this is at least directionally true for well-designed evals.

(There's https://withspecific.com/benchmarks/real-swe but since the tasks are private we still don't really know what they're representative of.)

> I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands. And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.

Good thread here.

https://x.com/DavidRBellamy/status/2099187370407112758

One thing we've noticed is that if you have model X implement AND review the same change, the reviewer often shares the same blind spots as the implementer. https://blog.brokk.ai/mjolnir-automated-cross-vendor-adversa...

If you're more of a SWE than a data scientist, check out Mjolnir: https://github.com/BrokkAi/mjolnir/

Built in containerization and EC2 support, allows moving sessions across harnesses/profiles/machines

This is only correct if you ignore subscriptions. I ran the numbers on what you get for your subscriptions last week: https://blog.brokk.ai/a-coding-subscription-tier-list/