News for hackers. Stuff that matters.
Re:Show HN: Strot – Compile an LLM's intent once, run decisi... by solohunter001 on Monday October 19, 2026 @12:36AM (#50148420)
Hi HN. STROT compiles a decision task — "route these tickets", "score this feedback", "classify this text" — into a standalone pipeline. A large LLM is used once, at compile time, to synthesize the labels, the input projection, and the handlers. At runtime a small local model (144M, CPU) makes every decision: ~60ms/record, $0 marginal, zero LLM calls, bounded typed-JSON output.

The goal was to move the expensive, nondeterministic part (an LLM) entirely to build time and keep runtime frozen, cheap, local, and auditable — the way a compiler moves work off the hot path.

What I think is actually interesting, beyond "small model does inference":

- The compiler measures instead of guessing. It scores candidate labelings on a teacher-labeled sample and keeps the winner; on the identical model that's +18pp on a routing benchmark (92.5% vs 74.8%) and +14pp on text classification (86.7% vs 72.7%) over naive integration. - It fits a conformal-calibration threshold at compile time, so low-confidence records abstain with a marginal error guarantee rather than guessing. - A golden-record gate rejects any compile that fails known-answer examples — synthesis is stochastic, and I show that variance in the benchmarks.

What it doesn't do: it can't make a small model understand things it fundamentally can't. On tweet hate-speech it hits the model's ceiling; the fix there is a task-trained model in the swappable slot, not the compiler. All of it — including the losses — is in the repo's benchmarks with reproducible scripts.

There's a browser demo (a 144M model ( which is changeable ) playing Pac-Man, one decision per frame) and a quickstart that runs in a few lines. MIT licensed, pure-stdlib core.

Happy to go deep on the calibration math, the compile/cache design, or where this does and doesn't make sense.

Re:Show HN: Strot – Compile an LLM's intent once, run decisi... by sysguest on Monday October 19, 2026 @01:08AM (#50148588)
> Happy to go deep on the calibration math, the compile/cache design, or where this does and doesn't make sense.

yes plz

Re:Show HN: Strot – Compile an LLM's intent once, run decisi... by razingeden on Monday October 19, 2026 @01:40AM (#50148772)
>On tweet hate-speech it hits the model's ceiling

This sounds like a sus endeavor to redline your LLM with, could you elaborate ?

Re:Show HN: Strot – Compile an LLM's intent once, run decisi... by dpacmittal on Monday October 19, 2026 @01:19AM (#50148652)
Was this entirely vibe coded? The readme is very obviously Claude with no human edits at all.
Re:Show HN: Strot – Compile an LLM's intent once, run decisi... by nodja on Monday October 19, 2026 @01:36AM (#50148741)
Yes, even the comment you replied to is very noticeably claude authored.
Re:Show HN: Strot – Compile an LLM's intent once, run decisi... by majorchord on Monday October 19, 2026 @11:48AM (#50153879)
You could not even take the time to write your own comment here?
Re:Show HN: Strot – Compile an LLM's intent once, run decisi... by solohunter001 on Monday October 19, 2026 @11:54PM (#50161164)
Thanks for feedback, I will try doing this going forward