News for hackers. Stuff that matters.

A comment on Show HN: Tier – Adaptive tool routing that makes small LLMs 10pt more accurate.

Re:Show HN: Tier – Adaptive tool routing that makes small LL... by aayushkumar121 on Wednesday April 15, 2026 @01:51PM (#47782673)
Interesting idea especially the “navigation vs selection” framing.

In practice, how are you measuring the +10pp gain? Are you using fixed eval sets or something more dynamic?

I’ve seen small models look better on benchmarks but regress pretty quickly once prompts/tools change slightly, so curious how stable these gains are over time.

[ Parent | on HN ]
Re:Show HN: Tier – Adaptive tool routing that makes small LL... by pranabsarkar on Wednesday April 15, 2026 @02:01PM (#47782837)
Fixed eval — 80 tools, 200 queries, 4 model sizes. +10pp came from "all tools" vs "tiered" on 1.5B.

You're right about stability. Haven't run rotated/rephrased evals yet. The 89% baseline (when models knew where to look) suggests selection capability is fine, but I'd expect some regression with adversarial prompts.

[ Parent | on HN ]