Re:Claude Fable 5: mid-tier results on coding tasks
by m-dot-reviews on Thursday June 11, 2026 @08:51PM
(#48498480)
I'm starting a repository of LLM reviews [1] with the goal of creating a catalog that is more task-oriented and less marketing-y than corporate blogs or benchmark leaderboards. You seem to have a lot of experience across a bunch of different models: if you have a chance and feel like sharing, you'd be one of the first.
[1] - https://model.reviews/ - all the user-submitted content is CC licensed and will be available for download in periodic dumps.
Re:Claude Fable 5: mid-tier results on coding tasks
by munksbeer on Friday June 12, 2026 @10:57AM
(#48505026)
Does your team then manually decide the results by going over the PRs? I suppose you know what you're looking for now, but isn't this still quite painful?
Re:Claude Fable 5: mid-tier results on coding tasks
by afro88 on Friday June 12, 2026 @10:22PM
(#48511990)
We selected PRs (real ones we merged over the 6 months prior) and have an "LLM as judge" score how close the AI generated code is to the PR. Same as how other benchmarks do it, but it's with tasks we actually do and code we have decided is actually up to scratch for us
Fable 5 sits ahead of Opus 4.7, but behind Opus 4.6, Sonnet 4.6, Opus 4.8, GPT-5.4, GPT-5.5.
Fable isn't a good coding workhorse. That doesn't mean it's not good for actually complex problems and long horizon tasks (big POCs, complex research and such). But I only have vibes and Anthropics own benchmarks and marketing to guide me there.