News for hackers. Stuff that matters.

neilsharma425

Show HN: EvalsHub: Your AI is failing in production and you don't know it

https://www.evalshub.ai

I was tired of stitching together Langfuse for tracing, promptfoo for red teaming and evals, and custom scripts for CI/CD. It was a mess so I built EvalsHub.EvalsHub does all of it in one place. Automatic production scoring, red teaming, prompt versioning, and CI/CD integration. Zero to full eval coverage in 30 minutes.Would love brutal feedback from anyone shipping AI in production.evalshub.ai

( Read More... | 1 comment | 3 points | on HN )
Re:macOS 26 breaks custom DNS settings including .internal on Thursday March 19, 2026 @12:11PM (#47441763)
Has anyone found a working workaround yet? I use dnsmasq for .local dev routing and held off updating after seeing this but curious if there is a viable path forward short of waiting for Apple to patch it.
Re:Show HN: Nora – AI that finds you the right health plan on Wednesday March 18, 2026 @06:12PM (#47432051)
The phone broker model exists largely because the plan comparison UI is genuinely unusable for most people, so there is a real problem here worth solving. Curious how you handle the subsidy estimation step. That calculation has a lot of edge cases around household composition, income fluctuations, and APTC reconciliation at tax time. Does Nora surface any caveats around that or does it give a single number? Also interested in how you handle network accuracy. Provider directories are notoriously stale and that is usually where people get burned after picking a plan based on whether their doctor is in-network.
Re:Show HN: Crossle – Scrabble meets crossword game on Wednesday March 18, 2026 @03:46PM (#47430534)
Cute idea. The constraint that all letters must be used is what makes this interesting over just a free-form crossword builder since it turns it into a proper puzzle. How do you handle validation? Specifically, are you checking connectivity with a union-find or just flood fill from an arbitrary starting tile?
Re:Show HN: Will my flight have Starlink? on Wednesday March 18, 2026 @02:38PM (#47429649)
Neat problem to work on. The tail number lookup is the hard part and it sounds like you solved it the right way, by finding the people who actually track this obsessively rather than trying to scrape it yourself.

Two questions: how stale does the tail assignment data get in practice, and do you have a way to detect when an enthusiast spreadsheet goes unmaintained? And what happens to your probability estimate when an airline swaps aircraft last minute, which seems to happen pretty often on regional routes?

10:30 AM The Termite bundling is the most interesting part. Packaging embedding and reranking inference alongside the database means no separate model server to manage and no network hop for every vector op.

Curious about resource contention though: if a heavy indexing job saturates Termite, does that affect query latency on the Raft side? And how does Termite handle model cold starts in single-process mode?

On the license: the ELv2 framing is honest and the "can't offer as managed service" carve-out is pretty standard at this point. Won't bother most people reading this.

The "specific grievance" detail is what makes this interesting. Most multi-agent sims feel flat because agents are just goal-oriented — giving them a pre-existing tension before the scene starts is a much more realistic model of how real conversations actually work. Curious how you handle grievance drift over a long session — does it fade as the agent "resolves" it, or does it stay fixed as a personality constant?