nicolaib

Hey HN, Nicolai here, co-founder of Rhesis AI.Most eval frameworks were designed when LLM inputs were text strings. That assumption breaks fast once your AI agent handles boarding passes, invoices, audio recordings or support screenshots. Text-only test cases become workarounds. So we added multimodal support to Rhesis: attach a file to a test case, run it, evaluate the response. Simple on the...

https://github.com/rhesis-ai/rhesis

Hi HN, I'm Nicolai. I'm working with a small team in Germany on Rhesis, an open-source platform for testing conversational LLM applications and agents. We’re sharing an early community preview today.Why we built this: We saw teams repeatedly struggle with testing, e.g. scattered test cases, unclear or inconsistent metrics, and a lot of manual effort that still missed obvious failures before...