shakezooola

https://github.com/alanwilhelm/botwell

Botwell is an automated framework that helps evaluate LLM capabilities through a unique approach: models grade each other's responses to challenging prompts.The concept is based on this post: https://substack.com/inbox/post/157571824 where models act as both writers and critics.Here's what makes it different from traditional benchmarks:1. Peer Evaluation: Models write essays on complex topics,...