There are tools like promptfoo that can be used for internal benchmarks and evaluations that allow you to compare results across different models and runs. This can be useful for personalized benchmarks and for business applications where you need to evaluate models and/or system prompts.
Re:Qwen4-Max
by satvikpendem on Monday November 16, 2026 @12:27PM
(#50520175)
Why wouldn't the LLM provider know about your internal benchmarks when they use prompts and chats as training data? They can easily learn about your benchmarks too.
Re:Qwen4-Max
by MetalBreaksAndB on Monday November 16, 2026 @01:00PM
(#50520636)
Just use ZDR.
This is a permanent problem in benchmarking really. Transparency is required for trust but invites over-tuning.