News for hackers. Stuff that matters.

A comment on Ask HN: Do people want to AB test LLMs?. Parent comment

Re:Ask HN: Do people want to AB test LLMs? by k11kirky on Thursday October 10, 2024 @01:58PM (#41801435)
Thanks, do you think there is value in evals? / do you use evals while testing prompts?
[ Parent | on HN ]
Re:Ask HN: Do people want to AB test LLMs? by hypoxia87 on Thursday October 10, 2024 @04:59PM (#41803541)
Jumping in based on our experience in case it's helpful:

- Evals are very useful and a core part of the best practice for creating LLM apps

- There are already excellent solutions for model and prompt eval (etc.) including Parea from YC and many others

[ Parent | on HN ]