DeepEval – Unit Testing for LLMs

Re:DeepEval – Unit Testing for LLMs by azzarcher on Thursday August 17, 2023 @02:53AM (#37158122)

How is this standing out from https://benchllm.com/?

[ Parent | on HN ]

Re:DeepEval – Unit Testing for LLMs by Eddygandr on Thursday August 17, 2023 @03:02AM (#37158180)

I really dislike benchllm's use of yamls for test cases - I'd rather it be in code.

""" input: What's 1+1? Be very terse, only numeric output expected: - 2 - 2.0 """

[ Parent | on HN ]

Re:DeepEval – Unit Testing for LLMs by jacky2wong on Thursday August 17, 2023 @04:46AM (#37158898)

Agreed. No one should ever have to touch YAML for writing unit tests for LLMs. Ever. Most people writing agents and LLM applications are Python developers/data scientists/ML engineers.

[ Parent | on HN ]