Re:Ask HN: What metrics do you track when building agents?
by sselrats on Monday July 06, 2026 @08:13AM
(#48803564)
I think cost is lagging indicator. (it truely does)
I am currently tracking tool count exposed, schema-token size, tool-call failure rate, duplicate tool calls, retries, wall time, tokens before first useful external observation, and “answered without required tool evidence", and some more.
I strongly recommend single metric: task success with evidence, segmented by task type.
I am currently tracking tool count exposed, schema-token size, tool-call failure rate, duplicate tool calls, retries, wall time, tokens before first useful external observation, and “answered without required tool evidence", and some more.
I strongly recommend single metric: task success with evidence, segmented by task type.