Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on /story/46993377.
Because you stop measuring.
Expert intuition works on 10 examples. It doesn’t generalize to 10,000 inputs and three interacting failure modes.
When you optimize by feel:
-results aren’t reproducible
-changes aren’t versioned
-trade-offs aren’t quantified
-regressions slip in silently
This isn’t a prompting problem. It’s an optimization problem.
Treat prompts like hyperparameters.
Dataset → Evaluator → Optimizer → Ranked prompts.
Once you introduce an objective function, intuition becomes optional.
We wrote a cookbook that lays out the full workflow step by step for teams moving beyond manual iteration.
There are no comments on this comment yet.
Because you stop measuring.
Expert intuition works on 10 examples. It doesn’t generalize to 10,000 inputs and three interacting failure modes.
When you optimize by feel:
-results aren’t reproducible
-changes aren’t versioned
-trade-offs aren’t quantified
-regressions slip in silently
This isn’t a prompting problem. It’s an optimization problem.
Treat prompts like hyperparameters.
Dataset → Evaluator → Optimizer → Ranked prompts.
Once you introduce an objective function, intuition becomes optional.
We wrote a cookbook that lays out the full workflow step by step for teams moving beyond manual iteration.