Re:A guidance language for controlling LLMs
by jaggederest on Tuesday May 16, 2023 @05:17PM
(#35967870)
I've used Cucumber on a few consulting projects I've done and had management / C-level interested and involved. It's a pretty narrow niche, but they were definitely enthusiastic for the idea that we had a defined list of features that we could print out (!!) as green or red for the current release.
They had some previous negative experiences with uncertainty about what "was working" in releases, and a pretty slapdash process before I came on board, so it was an important trust building tool.
Re:A guidance language for controlling LLMs
by jamiek88 on Tuesday May 16, 2023 @06:16PM
(#35968503)
> important trust building tool
This is so often completely missed in these conversations about these tools.
Great point.
“Incentivize developers to write externally understandable release notes” is an underrated feature of behavioral testing frameworks!
Re:A guidance language for controlling LLMs
by hitchstory on Tuesday May 16, 2023 @07:40PM
(#35969300)
Ive had product owners take an interest in docs autogenerated from tests. Stuff like this:
https://github.com/hitchdev/hitchstory/blob/master/examples/...
Gherkin isnt really a suitable language for writing test cases in though - it's verbose, lacks inheritance, has clunky syntax, is stringly typed and the parser demands you write regexes to parse it.
Ha this reminds me of the craze for BDD/Cucumber type testing. Don’t think I ever once saw a product owner take interest in a human readable test case haha