Well, you should definitely be able to read the output, agreed on that much.

But sometimes it's annoying to create such rules either due to having to click through multiple screens, or not remembering the exact syntax offhand.

Locally, something like iptables comes to mind. I've used it hundreds of times, but never regularly enough to remember how to write more complex rules without consulting the manual or Google each time.

The way I'd do it would be to have the LLM generate a rule in some fairly-readable rules engine language, and show the rule, examples, and counter-examples for the user to confirm.

This would be like writing email filters in gmail or smart playlists in iTunes.

Possibly, you could ask the LLM for the examples and counter-examples that look most likely to be a bug?

I think realistically, you can't use AI for anything where the output absolutely must be precise. AI is great for when close enough is good enough, but even just specifying "close enough" is hard.

If you need to know that the output is perfect then you need to check it, and if you need to check it then maybe it's better to just get an expert to do it in the first place.

>then maybe it's better to just get an expert to do it in the first place.

Yeah but also maybe an expert gets a shipload more done using an AI? Huge productivity increase? Dunno!

If you're an expert, it's much easier to do the thing rather than to carefully watch over the shoulder of a noob doing the thing.

Again, I'm talking about in situations where you need to be absolutely precise. This isn't for emails explaining simple concepts to a client (which is a good use) but rather a technical document explaining a process to another expert end user.