As an example: Stripe has sandboxes, I wanted to know how to create them via the CLI but found nothing in the docs. Asked the AI and it was like "Just do stripe sandboxes create <name>". The command does not exist and the whole functionality is not planned to be added. If another AI now reads this it could interpret it as answered successfully?
Just thinking - but I like the idea of the tool. Would love to use it but currently don't have any public docs to test with. Good luck with your project :)
--> I would say so. Not quite sure what Stripe is using on their docs, but sounds like its been given free reign and not grounded in actual docs.
Easiest way is to weed out hallucinations, hasnt been a problem for me at all to be honest.
P.S. For anyone reading, happy to refund all answers that you deem shouldnt be billed.
Thanks for trying it!
Something fair for customers, thats the goal. But right now my priority is seeing if anyone gives a f** about this to begin with.
Regarding hallucinations: It's not 100% solvable, but can be done pretty well in my opinion with prompting (aka only answer based on facts you see in the documentation)
P.S. if you have docs and want a more "live demo" you can sign up and just put in the docs url and you'll have a live demo without installing the script onto the docs site.