For simple things it's pretty safe. I tried pasting in HTML from the homepage of Hacker News and having it turn that into a list of JSON objects each with the title, submitter, number of upvotes and number of comments.
Here's a similar trick I did with Copilot: https://til.simonwillison.net/gpt3/reformatting-text-with-co...
I haven't done exactly that, but based on similar examples this is likely very vulnerable to hallucinations.