I spent this afternoon asking ChatGPT (3.5 not 4) to help me query AWS resources into a csv. It gave me a 90% correct answer but made up a native csv output option. When I told it that option didn't exist it got almost sassy insisting it was correct. Eventually it gave me a closer answer using json and jq after I prodded it.

I had a similar experience asking it to write an API client. It wrote something very plausible but just concocted an endpoint that looked real but didn't exist.