I ran into the same problem of "sounds true" when testing the limits of GPT-3 as a general purpose "knowledge engine", used to answer questions about the real world. Either it doesn't understand the difference between truth and fiction, or it doesn't care. The output is 95% truthful and 5% outright fabrication. Even worse, it writes better than most humans, so the fabrications often come out sounding extremely convincing.

I once had it generate an entire interview with an author, which was so realistic I was sure it had encountered it verbatim in the training data. The interview was about one of his books. Turns out such a book didn't even exist, but GPT-3 knew real facts like the name his publisher, the names of employees there etc. and wove them into the story.

The best use I've found for GPT-3 is text summarization, it seems to do very well on that front. I think OpenAI are working on a hyperlinked interface that lets you jump to the original source for each fact in the summary.