I see what you mean. The way I approach this is that I think of it as a very (very, very) elaborate auto-complete. Something that takes some text, and continues with the most likely continuation. Which is what it basically is, right?

But that means that whatever rather impressive internal representation of knowledge it has encoded, that knowledge itself can’t be trusted on a “factual” basis.

This is where you have stopped, but you shouldn’t. Because you’re missing what it’s great at. And it’s great at “continuing” text in a longer context. So just use it with a longer context. Use the Search functionality in ChatGPT when asking it things such that you see where it got it from. Use it in a codebase context where you point it to the relevant parts. Give it examples it should follow, etc.

For every task I have I try using AI in some way and form first. Because the downside is limited - I waste 5-7 minutes figuring out that it’s not going to work, and this is something I definitely need to do myself. But the upside is unlimited - the task is done in minutes.