It's also a solid rubber duck, it'll test you by giving you misinfo and you have to tease it back from the brink.
Yeah, a coworker tried an exercise like this, and it was basically (even with as-an-expert prompts) like trying to mentor a high school intern that's trolling you (except that it won't actually get better at anything in the process.)
There was a recent article on Slashdot titled something how metaverse may increase the gdp by like 2 or 3 percent. It was obviously laughed at. It's actually LLMs that will do this, and easily.