I got it to write a little story last night. My only creative contribution was the first two sentences and retrying the third paragraph a couple times to get it to commit to the surprising twist it made in the third sentence.
https://0bin.net/paste/obTsh6HjhH1yEN5a#n4pChlazivAujflU8hop...
It's not great, but .. in any case, you'll be convinced its almost certainly not a hoax when it generates responses like that for you in real time.
OpenAI could be massively misrepresenting the size or resource usage of the system and we couldn't tell... but I don't think they could be mechnically turking it.
GPT-2 could also do things like this... entirely locally. (I posted content here that people said they couldn't believe were machine written that were GPT-2). GPT-2 was just much less consistent and much more likely to go off the rails.
I do think GPT-3 can allow us to potentially learn about human ideas, just because it's a statistical model built from 200TB of text input written by humans. So even knowing there's no 'consciousness' there it would still be interesting to see how well it could be trained to answer questions...like a kind of statistical "database query" over that 200TB of human text.
And who would be able to tell if GPT-3 itself wasn't just internally doing massive plagiarism, just by lifting a huge block of text and then replacing one word with a different word. If it's just replacing words, then it's not "writing" any actual content, but basically GPT-3 is a very sophisticated cut-and-past plagiarizer engine.