It may be a "thing", because generating BS is a viable business model and ChatGPT makes it more efficient.
..but I submit as a working hypothesis, that it is completely impossible to gain knowledge you do not already possess from a language model, no matter how clever your prompting.
I'm very interested in counter-examples, but I have seen a few that turn out to be fake already.
Is Art Director just a "BS" job? I don't get it.
I think I understand the sense in which you claim it produces relevant facts not in the prompt.
It's not that we differ on easily observable behavior of the system.
It's that I question if GPT-3 is "producing" these identifiable facts, and if the user is "producing" them instead, whether they can possibly be "relevant".
I'm not sure what you're trying to say. That GPT-3 is just vomiting stuff up out of its training set and not producing any new knowledge? But that's totally irrelevant to the issue of whether it can transmit knowledge to a user, who presumably hasn't memorized the entire training set.
Hmm. Seems obvious to me that it's producing new output, but that output isn't knowledge and it can't be.
Sometimes ChatGPT tells me something that turns out to be correct and relevant. And I get excited, and then I Google it and what it told me is the first hit on Stack Overflow.
There's a subtle point here, that other people might say "well, ChatGPT is ok, but no better than Google" or something like that. But I differ on that. The key is that I don't know it's Stack Overflow until I check independently. So it's giving it too much credit to say it's as good as Google, and the amount of information it can output is not lower bounded by its training set, but is actually zero due to being adjacent to an infinite amount of BS that by its nature always requires external mechanisms to separate out.
You might synthesize new knowledge.
When ChatGPT produces new output, it's not synthesizing new knowledge. It can't even output the knowledge it was trained with, as long as it lacks the ability to tag it in a trustworthy way.
It's not that it's always BS, it's that it's almost always BS and if you don't know the answer in advance or independently, you can't distinguish it from anything within the model.
I reached the same conclusion as yourself, but do see a totally different path to take regarding information propagation (how GPT works). For example, cells merge information monotonically. This is how neural networks balance too, but could be applied in new/undiscovered ways.
It doesn't know what works.
It not only spit out the model but also the casts/fillable attributes on the model, as well. It even helped me work through an idea, that I didn't know what it was called, I was thinking it was EAV but instead it's metaform/metafields, to basically create something like how wordpress has the ability to dynamically create content 'types', django/wagtail can do this to, w/ chatgpt I think I've nailed down how to do this using polymorphism with the least amount of headache.
I'm wanting to create a CRM/CMS/ERP solution that can be very 'moldable' to different use cases, and this looks to be a good use, either way just being able to discuss with the ai my 'options', was like a major brain dump and increased the power of my flow.
YMMV, but if you can't get it to work like this, doesn't mean it doesn't, just means it doesn't for you, and while I can save 2-3 hours for every hour previously worked, that's valuable to me, esp as a freelancer who charges per project, not hourly.
Not true. Emergent abilities is an active research area in LLMs [0]. They even have pretty graphs on the topic.
[0] https://ai.googleblog.com/2022/11/characterizing-emergent-ph...
Checking some of the facts it gives me against other sites it’s all correct, but better organized and more accessible. There’s your counter-example. This works for basically any well-documented process.
You've got a huge blind spot if you think prompt engineer isn't already a thing.