News for hackers. Stuff that matters.
Re:All of ChatGPT's System Prompts by malux85 on Saturday October 14, 2023 @08:11AM (#37879867)
I theorise that since ChatGPT was trained on the internet, lots of its training data would include Q&A forums like Stack Overflow.

Perhaps it has learned by observation that friendly questions get helpful answers

Re:All of ChatGPT's System Prompts by corobo on Saturday October 14, 2023 @09:56AM (#37880503)
This also explains why it makes stuff up and confidently gives it as an answer instead of admitting when it doesn't know
Re:All of ChatGPT's System Prompts by malux85 on Saturday October 14, 2023 @03:49PM (#37883685)
I’m not sure it has the self reflection capability to understand the difference between knowing and not knowing, but I would love some evidence to show this.

The only thing I can think of is that it appears to be capable of symbolic manipulation - and using this can produce output that is correct, novel (in the sense that it’s not a direct copy of any training data) and compositional at some level of abstraction, so given this, I guess it should be able to tell if it’s internal knowledge on a topic is “strong” (what is truth? Is it knowledge graph overlap?) and therefore tell when it doesn’t know, or only weakly knows something? I’m really not sure how to test this

Re:All of ChatGPT's System Prompts by tough on Saturday October 14, 2023 @09:00PM (#37885800)
I tried asking ChatGPT about e/acc (accelerationism) moniker some twitter users sport nowadays. Not in training data. clueless
Re:All of ChatGPT's System Prompts by malux85 on Saturday October 14, 2023 @09:06PM (#37885840)
Of course it is, that’s domain knowledge. Its not magical, how would it know about things that it’s never been exposed to?!

Novel compositions of existing knowledge is totally different to novel sensory input.

Re:All of ChatGPT's System Prompts by tough on Sunday October 15, 2023 @05:45PM (#37893671)
Well I had no idea when the moniker was started being used so I wouldn' t know if it was on the cut off knowledge date or not
Re:All of ChatGPT's System Prompts by corobo on Sunday October 15, 2023 @04:37PM (#37893188)
I was more using "doesn't know" in the sense of has no evidence or training material suggesting the thing it said is true. I'm not associating actual brain functions to the current generation of AI.
Re:All of ChatGPT's System Prompts by Szpadel on Saturday October 14, 2023 @04:53PM (#37884169)
> Perhaps it has learned by observation that friendly questions get helpful answers

It tries to predict next words, and this is it's only goal, answering your question is like controlled side effect

Re:All of ChatGPT's System Prompts by og_kalu on Saturday October 14, 2023 @07:00PM (#37885050)
Side effect or not, Stuff like this works

https://arxiv.org/abs/2307.11760

Re:All of ChatGPT's System Prompts by malux85 on Saturday October 14, 2023 @07:22PM (#37885199)
Predicting the set of words that constitutes a helpful response when given a friendly question is still valid in the world of stochastic parrots.

Reducing it's actions to "just predicting the next word" does a disservice to what it's actually doing, and only proves you can operate at the wrong abstraction. It's like saying "human beings are just a bunch of molecular chemistry, and that is it" or "computers and the internet are just a bunch of transistors doing boolean logic" (Peterson calls this "abstracting to meaninglessness"), while technically true, it does a disservice to all of the emergent complex behaviour that's happening way up the abstraction layer.

ChatGPT is not just parroting the next words from it's training data, it is capable of producing novel output by doing abstraction laddering AND abstraction manipulation. The fact that it is producing novel output this way is proving some degree of compositional thinking - again, this doesn't eliminate the stochastic parrot only-predicting-the-next-word explanation, but the key is in the terminology .. it's a STOCHASTIC parrot, not a overfit neural network that cannot generalize beyond it's training data (proved by the generation of compositional novel output).

Yes, it is only predicting the next word, and you are only a bunch of molecules, picking the wrong abstraction level is meaningless

Re:All of ChatGPT's System Prompts by Szpadel on Sunday October 15, 2023 @03:48AM (#37887807)
all true, but those models are not thinking and slightly different prompt leads to dramatically different results quality.

it is true that those models can have amazing results, but they try to give most realistic answer and not correct or helpful one.

Because of fine tuning we very often get correct answers and sometimes we might forget that it isn't really what model is trying to do

To give you life analogy: you might think that some consultant is really trying to help you where it's just someone trying to earn money for living and helping you is just a way he can achieve that. In most cases result might be the same but someone eg. bribe him and results might be surprising