Re:Microsoft and OpenAI Working on ChatGPT-Powered Bing in C...
by arpa on Wednesday January 04, 2023 @07:16AM
(#34244663)
something something sufficiently advanced markov chains something something GAI
Re:Microsoft and OpenAI Working on ChatGPT-Powered Bing in C...
by eunos on Wednesday January 04, 2023 @08:05AM
(#34245053)
Explainable AI specifically Language Models will be a very interesting field to follow then.
Re:Microsoft and OpenAI Working on ChatGPT-Powered Bing in C...
by astrange on Thursday January 05, 2023 @05:17AM
(#34257987)
> ChatGPT models are language models; they represent closeness between text utterances. It works by looking for the chains of words most similar or usually connected to those indicated in the prompt, with no understanding of what those words mean.
No, it does not work that way. That’s how base GPT3 works. ChatGPT works via RLHF and so we don’t “know” how it decides to answer queries. That’s kind of the problem.
ChatGPT models are language models; they represent closeness between text utterances. It works by looking for the chains of words most similar or usually connected to those indicated in the prompt, with no understanding of what those words mean.
As a metaphor, think of an intern who every morning is asked to buy all the newspapers in paper form, cut out the news sentence by sentence, and put all the pieces of paper in piles grouped according to the words they contain.
Then, the director requests to write a news item on the increase in interest rates. The intern goes to the pile where all the snippets about interest rates are placed, will randomly get a bunch of them, and write a piece by linking the fragments together.
The intern has a PhD in English, so it is easy for them to adjust the wording to ensure consistency; and the topics more talked about will appear more often in the snippets, so the ones chosen are more likely to deal with popular issues. Yet the ideas expressed are a collection of concepts that might have made sense in their original context, but have been decontextualized and put together pell-mell, so there's no guarantee that they're saying anything useful.