News for hackers. Stuff that matters.
Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by PaulDavisThe1st on Thursday March 23, 2023 @08:13PM (#35283363)
> "we did not test for the ability to understand sarcasm"

I find it hard to see how detecting and eliminating sarcasm requires a theory of mind. It requires some association between various stylistic elements and the concept of sarcasm.

The same is true of irony.

I still wonder how many of these people have read Dennett's "The Intentional Stance", which holds that the best way to think about "intention" is as an explanatory model, not a mechanism. That is, we can say that the dog "behaves as if it has the intention to get inside" without making any claim about the internal state of the dog.

Dennett further speculates that our own self-experience of intention is a matter of turning the same explanatory model upon our own behavior, but that's an extension that isn't directly relevant to this speculation about language models.

Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by knaik94 on Thursday March 23, 2023 @09:31PM (#35283934)
The authors go into more detail about their reasoning and the nuances between mechanistic and functional explainability. The authors only said related not required. No clear reason was given for why the commented sections stayed commented out.
Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by nextaccountic on Friday March 24, 2023 @01:45AM (#35285473)
> I still wonder how many of these people have read Dennett's "The Intentional Stance", which holds that the best way to think about "intention" is as an explanatory model, not a mechanism. That is, we can say that the dog "behaves as if it has the intention to get inside" without making any claim about the internal state of the dog.

That's also a way to frame evolution and literally all of biology.

Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by paganel on Friday March 24, 2023 @04:55AM (#35286647)
> It requires some association between various stylistic elements and the concept of sarcasm.

That sounds very mechanic-ist, including Dennett's theory (I haven't read Dennett, to be exact, I'm going by your explanation). Which I guess it's par for the course when talking about a scaled up Mechanical Turk concoction like this GPT thing is. It won't "create" any Radio Erevan jokes anytime soon, that's for sure, though.

Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by PeterisP on Friday March 24, 2023 @06:07AM (#35287092)
> I find it hard to see how detecting and eliminating sarcasm requires a theory of mind. It requires some association between various stylistic elements and the concept of sarcasm.

I am quite certain that detecting sarcasm can't be done based on stylistic elements" and requires some (even if implicit) estimation of what the author is thinking that contrasts with what is being said.

E.g. a relevant example I have actually seen in doing sentiment analysis on tweets to evaluate how customers perceive a company is "#CompanyName Got my order delivered in just under three hours. Thank you for great service! thumbsup-emoji" - now is this a positive review or sarcasm? And the thing is, you can't tell from the message by itself, you need an understanding of the customers' expectation (guess you might call it "a theory of mind") that for a pizza chain this is obviously sarcasm, but for a web store that sells some electronics the same thing would actually mean a fast delivery and great service. IMHO sarcasm detection is mostly about 'world knowledge' about what the implied expectations are, and not about stylistic elements at all.

Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by megablast on Friday March 24, 2023 @06:37AM (#35287290)
You are assuming it is 100% perfect at detecting sarcasm at all times. There is no reason for this to be true.
Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by PaulDavisThe1st on Friday March 24, 2023 @11:20AM (#35290213)
Disagree.

If you have determined that a common element in reviews is a reference to delivery of goods or services within a given timeframe, and can identify that most of the quantitative descriptions of the timeframe are relatively short ... then coming across one that uses a much larger timeframe but is still positive will be quite noticeable.

This is actually typical of the problem with far, far too many people's interpretation of what language models are doing. The process I've described requires only a representation of language behavior. The LM can be said to understand how people talk (write) about a thing, but there is no knowledge of anything beyond language behavior.

Re:Microsoft's paper on OpenAI's GPT-4 had hidden information by d0mine on Friday March 24, 2023 @03:16PM (#35294098)
if it quacks like a dog, it is a dog.

Though, with LLMs, you might need quite a few (often more than 5) exchanges to find out that it can't actually quack.