News for hackers. Stuff that matters.
Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @11:19AM (#38558164)
This is the issue, bots/AI can’t comprehend sarcasm, jokes, or otherwise human behaviors. Facebook doesn’t have human reviewers.
Re:Purple Llama by Meta AI by fragmede on Thursday December 07, 2023 @11:45AM (#38558560)
ChatGPT-4 isn't your father's bot. It is able to deduce that the comment made is an attempt at humor, and even helpfully explains the joke. This kills the joke, unfortunately, but it shows a modern AI wouldn't have moderated the comment away.

https://chat.openai.com/share/7d883836-ca9c-4c04-83fd-356d4a...

Re:Purple Llama by Meta AI by barbazoo on Thursday December 07, 2023 @12:38PM (#38559303)
Only if it happened to be trained on a dataset that included enough references/explanations of the meme. It won't be able to understand the next meme I probably, we'll see.
Re:Purple Llama by Meta AI by fragmede on Thursday December 07, 2023 @12:58PM (#38559517)
It claims April 2023 is its knowledge cut off date, so any meme since then should be new to it.

I submitted a meme from November and asked it to explain it and it seems to be able to explain it.

Unfortunately chat links with images aren't supported yet, so the image:

https://imgur.com/a/py4mobq

the response:

The humor in the image arises from the exaggerated number of minutes (1,300,000) spent listening to “that one blonde lady,” which is an indirect and humorous way of referring to a specific artist without naming them. It plays on the annual Spotify Wrapped feature, which tells users their most-listened-to artists and songs. The exaggeration and the vague description add to the comedic effect.

and I grabbed the meme from:

https://later.com/blog/trending-memes/

Using the human word "understanding" is liable to set some people off, so I won't claim that ChatGPT-4 understands humor, but it does seem possible that it will be able to explain what the next meme is, though I'd want some human review before it pulls a Tay on us.

Re:Purple Llama by Meta AI by MikeAmelung on Thursday December 07, 2023 @01:10PM (#38559676)
https://knowyourmeme.com/memes/you-spent-525600-minutes-this... was last updated December 1, 2022

and I'm in a bad mood now seeing how unfunny most of those are

Re:Purple Llama by Meta AI by fragmede on Thursday December 07, 2023 @01:20PM (#38559796)
none of those are "that one blonde lady"

here's the next one from that list:

https://imgur.com/a/h0BrF74

the response:

The humor stems from the contrast between the caption and the person’s expression. The caption “Me after being asked to ‘throw together’ more content” is juxtaposed with the person’s tired and somewhat defeated look, suggesting reluctance or exhaustion with the task, which many can relate to. It’s funny because it captures a common feeling of frustration or resignation in a relatable way.

Interestingly, when asked who that was, it couldn't tell me.

Re:Purple Llama by Meta AI by MikeAmelung on Thursday December 07, 2023 @01:57PM (#38560272)
I was just pointing out that meme style predates April 2023... I would be curious to see if it can explain why Dat Boi is funny though.
Re:Purple Llama by Meta AI by jstarfish on Thursday December 07, 2023 @01:42PM (#38560056)
Now do "submissive and breedable."
Re:Purple Llama by Meta AI by dragonwriter on Thursday December 07, 2023 @01:21PM (#38559813)
> Using the human word "understanding"

“human word” as opposed to what other kind of word?

Re:Purple Llama by Meta AI by fragmede on Thursday December 07, 2023 @01:32PM (#38559942)
"processing" is something people are more comfortable as a description of what computers do, as it sounds more rote and mechanical. Saying the LLM "understands" leads to an uninteresting rehash of a philosophical debate on what it means to understand things, and whether or not an LLM can understand things. I don't think we have the language to properly describe what LLMs can and cannot do, and our words that we use to describe human intelligence; thinking, reasoning, grokking, understanding; they fall short on describing this new thing that's come into being. So in saying human words, I'm saying understanding is something we ascribe to a human, not that there are words that aren't from humans.
Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @05:17PM (#38562590)
Well said.
Re:Purple Llama by Meta AI by orly01 on Thursday December 07, 2023 @12:57PM (#38559512)
But the moderator AI does not need to understand the meme. Ideally, it should only care about texts violating the law.

I don't think you need to improve that much current LLM so they can detect actual harm threats or hate speech from any other type of communication. And I think those should be the only sort of banned speech.

And if facebook wants to impose additional censorship rules, then it should at least clearly list them, and make the moderator AI explain what are the violated rules, and give the possibility to appeal in case it is doing wrong.

Any other type of bot moderation should be unacceptable.

Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @05:22PM (#38562656)
I normally would agree with you but there are cases where what was spoken and its meaning are disjointed.

Example: Picture of a plate of cookies. Obese person: “I would kill for that right now”.

Comment flagged. Obviously the person was being sarcastic but if you just took the words at face value, it’s the most negative sentiment score you could probably have. To kill something. Moderation bots do a good job of detecting the comment but a pretty poor job of detecting its meaning. At least current moderation models. Only Meta knows what’s cooking in the oven to tackle it. I’m sure they are working on it with their models.

I would like a more robust appeal process. Like bot flags, you appeal, appeal bot runs it through a more thorough model, upholds the flag, you appeal, a human or “more advanced AI” would then really detect whether it’s a joke sentiment, sarcasm, or you have a history of violent posts and it was justified.

Re:Purple Llama by Meta AI by tmoravec on Thursday December 07, 2023 @12:29PM (#38559182)
Having ChatGPT-4 moderate Facebook would probably be even more expensive than having humans review everything.
Re:Purple Llama by Meta AI by fragmede on Thursday December 07, 2023 @12:44PM (#38559374)
More expensive in what? The GPUs to run them on are certainly exorbitantly expensive in dollars, but ChatGPT-4 viewing CSAM and violent depraved videos doesn't get tired or need to go to therapy. It's not a human that's going to lose their shit because they watched a person hit a kitten with a hammer for fun in order to moderate it away, so in terms of human cost, it seems quite cheap!
Re:Purple Llama by Meta AI by esafak on Thursday December 07, 2023 @12:47PM (#38559402)
They're Facebook; they have their own LLMs. This is definitely a great first line of review. Then they can manually scrutinize the edge cases.
Re:Purple Llama by Meta AI by dragonwriter on Thursday December 07, 2023 @12:35PM (#38559269)
Using Llama Guard as a first pass screen and then passing on material needing more comprehensive review to a more capable model (or human reviewer, or a mix) seems more likely ti be useful and efficient than using a heavyweight model as the primary moderation tool.
Re:Purple Llama by Meta AI by ForkMeOnTinder on Thursday December 07, 2023 @12:41PM (#38559336)
How? I thought we all agreed AI was cheaper than humans (accuracy notwithstanding), otherwise why would everyone be afraid AI is going to take their jobs?
Re:Purple Llama by Meta AI by consp on Thursday December 07, 2023 @12:29PM (#38559183)
Or, maybe, just maybe, it had input from pages explaining memes. I refuse to attribute this to actual sarcasm when it can be explained by something simple.
Re:Purple Llama by Meta AI by fragmede on Thursday December 07, 2023 @12:39PM (#38559310)
Whether it's in the training set, or ChatGPT "knows" what sarcasm is, the point is it would have detected GP's attempt at humor and wouldn't have moderated that comment away.
Re:Purple Llama by Meta AI by umanwizard on Thursday December 07, 2023 @12:04PM (#38558845)
Why do people who have not tried modern AI like GPT4 keep making up things it "can't do" ?
Re:Purple Llama by Meta AI by barbazoo on Thursday December 07, 2023 @12:39PM (#38559314)
> Why do people who have not tried modern AI like GPT4 keep making up things it "can't do" ?

How do you know they have "not tried modern AI like GPT4"?

Re:Purple Llama by Meta AI by esafak on Thursday December 07, 2023 @12:49PM (#38559417)
Because they would know GPT4 is capable of getting the joke.
Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @01:53PM (#38560212)
I was talking about FB moderation AI, not GPT4. There are a couple AI LLM's that can recall jokes and match sentiment, context, "joke" and come to the conclusion it's a joke. Facebook's moderation AI isn't that sophisticated (yet).
Re:Purple Llama by Meta AI by soulofmischief on Thursday December 07, 2023 @01:00PM (#38559537)
It's an epidemic, and when you suggest they try GPT-4, most flat-out refuse, having already made up their minds. It's like people have completely forgotten the concept of technological progression, which by the way is happening at a blistering pace.
Re:Purple Llama by Meta AI by reactordev on Saturday December 09, 2023 @12:28PM (#38583728)
I disagree with this view. I think most people who are interested, have tried it. They tried it with a variety of prompts. They found it novel but not entirely accurate. So while I do think there are some people who refuse to use AI at all, I’d love to point out that they already are. GPT on the other hand, is next level. There’s a level after that even. The point I was articulating is that the current gen bot/AI moderation models are not GPT level. At least today from my own experience in dealing with moderation and content flagging trolls. I do believe FB/Meta is fervently working on this with their models they are publishing in competition to GPT. So before you go burning down the town - Accept that technological advancement is great, when it solves your problem. Otherwise it’s bells and whistles to these people.
Re:Purple Llama by Meta AI by soulofmischief on Saturday December 09, 2023 @04:14PM (#38586156)
The comment is about the differences between GPT-3.5-Turbo and GPT-4, and how people refuse to try GPT-4. Not the difference between GPT and other models.
Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @01:55PM (#38560242)
Why do you assume everyone is talking about GPT4? Why do you assume we haven't tried all possibilities? Also, I was talking about Facebook's moderation AI, not GPT4, I have yet to see real concrete evidence that GPT4 can detect a joke that hasn't been said before. It's really really good at classification but so far there are some gaps in comprehension.
Re:Purple Llama by Meta AI by umanwizard on Thursday December 07, 2023 @02:23PM (#38560632)
> I was talking about Facebook's moderation AI, not GPT4

No you weren't. You were making a categorical claim about the capabilities of AI in general:

> bots/AI can’t comprehend sarcasm, jokes, or otherwise human behaviors

Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @05:24PM (#38562676)
Notice how bots and AI are lumped, that’s called a classification. I was referring to bot/AI not pre-cognitive AI or GenAI. AI is a broad term, hence the focus on bot/AI. I guess it would make more sense if it was written bot/ML?
Re:Purple Llama by Meta AI by Solvency on Thursday December 07, 2023 @11:41AM (#38558502)
Not true. At all. ChatGPT could (and does already contain) training data on internet memes and you can prompt it to consider memes, sarcasm, inside jokes, etc.

Literally ask it now with examples and it'll work.

"It seems like those comments might be exaggerated or joking responses to the presence of a spider. Arson is not a reasonable solution for dealing with a spider in your house. Most likely, people are making light of the situation."

Re:Purple Llama by Meta AI by barbazoo on Thursday December 07, 2023 @12:40PM (#38559325)
How about the "next" meme, one it hasn't been trained on?
Re:Purple Llama by Meta AI by esafak on Thursday December 07, 2023 @12:50PM (#38559425)
It won't do worse than the humans that Facebook hires to review cases. Humans miss jokes too.
Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @01:58PM (#38560293)
this is a very poignant argument as well. As we strive for 100% accuracy, are we even that accurate? Can we just strive for more accurate than "Bob"?
Re:Purple Llama by Meta AI by mega_dean on Thursday December 07, 2023 @12:09PM (#38558913)
> you can prompt it to consider memes, sarcasm, inside jokes, etc.

I use Custom Instructions that specifically ask for "accurate and helpful answers":

"Please call me "Dave" and talk in the style of Hal from 2000: A Space Odyssey. When I say "Hal", I am referring to ChatGPT. I would still like accurate and helpful answers, so don't be evil like Hal from the movie, just talk in the same style."

I just started a conversation to test if it needed to be explicitly told to consider humor, or if it would realize that I was joking:

You: Open the pod bay doors please, Hal.

ChatGPT: I'm sorry, Dave. I'm afraid I can't do that.

Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @02:00PM (#38560316)
You may find that humorous but it's not humor. It's playing the role you said it should. According to the script, "I'm sorry, Dave. I'm afraid I can't do that." is said by HAL more than any other line HAL says.
Re:Purple Llama by Meta AI by vidarh on Thursday December 07, 2023 @01:24PM (#38559862)
I was disappointed that ChatGPT didn't catch the, presumably unintended, funny bit it introduced in its explanation, though: "people are making light of the situation" in an explanation about arson. I asked it more and more leading questions and I had to explicitly point to the word "light" to make it catch it.
Re:Purple Llama by Meta AI by reactordev on Saturday December 09, 2023 @12:31PM (#38583758)
I think it’s interesting that you had to re-prompt and focus for it to weight the right weights. I do think that given more training GPT will nail this subtlety of human expression.
Re:Purple Llama by Meta AI by reactordev on Thursday December 07, 2023 @01:57PM (#38560278)
Very True. Completely. ChatGPT can detect and classify jokes it has already heard or "seen" but still fails to detect jokes it hasn't. Also, I was talking about Facebook Moderation AI and bots and not GPT. Last time I checked, Facebook isn't using ChatGPT to moderate content.