> This is just setting the bar far too high... It's far better than random chance

I don't think it is setting the bar too high, "far better than random" can still be abysmal.

You're right that humans are better than random, but there is a wide range of emotions. Assume for the sake of argument there are 10 emotions to identify and evenly distributed in frequency, then random is getting it right 10% of the time. Let's say humans get it right 30% of the time from still photographs of real-life situations (not staged) -- I'm making this specific number up but depending on the type of experiment, it's a reasonable ballpark one.

That's still horribly low, wrong more than half the time. I can't think of any situation where that could warrant any kind of responsible decision-making. If we were talking 95% accuracy then there could be value, I'm not asking for 99.99% accuracy. But if reliable detection is something like 30%, I stand by calling this "snake oil" even if it's not just random.

I saw a recent example of true snake oil on an HBO series called "The Knick" about the Knickerbocker Hospital in NYC 1900s. In that plot line a company had put together a potion that didn't harm you, but didn't do anything it claimed to. They approached the famous surgeon played by Clive Owen and ask him to front the product, just slap his name on it, boom instant hit.

Well Dr. John W. Thackery had too much pride to be associated with that and refused the money.

Cut to modern day version, I'm a computer programmer and I write code to correctly place humans from video feeds into various buckets based on the RGB colors I scan from bitmap image 2d arrays... and I say it places them into buckets like "possible upset" and "very much in flow state" or "bored, brain not engaged." And then sell this program/SAAS product to companies and they get value from these buckets... how is that still true snake oil?

>I can't think of any situation where that could warrant any kind of responsible decision-making. Assuming the errors are random, 30% is still accurate enough to get a signal in an aggregate data set.

But the errors won't be random at all.

E.g. take class A of students from one culture and class B of students from another culture, and assume the students all feel the same level of moderate anger, but the two cultures have different "anger display rules" as they're called.

The detector might judge all the students from one class as angry, but none from the other. Aggregating students into classes doesn't fix anything.

Similarly, if you took some students who have a certain expression when they're happy, and other students who have the same expression when they're afraid, and mixed them together in the same class, then whatever emotion the detector is interpreting the expression to mean is going to be triggered by completely unrelated things, and the signal won't be meaningful at all.

You've taken the idea that there are cultural differences in emotional display (which is true) and exaggerated it to an extent beyond any evidential support.

Yes, there are differences - especially in identification of intensity. But in general detection is pretty reliable across cultures.

Eg:

The results show that the overall accuracy of emotional expression recognition by Indian participants was high and very similar to the ratings from Dutch participants. However, there were significant cross-cultural differences in classification of emotion categories and their corresponding parameters. Indians rated certain expressions comparatively more genuine, higher in valence, and less intense in comparison to original Radboud ratings. The misclassifications/ confusion for specific emotional categories differed across the two cultures indicating subtle but significant differences between the cultures.

https://journals.plos.org/plosone/article?id=10.1371/journal...

> But in general detection is pretty reliable across cultures.

Unfortunately that's simply not true. Your example merely shows reliabiliy between two countries, not all countries. You can research, for example, how Japanese people tend to smile when angry, sad, or embarrassed, in total contrast to Westerners.

This is one of the classic "extreme" examples, but it demonstrates how my point holds -- errors are not random but are highly correlated with the culture -- not to mention the subculture, the individual (the person who hides all emotion when angry), etc.

If you agree that AI can reach human levels of accuracy is your point that we shouldn't ever try to assess someone's emotion in decision making, whether it's a person or machine deciding?

> is your point that we shouldn't ever try to assess someone's emotion in decision making, whether it's a person or machine deciding?

isn't that a pretty sane and basic assumption ?

No, I think I'll still assess the hitchhikers I pick up.

Seriously though, especially with individuals, why shouldn't you use all the information available, if it produces results even an iota better than blind acceptance (and for most people, their empathic-sense does)?

In the context of labeling emotions based on context-free still images of strangers, then yes humans are so inaccurate at this it would generally be a bad idea to assess emotions.

In the context of real life, where we know people, are interacting with them, and are reading cues from their whole body and their movement over time, then we're much better at assessing emotions and we'd be pretty foolish to ignore emotional signals.

I don't think there's a big industry of still image emotion detection. It's not the smartest approach. I think it would be necessary to use video for this task.

Also, it's conceivable for a machine learning algorithm to surpass humans in certain aspects, e.g.

- Identifying emotions of humans who grew up in a different culture, for which the ML algorithm may have piles of data from every part of the world and you don't

- Identifying emotions of babies based on instinctual behavior

- Identifying emotions of people who are undergoing a trauma response that most humans are not trained to recognize

- Identifying emotions of people during a negotiation process in which the people are actively trying to hide their emotions from humans but nevertheless leak certain signs of their true emotions