News for hackers. Stuff that matters.
Re:Are we in an AI Overhang? by ben_w on Monday July 27, 2020 @10:43AM (#23965523)
Is sentiment analysis really that good already?

Every time I’ve looked at the start of the art in sentiment analysis, it seems to be suffering from the same issue that bag-of-words has with modifiers like “not”. Or is that more a theoretical problem than a practical one?

I appreciate this is a rapidly moving field, so my knowledge could easily be out of date.

Re:Are we in an AI Overhang? by smallnamespace on Monday July 27, 2020 @11:11AM (#23965830)
Are modifiers an actual issue for many applications?

"This isn't a terrible horrible restaurant that nobody should ever go to" seems like 1) it doesn't mean it's actually a good restaurant either 2) the writer might be joking and sarcastic and 3) this will be very rare in actual reviews.

Put another way, certain modifiers contextually go with certain words and sentiments, so why shouldn't state of the art systems lean on that fact, notwithstanding the strict application of grammar?

Re:Are we in an AI Overhang? by ben_w on Monday July 27, 2020 @12:08PM (#23966427)
There was a story that a language lecturer had just explained how double-negatives were a sometimes a positive and sometimes an emphasised negative, and that likewise some languages used a double-positive to mean an negative. He claimed that English was not such a language, using double-positives only for emphasis, to which one of the students said “yeah yeah”.
Re:Are we in an AI Overhang? by thrownagain90 on Monday July 27, 2020 @12:56PM (#23966924)
An MIT linguistics professor was lecturing his class the other day. "In English," he said, "a double negative forms a positive. However, in some languages, such as Russian, a double negative remains a negative. But there isn't a single language, not one, in which a double positive can express a negative."

A voice from the back of the room piped up, "Yeah, right."

(https://www.ling.upenn.edu/~beatrice/humor/double-positive.h...)

Re:Are we in an AI Overhang? by gillesjacobs on Monday July 27, 2020 @01:12PM (#23967116)
To be fair, "yeah, right" is a sarcastic statement and linguistically the two words do not scope eachother so the positive statement is produced at the pragmatic level, double negatives are syntactico-semantic.
Re:Are we in an AI Overhang? by gillesjacobs on Monday July 27, 2020 @11:06AM (#23965794)
The issue you describe is typically called "Valence shifting" in this specific case "negation processing". It is of course a difficult problem to capture word-level sentiment and emotions but recent techniques in academic work obtain decent results.

However, industry typically relies on sentence- or document-level sentiment in, for instance, customer reviews with systems obtaining 80-90 F1-score which is very good. Often in e-commerce, aspect-based sentiment analysis is used in which a qualifying sentiment is attached to a target aspect, e.g. from a phone review systems extract: battery: large > positive; screen: dim > positive. You might have seen these types of reports in aggregate on review our e-commerce sites yourself.

It is however an ongoing field of research to process the scope of negation and uncertainty, but the field is making strides. State-of-the-art attention-based models obtain good scores on benchmark fine-grained sentiment analysis datasets such as the GoodFor/BadFor and MPQA2.0 of around 70% F1score [1]. This performance is nearly enough for commercial systems, depending on how you employ them.

1. https://link.springer.com/article/10.1186/s13673-019-0196-3