Ask HN: Sentiment Analysis – how to handle biased word list lengths?

Re:Ask HN: Sentiment Analysis – how to handle biased word li... by gibrown on Friday February 20, 2015 @05:40PM (#9083570)

Expanding on the point that bag of words doesn't work that great for sentiment... A good example of why are phrases like "not bad", "not great", "not a good idea". Just scoring based on unigrams really doesn't capture the context well, and people use negators a lot. You could maybe try filtering these out or detecting them with some clever rules.

I did a bit of work on this using the JDPA Sentiment Corpus for my thesis about 5 years ago. Its hand annotated for things like negators and inversions of sentiment. There's a bunch of code and examples here: https://verbs.colorado.edu/jdpacorpus/

Warning: code/corpus is academic licensed, but even reading the papers may give you some ideas.

[ Parent | on HN ]

Re:Ask HN: Sentiment Analysis – how to handle biased word li... by PaulHoule on Saturday February 21, 2015 @09:36AM (#9085607)

It is worse than than because of:

(i) negators aren't always next to the word they negate, to get good accuracy you need more of a parse (ii) sentiment is highly dependent on the domain. For instance, if one was looking at people's opinions on stocks, "Buy" and "Sell" are considered sentiment but these are emotionally neutral words in general. (iii) Also there is sarcasm, which sometimes people can't figure out right.

[ Parent | on HN ]