Pattern uses a whitelist heuristic approach that's got poor precision and is no longer state of the art.
Not that there's anything wrong with this code, but anyone who is looking to use sentiment analysis in their work should consider researching more up-to-date alternatives which don't necessary have Pythonic alternatives just yet.
Re:TextBlob – Finding Sentiments in Text
by shekhargulati on Wednesday November 06, 2013 @03:30PM
(#6685290)
Sentiment analysis is a hard thing to get correct. I was not aware that Pattern is no longer state of the art library. The tests that I performed showed promising results. I will try to get deeper into other alternatives.
Re:TextBlob – Finding Sentiments in Text
by rmrfrmrf on Wednesday November 06, 2013 @05:46PM
(#6686077)
It's not state of the art, but I just ran this guy's code with about 100 samples and it was spot on. It should be noted that the "state of the art" in sentiment analysis is actually pretty far advanced. For basic polarity testing, this works fine.
Re:TextBlob – Finding Sentiments in Text
by rmrfrmrf on Thursday November 07, 2013 @11:43AM
(#6690395)
As a library, I'd have no problem selling a service that used this, with the idea that for an MVP we could launch and then switch out with a more robust library.
Re:TextBlob – Finding Sentiments in Text
by primaryobjects on Wednesday November 06, 2013 @05:47PM
(#6686086)
An API like that is good for an introduction, but I think you'll get better results with a machine learning approach (my pet project http://www.sentimentview.com). When I was running tests with a baseline algorithm (just matching against positive and negative keywords), I saw results from 62% accuracy with the baseline to 80% with an SVM http://blog.sentimentview.com/post/59031004797/learning-curv....
Re:TextBlob – Finding Sentiments in Text
by nl on Wednesday November 06, 2013 @06:23PM
(#6686313)
I'm looking at your examples[1], and I'm seeing a lot of analysis that seems pretty wrong to me.
To be fair, I haven't counted and done the maths (and I understand that human sentiment analysis often doesn't agree with other humans), but.. well, look at these examples:
Positive: Thank you Urbana voter for pointing out what we knew. Cheat lie steal the election. We will NEVER become an Islam nation as Obama wants.
Positive: Fun fact: @kbzeese got 1.5% of the vote in 2006 US Senate General Election, but thinks he knows what the people want #headdesk
Negative: What is our present condition? We have just carried an election on principles fairly stated to the people.
Does your model give confidence? It looks to me like you are making it bi-modal, but if you had a "neither positive nor negative" category it might fix some of the issues?
Eg, this is rates as positive, but I'd rate it as neither positive nor negative: "Should ballot papers in Northern Ireland include photographs of the candidates standing? Make your views known. http://t.co/NySCHsmecE"
Of course tweets are a pretty difficult thing to run sentiment analysis on: those random hashtags break a lot of machine learning models, whilst a human can read them and realise something like "#headdesk" is probably bad (although in that case the phrase "thinks he knows" is something that a model could probably use if it understood n-grams)
Re:TextBlob – Finding Sentiments in Text
by primaryobjects on Thursday November 07, 2013 @02:34PM
(#6691852)
Yes, all very good points. I didn't use a neutral category, as it's difficult to gauge whether a tweet should be neutral. I've thought of detecting them based upon news-speakish tweets, as you've pointed out. That would require its own machine learning run, just to sort out neutral from containing-sentiment.
Also keep in mind, different topics work better. The term "election" has a lot of news headlines, which many are probably neutral, skewing the results. More consumer-ish topics yield better results. But yes, tweets are difficult to analyze. I've done another recent experiment with tweet analysis, if you like this kind of stuff http://primaryobjects.com/CMS/Article158.aspx
Pattern uses a whitelist heuristic approach that's got poor precision and is no longer state of the art.
Not that there's anything wrong with this code, but anyone who is looking to use sentiment analysis in their work should consider researching more up-to-date alternatives which don't necessary have Pythonic alternatives just yet.
That's basically my measure for real world vs academic.
"This isn't a bad idea"
Is flagged red
just curious, as an onlooker. :)
To be fair, I haven't counted and done the maths (and I understand that human sentiment analysis often doesn't agree with other humans), but.. well, look at these examples:
Positive: Thank you Urbana voter for pointing out what we knew. Cheat lie steal the election. We will NEVER become an Islam nation as Obama wants.
Positive: Fun fact: @kbzeese got 1.5% of the vote in 2006 US Senate General Election, but thinks he knows what the people want #headdesk
Negative: What is our present condition? We have just carried an election on principles fairly stated to the people.
Does your model give confidence? It looks to me like you are making it bi-modal, but if you had a "neither positive nor negative" category it might fix some of the issues?
Eg, this is rates as positive, but I'd rate it as neither positive nor negative: "Should ballot papers in Northern Ireland include photographs of the candidates standing? Make your views known. http://t.co/NySCHsmecE"
Of course tweets are a pretty difficult thing to run sentiment analysis on: those random hashtags break a lot of machine learning models, whilst a human can read them and realise something like "#headdesk" is probably bad (although in that case the phrase "thinks he knows" is something that a model could probably use if it understood n-grams)
[1] http://www.sentimentview.com/#examples
Also keep in mind, different topics work better. The term "election" has a lot of news headlines, which many are probably neutral, skewing the results. More consumer-ish topics yield better results. But yes, tweets are difficult to analyze. I've done another recent experiment with tweet analysis, if you like this kind of stuff http://primaryobjects.com/CMS/Article158.aspx