Re:Show HN: London Feels – Sentiment-analysis of Londoners T...
by cl42 on Monday October 27, 2014 @04:14PM
(#8517907)
What sort of accuracy did you have in your sentiment analysis algorithms? I'm curious because I find the error rate in such algorithms is typically higher than any sort of variance you are seeking, which causes significant problems in terms of any sort of pattern recognition.
When you're using things as short as Tweets, and as broad as "general sentiment", you're probably making accuracy even worse, to the point that simpler demographic analysis or bag-of-words clustering (i.e., cluster areas by diction rather than by sentiment) yields more reliable results, even for sentiment.
Geographical analysis tools should be used in these types of analyses, apart from just looking at blobs on a map. I used k-means based cluster analysis to find groups of happy and sad areas but again the groups turned out to be nothing conclusive.
The web GIS company I ended up working for used sentiment analysis of tweets by aggregated them into regions, so as to find positive and negative areas during a specific timeframe (for example, US elections). The regions had demographics which could be used statistically, and in general some interesting patterns were observed.