News for hackers. Stuff that matters.
Re:Show HN: See if a movie is happy or sad with sentiment an... by etjossem on Friday May 05, 2017 @02:26PM (#14275662)
One of the major problems with sentiment analysis is that it doesn't pick up on irony or subtext very well. Relying on word choice alone, one such analysis pinned "Fitter Happier" as one of the happiest Radiohead songs.

http://rcharlie.com/2017-02-16-fitteR-happieR/

Edit: sorry, I misremembered - it actually handled "True Love Waits" pretty well. It did pick up "Fitter Happier" as one of the happiest, that's what struck me as strange.

Re:Show HN: See if a movie is happy or sad with sentiment an... by johnnydoe9 on Friday May 05, 2017 @02:55PM (#14275926)
That says True Love Waits is the most depressing song though.. am I missing something?

>We have a winner! “True Love Waits” is officially the single most depressing Radiohead song to date. Rightly so, given that it tied for lowest valence (0.0378) and ranked fourth for highest percentage of sad words (24%). If the numbers still don’t convince you, just listen to it.

Re:Show HN: See if a movie is happy or sad with sentiment an... by bberrry on Friday May 05, 2017 @02:33PM (#14275719)
Everyone realizes there outliers difficult to get right. Point is you can get quite far with a relatively naive approach.
Re:Show HN: See if a movie is happy or sad with sentiment an... by rullgrus on Friday May 05, 2017 @02:51PM (#14275890)
Isn't "True Love Waits" detected as the most depressing? The happiest one is "15 steps".
Re:Show HN: See if a movie is happy or sad with sentiment an... by btbuildem on Friday May 05, 2017 @01:28PM (#14275166)
Interesting project, horrible presentation.

The colour gradient on the graph is confusing - does green mean happy, blue unhappy, and the movie changes tone over time? Axes of the graph are not labeled, what are we looking at? But, those concerns are secondary.

Bar graph seems a poor choice here, given the nature of the data. Given that there doesn't seem to be any correlation with time, the order of phrases doesn't seem important -- you could forgo the linear presentation, and display the distribution of the data instead.

For example, you could bucket the sentiment values (-3 to -2, -2 to -1, etc) and use a histogram to show the counts in each bucket. This would enable you to compare different movies (one histogram per movie).

Re:Show HN: See if a movie is happy or sad with sentiment an... by eljs on Friday May 05, 2017 @01:58PM (#14275438)
You're right about the graphs. The horizontal axis is the minutes of the movie though (on the second graph). Will get rid of the gradient and add labels!
Re:Show HN: See if a movie is happy or sad with sentiment an... by fratlas on Friday May 05, 2017 @09:35PM (#14278178)
I love the visualization! Is the website a template? What lib for the graphs?
Re:Show HN: See if a movie is happy or sad with sentiment an... by vogt on Saturday May 06, 2017 @12:10AM (#14278618)
Just looks some Bootstrap and ChartJS.org.
Re:Show HN: See if a movie is happy or sad with sentiment an... by aisofteng on Friday May 05, 2017 @07:53PM (#14277715)
I'm always a little wary of the robustness of sentiment analysis; in my experience, if you take the time to check sentiment analysis results sentence by sentence, you will find a high error rate.

I haven't confirmed by looking at the source, but my suspicion is that either most sentiment analysis implementations are rule based or are not well tuned.

My go-to example is IBM Watson's sentiment analysis service rating "I hope you die" as very positive because the sentence is categorized as "hopeful". Which I suppose it is, technically, and perhaps this is a case where I'm expecting too much because recognizing this particular example as having a negative sentiment requires much human abstract reasoning and inference, but the example remains nonetheless because real-world language usage that isn't dry and technical is rife with these sorts of linguistic usages.

Re:Show HN: See if a movie is happy or sad with sentiment an... by eljs on Friday May 05, 2017 @09:03AM (#14273077)
OP here! I was watching the Stanford NLP classes a while back (https://www.youtube.com/playlist?list=PLiNErZ5Bus8qNxNsFZFkh...) and ended up trying the part about sentiment analysis on srt files. The way it works right is quite primitive but you can still see some "trends" on most movies. If there are any suggestions on how I could make this smarter I would love to hear them!
Re:Show HN: See if a movie is happy or sad with sentiment an... by msapaydin on Friday May 05, 2017 @01:13PM (#14275035)
Maybe you could use RNNs to do the sentiment analysis, either Recurrent or Recursive Neural Networks. Some sample code is available in the book "Tensorflow for machine intelligence" in github.
Re:Show HN: See if a movie is happy or sad with sentiment an... by sawmurai on Friday May 05, 2017 @10:51AM (#14273831)
Maybe if you get information about cut frequency you could identify parts that are more actionish / exciting than others.
Re:Show HN: See if a movie is happy or sad with sentiment an... by sharemywin on Friday May 05, 2017 @12:11PM (#14274511)
has anyone tried opensubtitles.org

just found it on google

Re:Show HN: See if a movie is happy or sad with sentiment an... by curiousgal on Friday May 05, 2017 @01:39PM (#14275273)
Re:Show HN: See if a movie is happy or sad with sentiment an... by vit05 on Friday May 05, 2017 @12:54PM (#14274875)
Just sometimes, because is the only place that has subtitles for some old movies. But I personally prefer Subscene.com.
Re:Show HN: See if a movie is happy or sad with sentiment an... by sharemywin on Friday May 05, 2017 @12:12PM (#14274522)
never mind looks kinda scammy
Re:Show HN: See if a movie is happy or sad with sentiment an... by icebraining on Friday May 05, 2017 @12:16PM (#14274554)
Nah, it's fine, it tries to push you their downloader, but if you avoid the pitfalls the content is good. I've used it many times over the years.
Re:Show HN: See if a movie is happy or sad with sentiment an... by gh1 on Friday May 05, 2017 @03:58PM (#14276374)
I like it. What do you think of a recommendation engine which tries to recommend movies based on patterns of happiness and sadness?
Re:Show HN: See if a movie is happy or sad with sentiment an... by snissn on Friday May 05, 2017 @11:22AM (#14274083)
Why does the graph go from green to blue?
Re:Show HN: See if a movie is happy or sad with sentiment an... by eljs on Friday May 05, 2017 @12:12PM (#14274516)
Good question, I just thought it looked nice but maybe I shouldn't have
Re:Show HN: See if a movie is happy or sad with sentiment an... by snissn on Friday May 05, 2017 @05:28PM (#14276969)
Cool! I thought it was an indication of sentiment at first, I was like oh yeah la la land gets sad towards the end. Cool endeavor!

Curious if you investigated other libraries or algorithms. have you seen this - i have been playing with it, seems reliable: https://github.com/cjhutto/vaderSentiment

Re:Show HN: See if a movie is happy or sad with sentiment an... by zoidb on Friday May 05, 2017 @10:44AM (#14273784)
You should let people see what other people submitted although it's pretty easy by fiddling with the URL. I like seeing the word patterns but I really wonder how well it can detect true sentiment, not an easy thing to do. https://www.crealdo.com/story/movie-sentiment/movies/13
Re:Show HN: See if a movie is happy or sad with sentiment an... by rodrigocoelho on Friday May 05, 2017 @01:58PM (#14275442)
Also interesting: Character-to-Character Sentiment Analysis in Shakespeare’s Plays (PDF)

http://www.aclweb.org/anthology/P13-2085

Re:Show HN: See if a movie is happy or sad with sentiment an... by AznHisoka on Friday May 05, 2017 @09:13AM (#14273155)
What about a movie lile "Brazil"? would that be considered happy or sad?
Re:Show HN: See if a movie is happy or sad with sentiment an... by eljs on Friday May 05, 2017 @09:20AM (#14273201)
I don't know this movie but you can try it out by uploading the movies srt file.
Re:Show HN: See if a movie is happy or sad with sentiment an... by cholantesh on Friday May 05, 2017 @12:06PM (#14274458)
Brazil is a dystopian satire. I would wager that GP is asking about scenarios where there is significant ambiguity and/or situational/verbal irony.
Re:Show HN: See if a movie is happy or sad with sentiment an... by eljs on Friday May 05, 2017 @12:11PM (#14274509)
I just gave it a try: https://www.crealdo.com/story/movie-sentiment/movies/30 Never seen the movie so I can't judge myself.
Re:Show HN: See if a movie is happy or sad with sentiment an... by blacksmith_tb on Friday May 05, 2017 @02:09PM (#14275530)
That seems like a plausible analysis, the movie becomes darker as it proceeds, though there's a false happy ending (which in some cases of ham-fisted editing is the only ending - I saw it aired on network TV in the US this way) followed by a much more somber closing.
Re:Show HN: See if a movie is happy or sad with sentiment an... by blackflame7000 on Friday May 05, 2017 @06:06PM (#14277160)
It's broken. My sentiment of La La Land was nowhere near that cheerful lol. In all serious though, its pretty cool idea for data analytics.
Re:Show HN: See if a movie is happy or sad with sentiment an... by huangc10 on Friday May 05, 2017 @03:01PM (#14275971)
At the top, can you put a score or average of the analysis so we know if the overall movie is happy or sad? That number will be interesting :D
Re:Show HN: See if a movie is happy or sad with sentiment an... by kingkawn on Friday May 05, 2017 @11:21AM (#14274070)
Horrifying.
Re:Show HN: See if a movie is happy or sad with sentiment an... by opticals on Friday May 05, 2017 @12:47PM (#14274813)
Make it available in French!