News for hackers. Stuff that matters.
Re:Sentiment Analysis on Enron's Emails with Apache Spark by etjossem on Friday May 06, 2016 @06:02PM (#11647010)
If this type of analysis can't answer "who Ken Lay likes best", maybe that's not the best question to use as an example.

There are plenty of more interesting questions ("when did Enron employees start getting nervous or alarmed?") that can be answered by this dataset.

Re:Sentiment Analysis on Enron's Emails with Apache Spark by mindcrime on Friday May 06, 2016 @11:09PM (#11648215)
If this type of analysis can't answer "who Ken Lay likes best", maybe that's not the best question to use as an example.

"Who did Ken Lay like best" was originally going to be the title of the post, until we realized that you can't actually answer that question. It was someone else's idea anyway, but I'm not naming names!

That said, I only had so much time to work on this, and there was a limit to how much depth I could go into. I wanted to go deeper and introduce some network science / social network analysis but had other priorities.

But still, the point was just to illustrate a cool application of that particular technology stack and pique people's interest in it. If that interest helps us land business, that's a nice outcome. If not, maybe somebody learned something or came up with an idea for their own project, as a result of reading it. Those are nice outcomes as well. :-)

There are plenty of more interesting questions ("when did Enron employees start getting nervous or alarmed?")

That is a pretty good one... maybe somebody else will take the bull by the horns and write a post on that. If not, maybe I'll do it later for a "part 2".

Re:Sentiment Analysis on Enron's Emails with Apache Spark by kafkaesq on Friday May 06, 2016 @06:09PM (#11647044)
It's not that kind of an "analysis."

More like a, you know, "Look, Ma -- I got Spark, Hive, and HDFS on my resume now!" analysis.