News for hackers. Stuff that matters.

Realtime Twitter / capital market analysis in Haskell. Also, anyone hiring?

http://ego.fm

Just wanted to share my hobby project with y'all - it reads Twitter and tries to classify each message as "bullish" or "bearish", then keeps track of which ticker symbols are mentioned, calculates the most mentioned each hour and creates a leaderboard. The backend is Haskell (Snap framework) and Redis, the frontend is Raphael.js and JQuery.<p>As an aside, I recently have found myself unemployed - anyone hiring or have some contract work they need done? Take a look at my github account at http://www.github.com/texodus for some of my code, including (soon) this site.
( 14 comments | 26 points | on HN )
Re:Realtime Twitter / capital market analysis in Haskell. A... by texodus on Monday August 02, 2010 @02:38PM (#1568549)
Just wanted to share my hobby project with y'all - it reads Twitter and tries to classify each message as "bullish" or "bearish", then keeps track of which ticker symbols are mentioned, calculates the most mentioned each hour and creates a leaderboard. The backend is Haskell (Snap framework) and Redis, the frontend is Raphael.js and JQuery.

As an aside, I recently have found myself unemployed - anyone hiring or have some contract work they need done? Take a look at my github account at http://www.github.com/texodus for some of my code, including (soon) this site.

Re:Realtime Twitter / capital market analysis in Haskell. A... by tlack on Monday August 02, 2010 @08:53PM (#1569692)
Fantastic idea and challenging path to build it - kudos!
Re:Realtime Twitter / capital market analysis in Haskell. A... by yummyfajitas on Monday August 02, 2010 @05:15PM (#1569116)
Just curious, could you post an example graph with both bullish and bearish twits?

My company is looking for employee #4, we do high frequency trading:

http://meshcapital.com/Careers.html

Re:Realtime Twitter / capital market analysis in Haskell. A... by texodus on Monday August 02, 2010 @05:40PM (#1569189)
Here's a screenshot of Apple as they announced their quarterly earnings:

http://ego.fm/screenshot.png

The red & green spikes represent individual polls of Twitter, and the overall sentiment regarding the ticker - the dotted line is the cutoff for which scores are used to calculate the leaderboard.

I live in Brooklyn, so Jersey may work.

Re:Realtime Twitter / capital market analysis in Haskell. A... by m4wk3r on Wednesday August 04, 2010 @05:42AM (#1574092)
your site seems to be down.. how consistently is there this strong of a correlation? have you simulated any trading based on your system?
Re:Realtime Twitter / capital market analysis in Haskell. A... by wallflower on Monday August 02, 2010 @09:14PM (#1569733)
Beautiful. Very inspiring. Best Twitter visualization I've seen since TwitterVision and Twistori. I love that the data is beautifully visualized.
Re:Realtime Twitter / capital market analysis in Haskell. A... by steveplace on Monday August 02, 2010 @03:00PM (#1568620)
stocktwits is looking for a ruby on rails + wordpress guy. Considering your site's down the same vertical, you might want to look into it.
Re:Realtime Twitter / capital market analysis in Haskell. A... by texodus on Monday August 02, 2010 @05:04PM (#1569069)
Thanks, I'll check them out!
Re:Realtime Twitter / capital market analysis in Haskell. A... by cschep on Monday August 02, 2010 @03:54PM (#1568822)
nitpick: your
Re:Realtime Twitter / capital market analysis in Haskell. A... by steveplace on Monday August 02, 2010 @04:02PM (#1568849)
fixed, it's because i was adding an apostrophe on the following word, a little grammar slip. My deepest apologies.
Re:Realtime Twitter / capital market analysis in Haskell. A... by cschep on Monday August 02, 2010 @04:08PM (#1568869)
Ah! It's certainly not a problem. Forgive me if I came off harsh. I am bleeding karma these days...
Re:Realtime Twitter / capital market analysis in Haskell. A... by herrherr on Monday August 02, 2010 @02:31PM (#1568531)
More info would be quite helpful.
Re:Realtime Twitter / capital market analysis in Haskell. A... by korch on Monday August 02, 2010 @07:09PM (#1569453)
I love the look of the site! And kudos for using Haskell + Raphaeljs. It's a great example use-case for data mining Twitter. There are so many interesting data-mining mashup possibilities on Twitter that I've been disappointed by the lack of creative imagination shown in the various Twitter-based web apps aggregating and interpreting trends via Twitter.

However, I do wonder if it would be easy to spam Twitter and thereby ruin its usefulness for real-time data mining. If Twitter can be used to accurately track trends, as soon as everyone knows trends are being created based on Twitter, then they can manipulate it, similar to pump-and-dump spam schemes we've seen on various financial news boards, like Yahoo, Motley Fool etc.

Re:Realtime Twitter / capital market analysis in Haskell. A... by texodus on Monday August 02, 2010 @11:33PM (#1570022)
You're absolutely right - it would be trivial to manipulate the rankings. Disclaimer: I highly recommend no one make any investments based on my site's picks! This is just a data mining demo.