News for hackers. Stuff that matters.
Re:Can the Bloomberg Terminal be "Toppled"? by izyda on Wednesday March 19, 2014 @05:51PM (#7432136)
Great points. I think #2 (reliability) and #3 (reputation)are really key here.

I would argue that there is a caveat to #1. Agreed that a MVP for a Bloomberg replacement wouldn't work unless you had everything Bloomberg had and more (in which case, it's hardly an MVP and good luck). That said, I think an MVP for a niche fin-tech data product that Bloomberg does not have would work if the sales pitch is that I use your product in addition to Bloomberg. If you offer me an edge by providing data (or analysis) others don't have, an MVP would be fine. Heck, some traders/funds would pay for just a csv file if it contained unique (but directly useful) financial data/analysis (an obvious example that may have had potential 5 years ago was sentiment, although now everyone and their dog is offering that and it's not entirely clear there's direct value to it). I think the very fact Bloomberg can look like its "from the 1980s" just reflects the needs of the market it serves. Wall Street doesn't put great value on pretty interfaces, ease of use, or anything other than direct value add to their revenue generation process (ie. investing, pricing, selling financial products, etc.). Tell me how your product makes me more money today. I think any fin-tech startup needs to be driven by that philosophy if they want to succeed, I don't see how you'd convince a trader that the fact your app offers a nice interface or interesting but questionably actionable analysis is any real advantage if the other guy offers analysis or data that is directly applicable.

Another thought: while disrupting pricing data would be very difficult (given the speed + accuracy required + relationship with exchanges), what about other financial data like fundamentals/accounting/relationships data? That data is free - it's all on the SEC's EDGAR (and it's far more than just financial documents, there's plenty of data on individuals too). Would Fact Set or Capital IQ, for instance, have more competition if the SEC had API/machine readable data? There has been some effort through XBRL to do this - although the percentage of filing actually available in that format and the lack of consistency in the format (in some cases, there's at least several hundred 'tags' for the same or similar financial line item) makes any large scale data processing from EDGAR a massive undertaking.

Re:Can the Bloomberg Terminal be "Toppled"? by csandstedt on Thursday March 20, 2014 @07:17AM (#7434850)
"the lack of consistency in the format (in some cases, there's at least several hundred 'tags' for the same or similar financial line item) makes any large scale data processing from EDGAR a massive undertaking."

Izyda, you are absolutely correct on the issues with XBRL. I'm the co-founder of a company called TagniFi that is working on a solution. We have a standardized dataset that makes comparing this data across companies, industries and sectors possible. We are in beta so you are welcome to check out for API for free: http://www.tagnifi.com

Re:Can the Bloomberg Terminal be "Toppled"? by izyda on Thursday March 20, 2014 @12:03PM (#7436379)
This looks very cool - I will certainly sign up for the beta. I think there is real potential in this space.

The only concern - and maybe you have a solution - I have is that to my knowledge, XBRL has only been required on 10K's and 10Q's since around 2009 (I may be incorrect, 2010?). As someone involved in research with financial data, there's really no avoiding using pre 2009 data. Parsing that from raw text files is even harder than XBRL - is TagniFi able to do that as well? (I've perused your website but haven't found the answer). Unfortunately, without that capability, I'm forced to rely on services like Fact Set at least partially.

Re:Can the Bloomberg Terminal be "Toppled"? by csandstedt on Thursday March 20, 2014 @04:16PM (#7438359)
Thanks for the feedback. You are correct that the lack of history is an issue for us but we have to start somewhere and the cost to collect non-XBRL data is really high. The timing of the available history in XBRL depends on the size of the company. XBRL was phased in starting in 2009 with the largest 500 companies filing their 2010 10-Ks in XBRL. These 10-Ks generally had income statement data back to 2007 so that is where most of our data starts on the larger companies. The next largest 1,500 companies started filing in 2010 so their data generally starts in 2008. The remaining 9,000 companies started filing XBRL in 2011 so their data generally starts in 2009. This means we have 7 years of data for the largest 500 companies, 6 years of data on the next largest 1,500 companies and 5 years of data on the remaining 9,000 companies. Since the cost to go back and collect this data manually is really high we are going to focus on building the deepest datasets (footnotes and industry-specific) to offset the lack of history.
Re:Can the Bloomberg Terminal be "Toppled"? by seeingfurther on Wednesday March 19, 2014 @09:52PM (#7433300)
"(an obvious example that may have had potential 5 years ago was sentiment, although now everyone and their dog is offering that and it's not entirely clear there's direct value to it)"

Presumably the sentiment you are referring to is real time financial sentiment derived from online chatter. A source other than online chatter wouldn't yield enough data in terms of time granularity or symbols, i.e we've had sentiment data for decades in the form of the AAII and NAAIM, they just weren't useful as data sources for active trading.

5 years ago a statistically significant sample set of investors and traders wasn't chatting online to yield actionable data. Today there is, the data has alpha, and smart money is trading on it.

Re:Can the Bloomberg Terminal be "Toppled"? by izyda on Wednesday March 19, 2014 @10:40PM (#7433480)
No, actually, I am referring to real time sentiment analysis of any kind - services such as http://www.dataminr.com/ or http://www.infotrie.com/ do this today mostly using online chatter, which you correctly point out probably would not be possible 5 years ago. However, real time news sentiment analysis would have been possible and relatively novel 5 years ago. For instance, Columbia's Paul Tetlock published a very interesting paper in the Journal of Finance in 2007 ago on the subject here: http://www0.gsb.columbia.edu/faculty/ptetlock/papers/Tetlock... (he used words from a Wall Street Journal column). If you read the paper, you'll find he used daily data over the 16 year period 1984-1999. While this is not of the same granularity you could get from services like InfoTrie or Dataminr, it certainly is of high enough granularity to trade on (and could have been done 10 years ago apparently!) and appears more useful than AAII or NAAIM.

I am interested in your claim that the data has alpha and smart money is trading on it. I have been looking for any peer reviewed, academic articles actually documenting that alpha (ie. by anyone except the people trying to sell you the data). One such paper I found was Bollen et al's http://arxiv.org/abs/1010.3003. From my understanding, there are several problems with the methodology (I could find a rebuttal paper I read if you're interested) and the methodology was used in Derwent Capital - although the fund only traded it for 1 month: http://venturebeat.com/2012/05/28/twitter-fueled-hedge-fund-...

Care to point me to anything else like that?

Re:Can the Bloomberg Terminal be "Toppled"? by seeingfurther on Thursday March 20, 2014 @11:06AM (#7435948)
DataMinr is an event detection service not a 'real time sentiment'

infotrie does sentiment analysis on news stories they also mix this news sentiment with social media sentiment. This is a muddled approach and not pure real time sentiment.

Let's back up a moment. You wouldn't expect Gallup to have accurate surveys of the presidential election by processing news articles and blogs. They poll individuals in brief snapshots of time. Polling for sentiment is the same thing. News articles and blog posts are amplified opinions of a small set of people, they are a poor substitute for direct measurement of many many people.

The Bollen paper is widely discredited in our space. Derwent licensed the technology and subsequently ran into trouble because the tech and data was not sound.. though Bollen's initial theory was correct.

Not peer reviewed but you can see a few papers Deltix published documenting alpha:

http://www.deltixlab.com/research/an-automated-trading-strat...

http://www.deltixlab.com/research/an-automated-trading-strat...

Re:Can the Bloomberg Terminal be "Toppled"? by izyda on Thursday March 20, 2014 @12:32PM (#7436632)
Interesting points...

News articles and blog posts are amplified opinions of a small set of people, they are a poor substitute for direct measurement of many many people.

You might be correct (sounds very reasonable) but I guess the question is why do I necessarily care about directly measuring many many people? What if loud, amplified opinions are actually better indicator of market momentum?. Tetlock's article suggests that those amplified opinions actually do contain useful information. Again, I have yet to find any conclusive evidence that measuring people's chatter actually ... matters.

Derwent licensed the technology and subsequently ran into trouble because the tech and data was not sound.. though Bollen's initial theory was correct.

Right - I think there are more than a couple complaints about his paper. Why do you say his initial theory was correct though? If I recall correctly, his initial theory was that social chatter contained market information that could be extracted using neural networks and used to predict short term market returns (like a few days forward). I haven't really seen anything along the same lines (especially using his approach of neural networks) anywhere - although the articles you link to might be considered similar.

I am just reading through the articles you posted from Deltix now, sound interesting. One problem is that because of the relatively novelty of services like PsychSignal, the data does not go back as far as traditional indicators (this was a problem with Bollen's paper as well). When you say you have impressive alpha, sharpe, or whatever for 6 months.. that does not really prove much, especially in the eyes of academics. I'd love to replicate or work with this data presented in Deltix - but given the data is proprietary, that's difficult to do.

side note: This was just mentioned in the Wall Street Journal yesterday, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1807265 claims to find signal in blog posts. That's not to say measuring many people's chatter wouldn't work as well or better though. Also just heard of paper today, so no guarantees about quality but maybe someone finds it interesting.

Re:Can the Bloomberg Terminal be "Toppled"? by apaprocki on Wednesday March 19, 2014 @08:07PM (#7432842)
Re: "shipping a csv" -- this market is exactly what the Bloomberg App Store caters to. Bloomberg takes care of all the distribution, billing, discoverability headaches for your niche play and you just focus on the product. The nice thing about the store is that all apps are monthly recurring fees. There are no 0.99 downloads and some are quite expensive.
Re:Can the Bloomberg Terminal be "Toppled"? by izyda on Wednesday March 19, 2014 @10:43PM (#7433488)
totally agree - and probably why if all you have is a niche play, even if it has potential to be extremely profitable, attempting to replace the Bloomberg App Store is like trying to replace the Apple app store for one mobile game.