To be clear: I have mined and sold data to hedge funds before, including well known ones, and from that work I derived two conclusions:
1. It is fundamentally possible to beat the market consistently and legally by exploiting information asymmetry, and
2. Most hedge funds, especially the most successful ones, do not operate via illegal insider information (and will, in fact, fly into a compliance department initiated lock down on a security if, say, an overzealous data vendor compromises confidentiality).
Basically, I'm asking for clarification because as stated, I disagree with your assertion here. There are legal grey areas and some people are outright criminal, but overall I wouldn't characterize the top performing funds as being wholly or even close to partially dependent upon insider information, let alone at a systemic organizational level.
Are you able to tell more about what types of data this was? I'd be interested in hearing more. I love the story of hedge funds using satellite imagery of parking lots to predict retail store strength
I mined data in the real estate, QSR, automotive and airline sectors (and a few peripherally related ones). We would identify a source of data that was a demonstrably strong proxy for a specific company's revenue (that is to say, if we broke out a naive timeseries of the data it would map nearly 1:1 to earnings results each quarter). Then we would collect this data en masse, writing software over a custom crawler and infrastructure, bespoke to each case. During ingestion we'd process and normalize the data and dump it in a database. Once we had a significant amount of data we'd build a forecasting model, which could be very straightforward ("how many products have they sold this quarter", for simpler companies) to very complex ("can we reverse engineer the amount of business which is now online" or "can we determine the undisclosed amount of business being done in this specific region"). This analysis (and not the data itself) constituted the product - we never even provided the data directly to the hedge funds.
Once the crawling and analysis was mature, we'd incubate it for a few quarters while building interest; then, after showing a very strong mapping over several quarters, offer it as a data product. It was not uncommon to achieve <5% or even <1% margin of error for forecasting earnings announcements on a quarterly basis.
The data was always public and legal, though challenging to identify and difficult to effectively crawl. It was typically derived from very uncommon sources, and would range in complexity from the extremely simple ("we can crawl sequentially incremented integers from this third party that appear to map to products sold by this company") to the very sophisticated and complex (e.g. "we've collected a zero sum distribution of ad partners and their spend on the network, are more partners churning off, and is this a better or worse outcome for the company?"). I've spoken about this in comments here before.
This is still very doable, and I still do this sort of work for personal research (and profit). However, the work I do is now much more quantitative - I'm currently taking a similar approach for baskets of equities with the goal of forecasting macroeconomic trends (e.g. subprime auto loans) instead of the earnings results of individual equities. To give a very specific throwaway example (because it doesn't violate an NDA and it no longer works): it used to be possible to pretty accurately forecast large tech retailers' product sales each quarter (like Apple) by reverse engineering FedEx and UPS tracking numbers.
I once briefly chatted with a man who ran a company that used satellite imagery to predict crop yields. I found it amusing how rather than providing this information to farmers ("It looks like you're going to produce only 60% of what you did last year, time to cut some expenses!"), there was so much more money in providing this information to hedge funds and trading firms. Morality aside, the potential for technology like this is exciting.
It's kind of cool how the combination of big data, scraping, statistics, etc. allows this small niche of analytics to thrive. I have a decent background in most of these subjects, and this seems like a very freelance-able job, so I'd love to dabble in this area and learn how it works.
It is context dependent, but I can give a basic idea. In the example you've given, high four figures per customer per quarter, assuming you just give them a deliverable report once a month and the data is turnkey enough that many customers (30 - 40) can "subscribe" to it. For a fund that wants something more boutique, like exclusivity or a specific company, five figures becomes the floor.
The nature of the business is significant upfront work (setup of data collection and analysis model) followed by infrequent tweaks going forward. Lucrative data on a single hot company could be sold for mid - high six figures per year with very high profit margin for as long as the data is available and the analysis model is both functional and interesting (i.e. delivers an insight not already priced into the market).
I'm also curious as to the level of "polish" that you are expected to provide. Let's say you know how many widgets Acme sold this last quarter. Is that by itself enough information, or would you need to calculate projected expenses / other stuff to actually be able to deliver an earnings estimate? What if you just had projected revenue?
Thank you!
That's great. Were tracking numbers vendor-specific in some way? So you could, let's say, order an iPhone once a week and get an idea for how many iPhones (or total Apple products) were sold in that time period, just by using the tracking numbers? Was it dependent on geography in some way?
What changed BTW? Did the format of tracking numbers change?
Just two per carrier would do: one at the start of the quarter, and one at the end. Earnings dates happen weeks after the quarter ends, so you'd have heaps of time to peddle your analysis around.
https://www.ft.com/content/efda2ca2-ec69-11e6-930f-061b01e23...
https://www.nytimes.com/2016/01/09/business/dealbook/sec-and...
1) they can't buy in to using valuation as their sole investment criteria. They want to be "smarter", thinking they can see things others can't using psychology or charts or other pseudo sciences. 2) They fall sway to Mr Market ( read the Ben graham parable). 3) They can't sell value to clients and only want a strategy clients will pay for. 4) Buffett spends a great deal of time trying to make decisions free of bias. A CFA program teaches you nothing about this, but it's critical to good valuation work. For example, WEB does not want to know the stock price of a company he's analyzing until he's made his valuation estimate. He doesn't want to be biased into giving a too high valuation to a company he loves just to ensure he can buy it.
Lastly, there is a meta level to value investing. A great valuation at a current price isn't enough if the company lacks a competitive moat, or management can't be trusted to treat shareholders well. First level cigar butt value investors never learn this.
Its not an informational asymmetry, it's what you do with it. Walter Schloss is another great example. Beat the market by 5% a year for over 40 years, did it buying exclusively the doggiest public companies. He was the ultimate cigar butt hunter. His advantage wasn't information Wall Street didn't have, it was his willingness to use information Wall Street wouldn't, to invest in opportunities they would not.
Buffett does have one informational edge, his experience and judgement. He knows how value works, he has total confidence it works, and he never panics, even when down 50% in 2008. He never leverages himself in a dangerous manner, never gives someone else control over his decisions.
Or perhaps you worked at Reuters or similar, in which case you would not have much visibility into the funds.
So are your two sentences related in any way?
Also, it's not like actual illegal behavior is uncommon when it's hard to prove, and it's not like a lot of people aren't proud of it. Watch Jim Cramer's interview with Jon Stewart as an example and the reaction to Wolf of Wall Street as another.
In business school, a friend asked me to recommend firms where he could do some quiet insider trading by tipping off relatives who would trade on his behalf. He ended up at MF Global.
There aren't statistics on this kind of thing, but my experiences have all told me succeeding as a hedge fund requires breaking the same rules as everyone else, not unlike doping in pro sports.
Trading on non public information for a gain is going to land you in hot water with the SEC.
These two lines on the SEC site [1] define this.
>Corporate officers, directors, and employees who traded the corporation's securities after learning of significant, confidential corporate developments;
>Friends, business associates, family members, and other "tippees" of such officers, directors, and employees, who traded the securities after receiving such information
1. Bribery, or a monetary reward,
2. Friendship and good relations.
In both cases there must be an unbroken chain of confidentiality compromise. If you legitimately come to know non-public material information about a company and you didn't acquire this information through your own or someone else's confidentiality agreement, you're fine to use it.
Information asymmetry couldn't be functionally and profitably exploited if it was literally illegal full stop. You just have to acquire it without breaking a confidentiality duty to your own company and without aiding and abetting someone else in breaking such a duty (e.g. they tell you and you trade).
I'm surprised that anyone is surprised by my statement because it seems like an open secret. I don't think most of it is illegal, but sharing of non-public info is very, very common.