News for hackers. Stuff that matters.
Re:Fitting to Noise or Nothing at All: Machine Learning in M... by dsacco on Monday August 07, 2017 @12:39AM (#14945351)
Rentec uses machine learning, but more importantly the firm curates massive amounts of high-signal data. The most significant part of their work lies in process automation and the rapid testing of hypotheses, which empowers the optimal use of mind-numbing amounts of data that most other firms simply can't take advantage of. Very early on their success was due (in part) to the willingness of Simons et al to use correlations in disparate datasets which could be proven but which didn't really make sense, and which wouldn't be explained by anything intuitive.

In other words, Rentec is not just pointing machine learning models at data, they're investing in a very robust data processing pipeline. Everything before the analysis is just as important as the analysis at funds like theirs.

Re:Fitting to Noise or Nothing at All: Machine Learning in M... by ata6187 on Monday August 07, 2017 @01:18AM (#14945472)
Just want to second this comment, their data processes are the key strength of Medallion. Grandparent comment by murbard2 also talks about the importance of this component to quant work (in the last paragraph: "finding new data feeds that provide valuable information")

While Jim Simons is a mathematician and Rentec clearly has hired many brilliant people with PhDs, it's maybe worth mentioning the actual mathematics being used in their work isn't super high level difficult, impossible or secretive. Many of the PhD's working there do not have a PhD in math, but rather something like Physics, so I would say if you are familiar with graduate level math courses you can understand the math needed for this type of work. Math isn't where their edge comes from. Also Medallion is 30 years old, their early work in the mid 80s was done on computers with less processing power than your phone, "Machine Learning" as the term is being used lately or access to supercomputing hardware no one else knows about is also not where their edge came from.

Re:Fitting to Noise or Nothing at All: Machine Learning in M... by dsacco on Monday August 07, 2017 @01:37AM (#14945541)
Well said. Where most funds have the same problem 'chollida1 describes here[1], Rentec (and other similar firms) moved past that by establishing the right culture and investing in the right technology from the outset.

They need smart people, but hiring the smartest people and having the most sophisticated models won't do you any good if you can't acquire high signal data, can't clean that data properly and can't rapidly backtest. And if you can't do any of that, adding more data is just going to add more noise.

_____________________

1. https://news.ycombinator.com/item?id=13139638#13140352

Re:Fitting to Noise or Nothing at All: Machine Learning in M... by ata6187 on Monday August 07, 2017 @01:49AM (#14945577)
Know all about it, was a data lackey B* at a HF (not Rentec) when I first graduated from school.

By the way for anyone reading this, Rentec is hiring, need to know C/C++ and a "basic knowledge of statistics" is preferred but not required.

Re:Fitting to Noise or Nothing at All: Machine Learning in M... by dsacco on Monday August 07, 2017 @01:59AM (#14945607)
Given your background, I'd be interested in picking your brain a bit for a few projects I'm working on. If you're looking to remain anonymous would you mind sending me an email (in my profile), or throwing an email up in yours?
Re:Fitting to Noise or Nothing at All: Machine Learning in M... by RealAnalysis on Monday August 07, 2017 @11:50AM (#14948413)
but i feel like they're looking for PhDs and Math Olympiads.
Re:Fitting to Noise or Nothing at All: Machine Learning in M... by roksprok on Monday August 07, 2017 @02:13AM (#14945654)
Is there really that much high-signal data that is not already being used? Is their advantage in finding signals in data that other people overlook, or finding new data sources?
Re:Fitting to Noise or Nothing at All: Machine Learning in M... by dsacco on Monday August 07, 2017 @02:29AM (#14945704)
> Is there really that much high-signal data that is not already being used?

Yes.

> Is their advantage in finding signals in data that other people overlook, or finding new data sources?

Yes.

I won't go into any particular detail, but there is a lot of signal in the market for those who are imaginative. The obvious and low hanging fruit is long gone, but there are still many places that offer an edge.

Re:Fitting to Noise or Nothing at All: Machine Learning in M... by RealAnalysis on Monday August 07, 2017 @11:01AM (#14947960)
I wish there was some sort of full intro tutorial on finding strategies; ie: an example of a former signal (now traded away), the thought process, the data sourcing, statistical analysis, trading/signal strategy, etc..
Re:Fitting to Noise or Nothing at All: Machine Learning in M... by dsacco on Monday August 07, 2017 @11:21AM (#14948147)
The thing is that no one is really motivated to make a complete tutorial on finding strategies because it's economically irrational. You're either giving away specific sources of alpha or you're empowering potential competitors. This is why it's virtually guaranteed that anyone selling courses that teach trading or related skills is a fraud - they have essentially no incentive to just ramp up their own trading capital instead.

The industry is also extremely secretive (necessarily so). You'll hardly ever find a good treatise on finding novel signals, but there are tutorials on algo trading in general with examples of production strategies that used to work which have been, as you say, traded away. For that purpose I'd recommend you start here: http://www.decal.org/file/2945

Re:Fitting to Noise or Nothing at All: Machine Learning in M... by RealAnalysis on Monday August 07, 2017 @11:03AM (#14947980)
also, is this high-signal data publicly available? or only for purchases from vendors?
Re:Fitting to Noise or Nothing at All: Machine Learning in M... by dsacco on Monday August 07, 2017 @11:16AM (#14948103)
It's public (technically it has to be in order to be strictly legal for use). But for the most part it is unintuitive, unclean (needs to be heavily normalized) and not easily accessible. There are a variety of vendors that source it, clean it and analyze it to make it salable to firms. Quantitative firms also have teams devoted to doing all of that internally.
Re:Fitting to Noise or Nothing at All: Machine Learning in M... by RealAnalysis on Monday August 07, 2017 @11:46AM (#14948364)
how does one go about getting a quant position? (PhD, bachelors, masters?) I feel like there's a bias towards PhDs...