ghostofml

* But, crucially, an ensemble of linear models is just a linear model. So in fact, there should be no variance reduction in the authors' approach. *

You are right. But I disagree that " It should be better to train a single linear model on ALL the data " - because its impractical to find the right parameters for your training method. Since this is a large data problem, I assume author's are using a parametric optimization routine? - Hence a practical solution is to learn multiple linear models using different parameters and using different subsets of the data.