Because you might want, as I definitely wanted, to also iterate over the feature engineering, not only the network and training parameters. Thus, doing ETL "inline" is really cool, and speeds up your iteration.

It is also easy once you have a proper language that supports multi-threading. Since you do it "full speed" with full utilization of the GPU, there is no disadvantage of doing the ETL while training.