I've got ~10/1 ADSL (nominally 8 megabits on a good day). This console generation is really testing the limits of my bandwidth - Halo 5 is 60+ GB, and it doesn't help when you've got errors like http://www.iinet.net.au/status/4438711 happening. 60GB would take 16 hours which is overnight-and-a-bit.
Unfortunately the One is not fault tolerant at all. I'll turn it on in the morning and it won't have started at all (despite having the settings right for "download while in low power mode).
I can't imagine using the Xbox One as a download-only console if you're on anything less than true unlimited data (I did 400GB of XBox/Netflix content alone last month).
I should put this in a blog post at some point but I'm seeing this more and more from Microsoft and Google - assume everyone's on 100MB+ connections.
e.g.
* OneDrive has no LAN sync, placeholder files have been removed so you can't even have your files available "on demand"
* Google removing SD card support left right and centre.
Legacy DSL is indeed a problem for that. High end games (and PC ones are even more so than console ones) are huge.
As I wrote below - DRM free games can solve this with services which will just ship you a game on flash storage on demand. DRMed ones however (and most current consoles are sick with DRM) would have an issue with such method. Your option could be ditching such consoles and sticking to DRM-free games and platforms.
And it's indeed easy to make it a service on demand, when they get the digital game and ship it to you on a small flash drive. It's not a problem for DRM-free games, and may be GOG can even start offering such service in the future. DRMed games however shoot themselves in the foot by making it too hard, but I personally don't care about them.
There are lots of people on capped satellite/3G/"water tower" internet who enjoy gaming that are going to be cut out of the market if consoles go to a pure digital distribution system. Additionally, gamers in very remote locations (submarines, remote foreign villages, etc) couldn't have the latest releases mailed to them. Somehow they'd have to find a broadband connection. It doesn't make much sense to eliminate a class of paying customers by switching to 100% digital distribution.