It's very far from idle speculation at this point; many prominent economists and economic researchers[1] have grappled with the issue. Both the Lump of Labor Fallacy and Luddite Fallacy may prove themselves to be fallacious in the new era. Gwern has a good post with links:

http://www.gwern.net/Mistakes?cx=009114923999563836576%3Adv0...

Here are a few prominent ones:

http://economics.mit.edu/files/5554 https://hbr.org/2015/06/the-great-decoupling http://www.economist.com/news/briefing/21594264-previous-tec... http://ftalphaville.ft.com/2014/08/28/1942571/david-autors-g... http://krugman.blogs.nytimes.com/2011/03/06/autor-autor/?_r=... http://www.asymptosis.com/are-machines-replacing-humans-or-a...

[1] Autor, Ford, Cowen, Krugman, Acemoglu, Brynjolfsson, McAfee just to name a few