You may not think you take value from the publishing ecosystem (discovery, marketing, editing, publishing, etc. of content, and reliable delivery, storage, security, etc. of retail), but you probably actually do. And, you know, even if you actually don't, you can just go right ahead continuing to buy 1.99 or 3.99 or whatever is the "right" price of ebooks either direct from author websites (although it's still really hard to get the margins down to zero unless you're meeting on street and paying cash and accepting delivery over a thumb drive or something) or from Amazon or B&N's self-publishing programs.
In any case, comments like this are really hard to square with how much criticism my employer (B&N: although I don't work on ebooks, and certainly this is my own opinion) has gotten from investors for the money they've put into NOOK and the ebook ecosystem.
Perhaps eventually a netflix or spotify-like e-book model will emerge, where you pay a monthly fee (or see ads in books) to get access to most any e-book. 24symbols is trying this, but apparently not nearly there yet. Amazon is working on it too http://mashable.com/2011/09/12/amazon-netflix-books/