Maybe. Maybe not.

I can borrow e-books from my library, nearly as fast as buying them from Amazon.

I can borrow physical books by hitting their site, putting a hold on it, and then next time I'm in town, I pick it up. It's not UPSd to my door, but it's not that big of an investment. (or, more likely, Wife + kids pick it up next time they bring home 40 books from the library. We sometimes have a good couple feed of the holds shelf.)

You're saying that time is valuable. The cost of the book is the dollar price + the time to acquire + the time to read it.

You're saying that the middle cost is significant. That suggests that the latter cost is also significant. Therefore lowering the dollar price of the book has little effect on the total in the above equation.

Grossly misguided equation. The cost of the book is the price + the time to acquire. The benefit of the book is in the enjoyment of reading it -- which implies that readers don't see time spent reading as a "cost", unless you believe they're seeing reading as a benefit and a cost at the same time. Believe it or not, people actually enjoy the process of reading and seeing a work unfold, not just checking a completed book off the reading list.

The cost is the movies, video games, time with family etc that you didn't enjoy because you spent time enjoying the book.

Let's put it another way. When do avid readers start reading a new book? When they finish or get bored of their previous book.

The fact that the price of something is high does not mean the elasticity is low.

Even if we agree that the total cost (including time) of acquiring and reading a book is high, that does not indicate that demand is inelastic.