I am not sure what time spans you speak about. Thousands of years is possible on fission with current technology and at current levels of demand.
http://www.theoildrum.com/node/2379
(and understand that "thousands of years on fission with current technology" is pretty far from the truth)
I can envision a future where libraries have few books. Here's one at a school in Florida - http://www.reuters.com/article/2014/08/25/us-usa-florida-lib... . Here's a broader commentary about libraries which don't have physical collections, including a public library in San Antonio - http://www.mnn.com/lifestyle/arts-culture/stories/is-a-libra... .
I doubt that small-town libraries will get rid of all books. Libraries do have collections of local history which are unlikely to be part of a digital archive. (Eg, phone books from the 1950s, local high school yearbooks, photo collections and films about the annual downtown parade, publications from the local genealogical society, etc.)
But if in 100 years small-town libraries have mostly switched to digital collections, then yes, it's possible for the knowledge to disappear even if the infrastructure remains.
I don't think it's likely, but it's not completely inconsistent.
However, for other scenarios, it is indeed a viable option, considering that it's statistically the safest (last I checked), very clean, and packs a high energy density.
The problem, though, is that fissile material is subject to a lot of the same availability problems as fossil fuels (if not worse), plus quite a few safety concerns (particularly when it comes to radiation, though coal mining poses this sort of risk, too). The energy density is much better, so we could probably get by for a very long time (especially with things like breeder reactors, increased use of thorium instead of or alongside uranium or plutonium, etc.), but ultimately, we'll need to transition to something like fusion power at the very least.