I don't take notes because I read primarily for entertainment, in both fiction and non-fiction -- I have no need to recall in detail the contents of a book or to tie them into some overarching system, because I'm not an author or academic or even somebody who writes blog posts, and very little of what I read is related to my work. If I really need to I can always go back and dip into a book again if I want to look something up.