Having a couple notes about the things that stood out to me at the time is a great way to remind myself how I felt about books I’ve long since stopped thinking about.
On the flip side, I can enjoy a book more than once. This is a definite plus.
On the flip side, I have a firm belief that every book worth reading changes you at least a little, so even if I don't remember it consciously, it has had an impact on me.
I did try writing notes at some point, and stopped after the first attempt. That was several years ago and I never read those notes :)
If it’s not worth remembering, you won’t remember if you read it or not — so you could waste time rereading a forgettable book until you realize. ;)
Same with books. I've had plenty of times where I'd start reading a book and then a chapter in recall it. It's just nice to have a list.
If it's worth remembering, you'll remember you've read it, and if you want to refresh your memory what better than to reread?