I mean, suppose you have a 100 books. You're not reading more than 2 or 3 of them at once, if that; and it's not like you're doing anything with the rest other than keeping them on a shelf.
Now, Once you either have a family, or have friends with similar interests, or with kinds of similar interests etc - those 98 (or 100) books become something you lend, or even give away. And when you get older, you will pass on your entire library, at some point, or sell it off, or bequeath it etc. At any of these points, you want the book to be free of your personal jots and notes, and as close to pristine condition as possible, for the other readers.
Write your notes in a notebook, or on your PC or laptop (or whatever). They'll also be searchable and easily editable.
While I generally have a communitarian orientation, I predict that as I approach this point, other end-of-life matters will occupy my headspace to a greater degree than any psychic rumblings about the odd marginalia in my dusty collection.
As an aside (because for the mere mortals among us it doesn’t apply) occasionally some historical inferences are made based on the reader’s margin notations.
A book you won't write in is a book that owns you, rather than you owning the book. A book is designed to convey information on the page, writing in your book only adds to and fulfills it's purpose.