If I disconnect and remove the part of my brain that holds my memory of my house, and walk in front of my house while holding that part of my brain in my hand as lumpy inert matter that's not understood/wired to the rest of me, we won't recognize the house either.

A notebook is swap space that has to be causally linked to you somehow (through your eyes, etc) to be useful just like a part of your brain has to be linked to you (generally through direct wiring).