The first USB sticks came on the market in 2000, so I very much doubt you stopped using them before the Internet showed up.
Not only do I divorce the OS from the hardware, but I separate the OS from my personal data. Unless I have both wholly residing in RAM (which I find generally faster than HDD or USB/SD), they are not on the same media. As much as possible I try to make the OS read-only and the media used for data (e.g. RAM for short-term data storage or HDD for long-term data storage) read-write. Perhaps there are parallels to certain object file formats and their separation of code and data.
I've also thought a lot about and experimented with the use of separate, simplified, fast-booting OS's for different purposes. I am forced to use different OS's to perform certain tasks. But the popular all-purpose OS's are huge and often I only need a small fraction of their functionality.
What if we did not think in all or nothing terms about OS functionality? What if some OS's were small and only limited-purpose?
Rebooting is the slow step. But these limited-purpose OS's, being small and simple, can be very robust and responsive once booted.
The smaller the kernel and userland, the more RAM I have for storage.
Yes, agree with Mr Binary - in a perfect world, computer hardware and software are completely divorced. The hotel wouldn't worry about viruses, because there would be no storage on their computer at all - I provide that with my stick. All I need is a 'kiosk computer', hardware that I can repurpose.