But beyond the UX, and the stupidity of saying “private browsing is reserved for our logged in users”, think of the hypocrisy here: they want us to support freedom of the press and anonymous sources but don’t allow us to read anonymously.
It's not that they don't want people to read anonymously, it's that they don't want people to read without paying. If there were an easy and anonymous way to pay for reading publications like the NYTimes, I'd bet the NYT would support it.
Okay, okay ... I know NYTimes is a big corporation, and like any other, they love sucking up as much data they can about their users, so they probably won't do this. However, without an easy and anonymous payment system, NYTimes still has moral cover for their behavior. The bottom line is that people should pay for what they read and providers should not require more than that. There's plenty of technology that can handle this relationship, it's just that most people don't.
It should be possible to set up a lighter container with a browser whose data is completely wiped after the session competes for people who don't need or don't want to go all the way to use Qubes OS as their primate system.
Surely someone must have built this already? If not, it would be an interesting project.
Also, re: your browser idea - checkout the sibling comment to yours on using Chrome with a temporary data directory.