But you have to balance these concerns with what's practical. After all, a search engine is supposed to help people find things.
If an NYT article actually answers your search query, I don't think the greater good is for Google to not show you that result. I don't think that's what most people would want, either.
I feel the same way about Google hypothetically only indexing the part of the article that is shown above the paywall fold. What if the rest of the article can answer your search query? I don't think you're better off not knowing such an article exists.
There are only trade-offs.
The other things that Google bans are more obviously deceptive with few upsides to be found for the end-user, like cloaking a malicious website. I don't think you could enumerate the other sorts of things that Google blocks and compare them honestly to NYT's paywall, but I'd be curious to hear you build that case.
I argue that the only reason people are interested in this article is because of Bill Gates opinion not the ny times.
I think search results for the original opinion would be much more relevant than a random persons opinion of the opinion. The opinion of some one else's opinion is not worth money, if ny times considers that as quality journalism I think it says a lot about the quality of their journalism.
Like a lot of things in life, you get what you pay for.