The curl'ing is not necessarily bad (and can continue just fine), but rather its blind to these types of problems. A false positive in your testing framework (especially something like this) is the worst case scenario.

Google wouldn't know that someone is curling it which their script said everything was ok. While the website was basically down because Chrome and Firefox will both block a site based on Google's safe browsing list.

They could use Google's safe browsing api to check if they're on that list as well as curl.

Frankly, Google's Safe Browsing list is one of those things which should be broken out of Google and ran by an independent entity, much like the Let's Encrypt model.

I don't know how to help make this happen.

I'd suggest:

1. Mozilla and the Chrome clones (Edge, Brave, etc.) partner to make an (open as possible) standard for blocking and reviewing, and start maintaining their own list upstream from Google.

2. When Google adds to their blocklist, independently check it according to the consortium's own standards.

3. Cut a deal with Bing or Yandex to scan for malware as part of their crawls, to get technology independence.

4. Put pressure on Google to get onboard.

Step 4 is the hardest.

I still don’t understand what the problem is here.” With Google doing this.

We don’t know that this whole thing is not invented by the author or that they were indeed not doing anything malicious.

All of this, as credible as it may seem, could just be invented bad PR against Google, which if true should make us take a hard look at whomever is behind this.

Seriously, read the Lessons Learned again and tell me for certain this really happened. How in the world would it look so staged, with the bold words and thought-out structure. I’m not going to jump on the bandwagon everytime someone makes claims.

My point is: How do we know what the sit author is saying is accurate?