In the past couple of years, many large tech companies developed a legal theory that scraping the public web for "internal" purposes is OK, and that any ToS-based or technical restrictions are just suggestions that they don't have to follow.

These "internal" purposes included growing your social network, monitoring or reverse-engineering the algorithms of competing search engines, and now, it includes training ML.

Which is funny given that when others are doing it to them, they go to great lengths to stop it, and sometimes complain loudly or threaten lawsuits.

I think the main reason the big players don't sue each other is that it's a bit of a Mutual Assured Destruction kind of a deal. Google is doing it to Microsoft, Microsoft is doing it to Google...

Technology patents create a complex MAD situation. At the end of the day Alphabet will be paying tons of money to Microsoft who will pay it to Apple who will pay it to Samsung who will pay it to probably Nokia or Xerox or Nintendo or whatever. And this goes on for each and every company with wealth in these spaces.

They (BigCo) traditionally cross-license to benefit each other and lock out new entrants.

and too late companies like Reddit and DeviantArt realized the billions of $ in UGC value that their websites had laying stagnant.