It is a public policy issue also outpaces "competition" which is merely a subject change.
Trying to prevent certain classes of behaviours via legal means is more like trying to prevent certain types of play, by appealing to the referee, while still doing them yourself. Clearly, this often does happen in sports, but _is_ generally seen as hypocritical.
There's NOTHING natural about our economic systems. They're all COMPLETELY made up, let's treat them that way.
(and yes, here it is about 'lobbying the ref')
Genetics however is not only a useful model, it's hard science. You can experimentally find out whether some characteristic is e.g. Mendelian (I'd doubt the greed is, as normally defined).
It got me thinking that to cross the two domains, there is also a meta-concept of cultural viruses ("memes") to which Dawkins applied Darwinian model. Definitely not hard science, but they kind of counter your point that "there's NOTHING natural about our economic systems".
Honestly, what helped me a lot is: Economic systems are more like video games than "nature." Sure, video games can look like nature, but also are extremely malleable.
In football, the rules have been extensively tuned to promote a fair fight.
Perhaps we should do a bit more of that sort of thing in corporate law.
that's the expected cost of publishing something to the public internet. People are going to access it. No one has a right to complain when people access something that was put there for the public to see. Scrappers can be dicks about it too, they can get lazy and endlessly hammer away at some server or repeatedly pull down the same content because they messed up, but we don't need need litigation for that. If something raises to the level of DoS that's already covered under existing laws.
> it's completely in keeping with what amoral corporations locked in constant battle would be expected to do: maximize benefits to themselves while minimizing benefits to others.
Maybe we need to rethink giving some of these corporations the privilege of corporate personhood if they are just going to make things worse for everyone else while only enriching themselves. We don't need to allow parasites and pillagers to take whatever they want at our expense.
It's not always about individual bad actors. You can have lots of small players causing problems. I wonder how many python developers there are right now trying to make their own offline copy of stackoverflow.com.
Wikipedia has a great defence against this. They ask you not to scrape, and at the same time, provide torrents of the data (https://meta.wikimedia.org/wiki/Data_dump_torrents)
There was a hiccup around June but that seems resolved now: https://meta.stackexchange.com/questions/389922/june-2023-da...
What is your opinion on spam email?
Some people don't publicly publish their email address, they instead selectively give it out only to those they want to get email from, but their address gets leaked/sold and abused. Ultimately people who do publicly publish a contact address (email or even a physical mailing address) are basically on the hook for deciding what to do with whatever people send them.
The spam situation got out of hand pretty fast though. The only thing that kept email spam from reaching the level of a DoS attack were blacklists and server-side filtering, and even with those things (plus client-side filters) spam is still a huge problem today. Spam is just a much bigger problem than web scraping. Even the junkmail the mailman delivers to my door has an environmental cost that's much worse than the "harm" of a web scraper's http GET requests.
We have many alternative ways to contact each other online that aren't as vulnerable to spam, but for all of its shortcomings email continues to be widely used because at the end of the day people think giving strangers the ability to reach out to them uninvited is valuable. Anyone can set up a whitelist and trash everything that comes into their mailbox unless it's from an approved sender, but almost nobody does because they want to be more reachable than that.
- it seem a lot of web developer / product manager either do not know or do not care about robots.txt
- some web application are so badly optimized, that some of them are not able to handle more than 1 hit per second at a sustained rate, which admittedly have worked fine so far. But crawlers are persistent, causing the normal crawling activity to cause denial of service for normal users.
You also would not let one football team buy up 60% of the other football teams in the world and merge them into one MEGA-TEAM.
But apparently we still cannot summon the willpower to enforce simple and straightforward 100+-year-old antitrust laws (so instead we make new ones and just won't enforce those either).
Why not? If that mega-team wants to destroy their revenue stream (People want to watch competitive football, as opposed to curb-stomps), they are free to do that.
The whole point of competitive sports is for there to be, well, meaningful competition. All the money in that industry only exists if there's meaningful chances for either team to win. Winning too hard is a detriment to the winner.
That is incredibly different from capitalism, where the whole point of it is for one of the participants to win the competition, to the deteriment of the rest of us, which is why society comes up with elaborate rules for preventing that.
Naturally, they're going to say "web scraping uses resources, stop it!" but then keep web scraping in the background.
To be clear, it's bad behavior, it's just not hypocritical behavior, as it's completely in keeping with what amoral corporations locked in constant battle would be expected to do: maximize benefits to themselves while minimizing benefits to others.