It has nothing to do with greedy capitalists. I don’t want anyone scraping my site at all. I don’t charge anything for it.

Let's take as an example a website which compares prices in different stores and shows which has the cheapest price. You can do it manually, but using automation it is faster and more convenient. Is doesn't make sense to read websites manually when you can use a script or a language model.

Obviously for consumer it is better to be able to scrape sites. It is only those store owners (greedy capitalists) who do not want consumer to know that their prices are inflated.

Another thing is looking for some information, it is better just to have a language model go around the web and summarize the data for you rather than read someone's site with white letters on black background and weird font.

It's not uncommon that companies ask other companies to scrape their site as they don't have the tech resources to build an API / integration for whatever they want to have.

if u run a small site and ppl scrape it aggresively that can rack up ur bills.depending on where u host ur site. ofcourse not an issue for billion dollar companies, but a line needs to be drawn somewhere. also, whats the purpose of the scraping? usually its greedy capatalist purposes, so.then ur point is a bit moot, dont u think?

Let's say someone makes a site that compares prices in different stores. It needs a lot of scraping, but is useful for consumers. Obviously, the only ones who are against it will be the store owners.

Unless you go full blown DoS you will not be able to scrap 100.000 of articles multiple times a day. Geizhals for example compares prices every 10 minutes. It does this by working with the stores (they provide a price list) not against them.

If a store owner doesn't want the reach, it's their loss. IMHO no need for a DoS attack.

And then someone else makes another site that also compares prices in different stores. Instead of scraping all the stores they can just scrape the first comparison site and invest the time saved into SEO :)

Sites have a right to try to block scrapers. Scraping can incur significant costs and users may not want it - as a LinkedIn user, I want my profile to be indexed and able to viewed without logging in from the Web, but I also don’t want my entire profile scraped and resold/rehosted for marketing. It’s also often the case that the website put in significant working acquiring/curating/cleaning their data, and they are being scraped by other commercial entities trying to just resell it (blocking that isn’t being greedy, the data is basically being stolen for other commercial entities). So it’s not just being greedy.

Individuals also, IMO, have a right to sell access to their network for scraping-via-proxy. But they should be prepared to deal with the consequences, like a potential IP ban. Most people using VPNs that resell their residential network for scraping probably don’t know that’s happening, and many scrapers are indeed doing something bad, which is why there is a disdain for the practice.

I guess secret shoppers are illegal too.

Secret shoppers are paid by the company in question. How is that in any way relevant?

Not only did I not say anything about legality, I did not even say scraping is “bad”, just that it often is.

Not to worry linkedin already packages and sells your data no scrapers needed.

The difference is why else would I use LinkedIn except for my job data to be available for recruiters and employers?

Well part of that is just the direct value prop of the platform, by having a profile there I’m creating a channel for recruiters and stuff and I suppose making it easier to get bank loans/have people verify my identity. What I don’t want is for the data to be hosted elsewhere and resold to every Tom, Dick, and Harry. Fortunately LinkedIn also wants to prevent that because it undercuts their monetization schemes.