News for hackers. Stuff that matters.
Re:Web Scraping in 2016 by nmstoker on Tuesday August 23, 2016 @06:57PM (#12348444)
With a risk of stating the obvious, this is a double standard simply because there are two standards - one for Google and one for others. I can't speak for the poster you were replying to, but whilst I see it as logical self-interested behaviour by site owners, it still feels unfair.
Re:Web Scraping in 2016 by bendbro on Tuesday August 23, 2016 @07:11PM (#12348518)
There isn't: the function for this standard includes expected benefit as an input. Every standard has inputs, so that certainly isn't the quality for making something a double standard. The only remaining quality is how unfair it feels, so it would probably be better to just address that, since it is obviously the only thing you disagree about.
Re:Web Scraping in 2016 by lugg on Tuesday August 23, 2016 @11:12PM (#12349429)
With that logic decreased wages for women are not a double standard due to the potential for maternity leave affecting their output at work.

This is a double standard plain and simple, and a very dangerous one at that.

Re:Web Scraping in 2016 by sp527 on Wednesday August 24, 2016 @12:27AM (#12349674)
Your example is a case of discrimination, but the economic rationale is unquestionable. There is a tremendous upfront cost for new employees, who are not valuable contributors for some lengthy ramp up period and furthermore accrue experience over the course of employment. So the lifetime value curve for any given employee is typically skewed left.
Re:Web Scraping in 2016 by bendbro on Wednesday August 24, 2016 @03:37PM (#12354641)
I think I'm just being difficult.

My point was that when you call something a double standard, you're arguing two things of equal value have been judged differently under the same standard. But by acknowledging they've been judged differently, you're acknowledging that there is a judgement, a standard, that applies the same to both, and produces the results you object to. What you really object to is the fairness of the qualities checked by the standard.

Since the outcome of calling things that, vs calling them a double standard is the same, I think most people already know and have no trouble with this. My protests were worthless.

It could gain value if there were certain whitelisted judgable aspects (like expected value), and judgements that aren't based on things from the whitelist are considered outside the scope of a standard. Then, calling the standard unfair and calling it a double standard would have a different meaning (if only in some contrived way, since any aspect is just an argument away from the whitelist)

Re:Web Scraping in 2016 by lerpa on Wednesday August 24, 2016 @02:22AM (#12350097)
It's their site they can block whatever they want. The problem is the stupid far reaching conclusion that this is trespassing.

Even normal trespassing laws are way too overreaching (see how it is handled in the UK for a saner example) but now you have the amazing possibility of remote trespassing.

The fun part is that it's just a matter of someone hiding something that says you cannot access the site in a place that you have to access the site in order to read -- the ToS. Suing people over this is idiotic.

The real problem is the involvement of Govt, and this kind of absurdity regarding ToS, EULAs and so on, is something that has been going on for decades. If you have the money you can make Govt your personal watch dogs.

Re:Web Scraping in 2016 by mgalka on Tuesday August 23, 2016 @07:21PM (#12348568)
Whether or not it technically qualifies as a "double standard," in practice I don't see anything inherently unfair about it.

If a stranger enters my house without my permission, that's trespassing. But there's nothing unfair about letting in someone who I invite over.

Re:Web Scraping in 2016 by nercht12 on Tuesday August 23, 2016 @08:10PM (#12348780)
That's a terrible analogy. Your home is private, websites are not. The fact is that websites are posted online for all to see, so it's more like saying certain people at a park may take pictures while others are not allowed. That's unfair. If everyone could take pictures, it would be fair. Yes, someone with an old bright bulb camera might be annoying people, but nobody said "fair" meant all players would be nice or that having a "fair" policy would somehow be more beneficial to the website owner. It's not, that's why site owners are selective. So they have a double standard, but it's for their benefit, not that of the site visitors (be they human or bot).
Re:Web Scraping in 2016 by sjwright on Tuesday August 23, 2016 @11:24PM (#12349457)
How about the analogy of an art gallery disallowing photography? Is the gallery being hypocritical when they allow the local paper to take photos for publicity, or when they permit an archivist that has a known reputation to take photos for archival purposes?
Re:Web Scraping in 2016 by Grishnakh on Thursday August 25, 2016 @02:04PM (#12361096)
You can still deal with the old bright bulb cameras: you can have rules which apply to everyone. So you can make a rule at the park that pictures are allowed, but only without flash, or that only digital cameras are allowed, or only digital cameras with the fake-shutter noises turned off, etc. As long as the rule applies to everyone equally, it's fair, even if you think the rule is silly.

For websites, it's not fair to have different rules for Google than others. What would be fair is some kind of rule about how often visitors can visit, how much they're allowed to download, etc.

Personally, though, I think all this is total BS. Sites are open to the public, but they also serve the whims of their owners. If the site wants to prevent access to people from a certain IP range, that should be their right. If they don't want any scrapers, that should be their right too, or if they want to allow Google and not anyone else, that should also be their right. What isn't right is that they can use the government to enforce these arbitrary rules. If they want to block my scraper, that's fine, if they can do it on their end technologically. If they want to block my IP, they can do that too. But suing me or having the cops come to my door because they're too incompetent or lazy to do these things technologically is unacceptable. The role of government is not to enforce arbitrary policies made up by business owners.

Re:Web Scraping in 2016 by gakada on Tuesday August 23, 2016 @08:00PM (#12348736)
Using the law to block crawlers is more like saying:

1. Google can come in

2. Other Americans can't come in

3. Chinese people can come in (or anywhere else where US laws don't apply)

It might not be unfair, but it is certainly pointless and arbitrary.

Re:Web Scraping in 2016 by jessedhillon on Tuesday August 23, 2016 @08:08PM (#12348773)
To be fair, many companies which take anti-scraping seriously will also take inputs like geographic origin of a request into consideration when applying request throttling and filtering.