News for hackers. Stuff that matters.
Re:Show HN: Till – Unblock and scale your web scrapers, with... by capableweb on Wednesday August 04, 2021 @08:49AM (#28060384)
Of course if there is another, unlimited way of getting the data, then it's perfectly fine to "redirect" people to another avenue.

What this conversation is about (since we're talking about web scrapers here, not "data downloaders" or whatever you would call it) is when there is no other avenue to get the data, you should be able to access the same data via a machine as you could when you're a person.

Re:Show HN: Till – Unblock and scale your web scrapers, with... by Doctor_Fegg on Wednesday August 04, 2021 @08:59AM (#28060494)
Sure, but Till/DataHen appears to have a host of measures in place to ignore that "redirect".
Re:Show HN: Till – Unblock and scale your web scrapers, with... by capableweb on Wednesday August 04, 2021 @09:07AM (#28060567)
Yes, how are they supposed to know if the "redirect" is good or not? Facebook would helpfully redirect you to their Terms and Conditions, something you definitely should be able to ignore and scrape your data til your heart is content anyways.

Similarly to HTTP, the tool does not decide if usage is "nice" or "evil", only the user with their use case decides that.

Re:Show HN: Till – Unblock and scale your web scrapers, with... by tyingq on Wednesday August 04, 2021 @09:58AM (#28061065)
>you should be able to access the same data via a machine as you could when you're a person

I generally agree with this, but I do see the problem for certain spaces.

Concert tickets, for example. People write scrapers to get the best seats and sit on them so they can scalp them later at inflated prices. Or other first-come/first-serve situations, online auction "sniping", etc.

Re:Show HN: Till – Unblock and scale your web scrapers, with... by capableweb on Wednesday August 04, 2021 @10:07AM (#28061165)
Same thing applies here as I said in another comment (https://news.ycombinator.com/item?id=28060567), some people (me included) also write scrapers for getting the best seats in concerts/shows as I'm a big fan of the artist and really want to go there with good seats.
Re:Show HN: Till – Unblock and scale your web scrapers, with... by osmarks on Wednesday August 04, 2021 @03:41PM (#28065298)
Perhaps it would be better to not gate by whether you happen to have visited a website fast enough relative to other people. For auctions, at least, there is a simple solution in the form of extending the auction timer whenever someone bids.