News for hackers. Stuff that matters.

A comment on Show HN: Till – Unblock and scale your web scrapers, with minimal code changes. Parent comment

Re:Show HN: Till – Unblock and scale your web scrapers, with... by Havoc on Wednesday August 04, 2021 @12:38PM (#28063085)
Why scrape at all if there are agreements in place. Seems like a API task
[ Parent | on HN ]
Re:Show HN: Till – Unblock and scale your web scrapers, with... by selcuka on Wednesday August 04, 2021 @07:55PM (#28067987)
> None of this gets around over-eager Cloudflare or Akamai rules set up years ago by some contractor that the businesses have no real ability to change.

If the business has no ability to even change the CloudFlare settings I don't expect them to be able to provide an API.

[ Parent | on HN ]
Re:Show HN: Till – Unblock and scale your web scrapers, with... by danpalmer on Friday August 06, 2021 @11:06AM (#28088161)
Only about 1 in 20 of our partners has the in-house tech necessary to provide us an API. In other words, we were able to accerelate our integrations by 20x by scraping.
[ Parent | on HN ]
Re:Show HN: Till – Unblock and scale your web scrapers, with... by hestefisk on Wednesday August 04, 2021 @08:21PM (#28068187)
In that case, the web site IS the API… only difference is data is wrapped in HTML not JSON.
[ Parent | on HN ]