News for hackers. Stuff that matters.

A comment on Show HN: Instagram-scraper – Scrape instagram photos by tags, without API.

Re:Show HN: Instagram-scraper – Scrape instagram photos by t... by fapjacks on Sunday June 24, 2018 @04:41AM (#17385630)
Well, this is just a Python web scraper, and Instagram does in fact attempt to detect and prevent/rate-limit this kind of scraping. They rely very heavily on the source IP to help them determine when to cut you off.
[ Parent | on HN ]
Re:Show HN: Instagram-scraper – Scrape instagram photos by t... by donjoe on Sunday June 24, 2018 @04:48AM (#17385651)
... indeed. Isn't the script 'under the hood' calling the API anyways (looking at 'scrolldown' here)?

I reversed engineered the API myself a couple of weeks ago which was great fun - especially figuring out Instagram's rate limits on interactions such as comments and likes per day/hr.

[ Parent | on HN ]
Re:Show HN: Instagram-scraper – Scrape instagram photos by t... by meetmangukiya on Sunday June 24, 2018 @05:32AM (#17385774)
Not the Instagram developer APIs, but the one that instagram's frontend consumes. The script scrapes instagram's frontend here.
[ Parent | on HN ]
Re:Show HN: Instagram-scraper – Scrape instagram photos by t... by kall1sto on Monday June 25, 2018 @05:28AM (#17391021)
I put a timeout of 2-3 sec between each image download. Do you think this will prevent instagram from detecting the scraper?
[ Parent | on HN ]
Re:Show HN: Instagram-scraper – Scrape instagram photos by t... by meetmangukiya on Sunday June 24, 2018 @05:30AM (#17385768)
Interesting, I might reverse engineer and see if we can introduce some kind of backoff and then resume the scraping again.
[ Parent | on HN ]