News for hackers. Stuff that matters.
Re:Ask HN: Best tool for technical web crawler data (entire ... by ghawkescs on Tuesday February 13, 2018 @07:06PM (#16372155)
That sounds promising, can they run a custom report across their entire index? I'm trying to get a count of competing products to gauge market size and interest.
Re:Ask HN: Best tool for technical web crawler data (entire ... by veggies_4us on Wednesday February 14, 2018 @09:37AM (#16375966)
They gave us the option of either running a report on their entire index, OR if we had a list of domains to search explicitly (which we didn't have).

Our use case was different, though, we wanted to find websites using a javascript library, and then run a javascript function on all of those sites to extract account information and specific html patterns.

But yes, their larger custom reports index did the trick, would recommend.

Re:Ask HN: Best tool for technical web crawler data (entire ... by ghawkescs on Wednesday February 14, 2018 @11:12AM (#16376839)
Great to know, thank you. Was the pricing reasonable?