Re:Ask HN: Best tool for technical web crawler data (entire ...
by ghawkescs on Tuesday February 13, 2018 @07:06PM
(#16372155)
That sounds promising, can they run a custom report across their entire index? I'm trying to get a count of competing products to gauge market size and interest.
[ Parent
| on HN ]
Re:Ask HN: Best tool for technical web crawler data (entire ...
by veggies_4us on Wednesday February 14, 2018 @09:37AM
(#16375966)
They gave us the option of either running a report on their entire index, OR if we had a list of domains to search explicitly (which we didn't have).
Our use case was different, though, we wanted to find websites using a javascript library, and then run a javascript function on all of those sites to extract account information and specific html patterns.
But yes, their larger custom reports index did the trick, would recommend.