Ask HN: Best tool for technical web crawler data (entire internet)

Re:Ask HN: Best tool for technical web crawler data (entire ... by mgliwka on Wednesday February 14, 2018 @09:29AM (#16375913)

Besides nerdydata there’s also the similar publicwww.com for custom searches.

Then there’s the usual suspects with predefined technology definitions (builtwith, similartech, datanyze).

Last but not least I’m also building such a product and will be launching next month.

I would love to discuss your needs and see if we can accommodate them - you can reach out to me at [email protected]

[ Parent | on HN ]

Re:Ask HN: Best tool for technical web crawler data (entire ... by veggies_4us on Wednesday February 14, 2018 @09:38AM (#16375979)

I have tried publicwww but over 80% of the results were either duplicates domains, dead links, or didn't contain my search term.

I'm glad I only wasted $49 and didn't opt for the annual license.

[ Parent | on HN ]