Re:Show HN: Data on Shopify Stores
by mritchie712 on Thursday February 04, 2021 @11:18AM
(#26026723)
how'd you get the list of shopify merchants to start with?
[ Parent
| on HN ]
We've extracted a list of IP addresses for shopify address (by checking whois on a relatively small number of stores we already know they're on Shopify) Then we've used some reverse whois service (like https://myip.ms) to find other domains that refer to the same IP. Those stores are definately on Shopify as their domain points to an IP address that belongs to Shopify.
This is really interesting. Thank you for the explanation.
I'd love to hear about how you crawl and store the data. Doesn't shopify block you as you are hitting there servers soo much for all the site?
Till now, No :)
Re:Show HN: Data on Shopify Stores
by armenarmen on Thursday February 04, 2021 @11:34AM
(#26026912)
Crawling all subdomain.myshopify.com I’d expect
Isn't this limited to the guestimated subdomain ?