News for hackers. Stuff that matters.

A comment on Show HN: I scraped 25M Shopify products to build a search engine. Parent comment

Re:Show HN: I scraped 25M Shopify products to build a search... by capableweb on Wednesday December 13, 2023 @07:53PM (#38636419)
Seems it might be looking at the referrer. Loading https://www.shopify.com/robots.txt from clicking the link shows the weird line while opening it in a private browser window shows the right one.
[ Parent | on HN ]

There are no comments on this comment yet.