Show HN: I scraped 25M Shopify products to build a search engine

Re:Show HN: I scraped 25M Shopify products to build a search... by ljm on Wednesday December 13, 2023 @08:31PM (#38636717)

2.2k/mo right off the bat is pretty steep, especially if you're paying that while the search response reliably takes over 10 seconds.

Why would you shovel 1.5k into MongoDB's pockets right off the bat? Especially when ElasticSearch is much better suited to what you're trying to do?

[ Parent | on HN ]

Re:Show HN: I scraped 25M Shopify products to build a search... by dinobones on Wednesday December 13, 2023 @09:57PM (#38637257)

You could run this entire stack (yes, even for 25 million products) using Kubernetes in a $40/month Linode + Elasticsearch + Cloudflare free plan.

[ Parent | on HN ]

Re:Show HN: I scraped 25M Shopify products to build a search... by altdataseller on Thursday December 14, 2023 @12:52AM (#38638250)

Sounds like someone drank the Mongo kool-aid. You absolutely do not need Mongo, let alone Mongo Atlas. 25 million documents with ecommeece products is measly and should fit in a single 600 GB server

[ Parent | on HN ]

Re:Show HN: I scraped 25M Shopify products to build a search... by ljm on Thursday December 14, 2023 @05:23PM (#38648170)

Probably not even that - 25mil is nothing really. A normalised schema in an RDBMS would handle that without sweating.

[ Parent | on HN ]