pencildiver
Building a reinforcement learning system for e-commerce search
We kept renting GPUs after getting 7 quotes to buy them
We're switching to the V4 to store unified embeddings of our products. From the early tests we ran, this should help with edge case relevancy (i.e. when a product's image and text mismatch, thus creating a greater need for multi-modal embeddings) and improve our search speed by ~100ms.
We indexed 2.2M customer reviews from Shopify products
Show HN: I indexed 25,000 Shopify stores to build an embedded shopping API
https://developer.searchagora.com/#
Hi HN! I’m working on Agora, an AI search engine for e-commerce products.Excited to launch the Agora API that helps other developers embed shopping experiences into their app. We provide endpoints to search, purchase, and track products sold on 25,000 different Shopify and WooCommerce stores. Early customers are building social shopping apps, marketplaces, and AI shopping assistants.I’d love...
Show HN: I'm building an embedded shopping API
Search 200M Shopify products in 300ms
Guessing Vercel Deployment URLs
Show HN: I scraped 2.2M Shopify product reviews
We scraped 2.2M Shopify customer reviews
Great idea! We do have a concept of the "Agora Score" for products, to analyze more than just the customer reviews. We'll come up with something similar for merchants to help API customers know which merchants to prioritize.
>If you really want to be innovative try adding brick and mortar grocery stores too.
Absolutely. Just don't know how to efficiently crawl and recrawl their products, without partnering with them directly. Maybe plugging into something like the Instacart API to index local products.
We've been careful to abide by web scraping precedence. So all data we collect from Shopify and WooCommerce stores is publicly available. https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
We don't own or claim to own the copyright for product text and images. And make this clear with the store name, brand name, website, and more. That all said, we may be able to add a field on the Product Detail API that makes it clear who owns the copyright. https://docs.searchagora.com/api/endpoints/product-detail
>Which legal entity will end customers have a monetary relation with?
Using the Agora API, you become the merchant of record. Customers have a relationship with your app (i.e. Agora is hidden to the end customer). We are working on providing support for passing the payment to the end Shopify or WooCommerce store. We imagine having several payment flow options in the future: process payments on your own Stripe account, process payments with Agora, or pass through payments to the stores.
The Agora API enables developers to search, purchase, and track orders for 10 million products sold on 25,000 Shopify and WooCommerce stores. You can search by text, image, URL, or location to find products.
Every Shopify store has several public JSON files that are available on the same URL route. The file for general store information appears at [Base URL]/meta.json. Product information appears at [Base URL]/products.json. Variant and inventory information appears at [Base URL]/[Product Route].js.
Every WooCommerce store has a similar file that has a different data structure. The store information appears at [Base URL]/wp-json/. Product information appears at [Base URL]/wp-json/wc/v1/products.
I bought a list of e-commerce stores and built a crawler to index the store and product information. The crawler runs every day to add, edit, and delete product information for all stores. When you retrieve product details via the API, we dynamically check the price, stock, and variant information. This ensures that we only sell products that are currently available, at the real-time price.
We're starting with 25,000 Shopify and WooCommerce stores and will soon have products from other e-commerce platforms and marketplaces including Amazon, Walmart, Etsy, Wix, Squarespace, and more.
I'd love your feedback!
Show HN: I indexed 10M Shopify products to build an API
Show HN: I scraped 10M Shopify products to build an API
https://developer.searchagora.com/
Hi HN! I'm Param, founder of Agora.The Agora API enables developers to search, purchase, and track orders for 10 million products sold on 25,000 Shopify and WooCommerce stores. You can search by text, image, URL, or location to find products.Every Shopify store has several public JSON files that are available on the same URL route. The file for general store information appears at [Base...
Show HN: Search engine that helps Shopify stores sell more
https://merchant.searchagora.com/
Hi HN! Agora is a search engine for e-commerce products. We've indexed 4 million products sold on 8,100 Shopify and WooCommerce stores and made them easy to purchase with a single cart transaction. Think of a decentralized Amazon.I originally launched Agora as a side project here on HN to find a pair of red shoes for my wife last Christmas. I quickly realized that some of the best products on...
I just don't understand what the actual product is? Is it a web app? Also, how do you solve the empty state to build the community? Seems like a big part of the value proposition.
Show HN: I used AI to build holiday gift lists
Show HN: I built a simple alternative to ProductHunt
https://showcasemywebsite.com/
Hi HN! I built a side project that is an alternative to ProductHunt called Showcase My Website. To showcase your website on our home page, pay any amount higher than the last person.After you submit, we use Puppeteer to grab a screenshot of your website, store the image in Mongo, and then display it on the home page. The tough part was getting this all to happen within a couple seconds of the...
Show HN: I built an AI shopping assistant with 200M Shopify products
Hi HN! I previously launched Agora, a search engine for e-commerce products, here on HN: https://news.ycombinator.com/item?id=38635695Sticking to the Ancient Greek theme we just released Athena, an AI assistant that helps users find products in a personalized way. Athena gathers context of what you're looking for, asks specific questions about the product category and budget, and then presents...
Generally, Google Shopping shows big retailers or ads. E-commerce stores have a tough time competing against these big retailers, on both Google Shopping and a normal Google Search.
That all said, we are running tests to track engagement of a "search result page" indexed by Google or Bing. For example, searching for a "backpack" and then landing on the below link with selection to choose from.
For users (demand-side), the problem we solve: There are user groups that currently have a fragmented experience. For example, a specialized solar technician (just throwing out a random example) has to look through a handful of speciality stores to find and compare products that are only sold there. I think there are specific user-groups we can go after that really feel the pain right now of this process. Additionally, as the number of e-commerce platforms increases, it becomes tougher for every day users to find products they are looking for. They have to either go to Amazon or go store-by-store to discover products. The shop.app solves it for Shopify store but there's also millions of sites on WooCommerce, Squarespace, Wix, etc. We get around the empty-state problem with the crawler and now have merchants signing up to get their products indexed.
For merchants (supply-side), the problem we solve: If they sell on their own website, they have to compete against non-product pages on Google. For example, if you sell "red shoes" on your own site, you have to compete against the IMDB entry for the movie "Red Shoes" for people to find you. Additionally, if they sell on their own website and use Amazon (or any physical retailer) for distribution, they give up a percent of their margin. This increases your sell-through but is a smaller amount of money in your pocket.
I'll note that I've seen this problem first hand. In 2016 I launched a game called The 2016 Election Game, which was like Cards Against Humanity for the 2016 US elections. Sold about 5k units fulfilling order myself. And then again in 2020 called DoneWith2020, which was like Cards Against Humanity for the absurdity of the 2020 year. Sold about 34k units using mostly dropshipping. I remember losing out on search / discovery by choosing to sell on my own store but made a much higher margin on each sale (i.e. made about $15 on each $24.99 unit sold). We did work with a company to get on Amazon but always preferred people purchasing on our own site. It was also really hard to get high intent traffic to my store from ads. Would have been nice to send people searching for "funny card game" to my site. Now if everyone has my same dark sense of humor once they landed on the site, is up for debate.
The goal isn't to catch up to Shopify, WooCommerce, etc. but to rather aggregate products across platforms. I do think we can index most of e-commerce products sold on these platforms (my best guess is that it's somewhere between 10 - 20 billion products). This is obviously a very tough data hosting and search problem at that scale. Even Mongo, which is what we use as our primary database, has a limit of 2 billion records.
I agree that it's a traffic problem. Everything comes down to getting users. Based on the number of merchants signing up, we are validating that others have the supply-side problem. It's a matter of nailing down the demand-side problem (i.e. finding the right user groups, building the right features for engagement, etc). We use 'search' as the conduit, assuming that exceptional search will lead to more traffic. But agreed that there are several other factors to solve.
So, Shopify does have something similar called the shop.app which is only for Shopify stores. My best guess is that the e-commerce platforms are solving a different problem: store creation. I'll also note that each individual e-commerce platform isn't incentivized to aggregate and send users to stores not built with their platform. When we launched in December, we only had Shopify stores. Now we have Shopify and WooCommerce, and working on support for Squarespace and Wix sites as well.
For user traffic, the primary strategy is to build features with a viral loop back to the product. I mentioned this in another comment but we have have the concept of making shareable 'lists' of products you like. This is already working for us, at a small scale.
The general plan is to aggregate as many e-commerce products as possible from different platforms, keep improving the search experience, add automated filters to ensure high quality products, and then keep layering in features that drive a viral loop back to Agora.
We have run tests with image-detection to try to categorize products. We currently do search based on the name, description, price, store, and brand.
The problem with image-detection is cost. Given the size of our data set, it's very costly to run 800m - 1b images through a model (i.e. most products have 4 - 5 images). We've considered only doing the first 'hero' image to start though. Open to any cost-effective ideas though.
For example, if you search for "wooden chair", it would be nice to select a filter for 'category' to narrow down if I want to see "office furniture", "dining room", or "art".
https://www.searchagora.com/search?query=wooden+chair&count=...