News for hackers. Stuff that matters.
Re:Show HN: Instantly search 2M recipes by jabo on Wednesday December 09, 2020 @04:30PM (#25365402)
Some quick context: I was inspired to build this by this HN post earlier today [1]. So thank you glorf for making the recipe dataset available.

Thought this would take me 1-2 hours to build, ended up taking about 6 hours - engineering estimates and all!

> Details about the Tech Stack:

The dataset has 2,231,142 recipes and is indexed on Typesense [2], an open source alternative to Algolia/ElasticSearch that a friend and I are working on.

The UI was built using the Typesense adapter for InstantSearch.js [3] and is a static site bundled using ParcelJS.

The app is hosted on S3, with CloudFront for a CDN.

The search backend is powered by a geo-distributed 3-node Typesense cluster running on Typesense Cloud [4], with nodes in Oregon, Frankfurt and Mumbai.

Here's the source code: https://github.com/typesense/showcase-recipe-search

[1] https://news.ycombinator.com/item?id=25356156

[2] https://github.com/typesense/typesense

[3] https://github.com/typesense/typesense-instantsearch-adapter

[4] https://cloud.typesense.org

Re:Show HN: Instantly search 2M recipes by graderjs on Thursday December 10, 2020 @01:10AM (#25370376)
Jason, sorry to ask here rather than read your GitHub docs, but how does Typesense fare against non-romance languages that can't be segmented by whitespace?
Re:Show HN: Instantly search 2M recipes by jabo on Thursday December 10, 2020 @11:05PM (#25382162)
I'm guessing you meant to say logographic languages. We don't yet support tokenization for logographic languages (like Chinese, Japanese, etc) but it's on our medium-term radar: https://github.com/typesense/typesense/issues/86
Re:Show HN: Instantly search 2M recipes by graderjs on Friday December 11, 2020 @02:30AM (#25383306)
Basically...but I didn't know to use that term! So thanks for teaching me. Then there's also languages like Thai that are not whitespace separated on the word level, but that use an alphabet. So...I meant more 'non-Latin' but I think that's not actually a tight category. It's actually quite difficult to come up with the right term. I guess I was trying to be too clever, the best term is probably "non-whitespace delimited languages". Thanks for your response, and awesome speed to index the dataset and have it up and running in the same day.

Could I ask you a few more questions? What was the dataset size? What was the size of your index? How long (and how much RAM) did it take to index the dataset and what machine (and how many cores) did you do it on?

Re:Show HN: Instantly search 2M recipes by jabo on Friday December 11, 2020 @07:31PM (#25393739)
> Then there's also languages like Thai that are not whitespace separated on the word level, but that use an alphabet.

I did not know that! Good to know.

> What was the dataset size?

2.2GB in size, with ~2.2M records

> What was the size of your index?

2.7GB

> How long (and how much RAM) did it take to index the dataset

It took about 8 minutes to index that data. Typesense stores the entire index in memory, so the index took 2.7GB in RAM

> What machine (and how many cores) did you do it on?

It's running on a 3-node cluster, with each node having 4vCPUs and 8GB of RAM. The nodes are distributed across data centers, so search requests are served by the closest node (like a CDN).

Re:Show HN: Instantly search 2M recipes by graderjs on Friday December 11, 2020 @09:58PM (#25394972)
That's great, thank you for that info! Very impressive performance specs for your indexing.
Re:Show HN: Instantly search 2M recipes by someone7x on Thursday December 10, 2020 @10:24AM (#25374636)
> non-romance languages that can't be segmented by whitespace

That can't be right? Surely Greek, Russian, Turkish, etc are whitespace delimited?

Re:Show HN: Instantly search 2M recipes by graderjs on Friday December 11, 2020 @02:25AM (#25383276)
Yeah I meant some concept like "non-Latin derived" or "non-Roman alphabet" languages but then there's Cyrillic, etc. I was pretty sure "non-Romance" sounded like that right term, but not totally sure. I looked it up after and yeah, it wasn't. Actually I have no deep idea of the terms in this and just grabbed the first term that came to me. I thought I did pretty well and I appreciate the learning experience!
Re:Show HN: Instantly search 2M recipes by bobthepanda on Thursday December 10, 2020 @01:25PM (#25376696)
Chinese and Japanese wouldn’t be.
Re:Show HN: Instantly search 2M recipes by amluto on Wednesday December 09, 2020 @11:39PM (#25369785)
You have a UI bug: dismissing the modal recipe popup isn’t entirely reliable, and the site can get stuck in a state that doesn’t allow user interaction. This even survives the back button.
Re:Show HN: Instantly search 2M recipes by jabo on Thursday December 10, 2020 @11:05PM (#25382163)
Hmm interesting, I can't seem to replicate this issue. What browser are you using?
Re:Show HN: Instantly search 2M recipes by amluto on Friday December 11, 2020 @08:15PM (#25394180)
iOS 14’s Safari. It only happened once.
Re:Show HN: Instantly search 2M recipes by the_arun on Wednesday December 09, 2020 @11:11PM (#25369608)
Very nicely done! Also, appreciate sharing due credit to dependent stories.
Re:Show HN: Instantly search 2M recipes by vincentmarle on Thursday December 10, 2020 @12:03AM (#25369947)
Thanks I was just looking for a cheap alternative for a search engine just today (Algolia is cool but very expensive if you need to index millions of records) - I will check out Typesense.
Re:Show HN: Instantly search 2M recipes by dawnerd on Thursday December 10, 2020 @03:14AM (#25371151)
Type sense is looking great! I was going to use it on a side project. Not sure why I didn’t but it must have been missing something I needed. Been using meilisearch but I’ll definitely be checking out typesense again.

Huge fan of instant search results, well done!

Re:Show HN: Instantly search 2M recipes by mariushn on Thursday December 10, 2020 @04:11PM (#25378750)
Hi! Would typesense be good for a general web page search system (algolia-like), or it's designed for structured entities only (products, recipes...)

Why did you built it instead of using other open source engines? eg postgres text search

Re:Show HN: Instantly search 2M recipes by jabo on Thursday December 10, 2020 @11:09PM (#25382180)
Typesense would indeed be good for a general web page search system, just like Algolia. In fact, even Algolia stores web page data as structured JSON entities.

re: why not just postgres text search, I'll post a more detailed response in the github issue you opened (thank you) for posterity: https://github.com/typesense/typesense/issues/167

Re:Show HN: Instantly search 2M recipes by mariushn on Friday December 11, 2020 @01:00AM (#25382796)
Thanks :)
Re:Show HN: Instantly search 2M recipes by ajitk on Thursday December 10, 2020 @12:15AM (#25370020)
This is refreshingly fast! Definitely going to try typesense in my projects.
Re:Show HN: Instantly search 2M recipes by jftuga on Wednesday December 09, 2020 @07:03PM (#25367438)
Did you use CloudFormation to crate the infra? If not, I'd love to hear some details on how you did this. Any API Gateway being used? Seems to be offline at the moment.
Re:Show HN: Instantly search 2M recipes by jabo on Wednesday December 09, 2020 @07:08PM (#25367503)
The front-end is a static site. I used terraform to setup the S3 bucket & Cloudfront.

The search backend is running on Typesense Cloud, which is point and click to provision.

This is my 1-line deployment command: https://github.com/typesense/showcase-recipe-search/blob/7b5...

That's it for the infra! No API Gateway.

Hmmm, seems to be up for me. Could you show me a console+browser screenshot of what you see?

Re:Show HN: Instantly search 2M recipes by blntechie on Thursday December 10, 2020 @05:17AM (#25372021)
We run similarly on CloudFormation. Interesting with the ‘aws s3 cp’ command. I started using ‘aws s3 sync —delete’ nowadays after having issues with pre cleanup required for ‘cp’.
Re:Show HN: Instantly search 2M recipes by jabo on Friday December 11, 2020 @07:34PM (#25393775)
In my case since the assets have fingerprinted filenames and index.html references them, if I delete old files with each deployment, then a user who has a cached version of index.html will see a broken page. So I just leave old asset files as is.
Re:Show HN: Instantly search 2M recipes by yawnxyz on Thursday December 10, 2020 @10:19AM (#25374571)
it boggles my mind that a two person team built Typesense...