Re:Show HN: I built haystack – workplace search engine for devs
by bsima on Monday January 02, 2023 @09:28PM
(#34226789)
Very cool tech stack. How do you run t5-small in the browser?
Re:Show HN: I built haystack – workplace search engine for devs
by yuvalsteuer on Tuesday January 03, 2023 @01:06AM
(#34228300)
I load the 90mb model into memory from IndexDB, and then some rust code compiled down to WASM does calculations with the model.
Haystack runs entirely client sided in the browser, so it has a unique tech-stack:
Storage
using IndexDB, haystack stores user indexes locally, + a compressed 90mb NLP model (t5-small) is stored.
Indexing
Locally in the browser, using a t5-small bi-encoder, and some parsing of documents happens in wasm.
Search
Query converted to embedding, then searched over index, atlast results are reranked with a t5-small based cross encoder, and top results go through a seq-to-seq transformer to produce a nice consise textual answer.