News for hackers. Stuff that matters.
Re:Show HN: Stork: A customizable, WASM-powered full-text se... by t0astbread on Sunday December 27, 2020 @08:03PM (#25555743)
Just a quick non-thought-through idea but would it be possible to build an index in a way that allows clients to download only parts of it based on what they search? I.e. the search client normalizes the query in some way and then requests only the relevant index pages. The index would probably be a lot larger on the server but if disk space is not an issue...?

(Though at some point you have to ask yourself what the benefits of such an approach are compared to a search server.)

Re:Show HN: Stork: A customizable, WASM-powered full-text se... by lgierth on Sunday December 27, 2020 @11:12PM (#25556604)
Merkle Search Trees: Efficient State-Based CRDTs in Open Networks https://hal.inria.fr/hal-02303490/document

Peer-to-Peer Ordered Search Indexes https://0fps.net/2020/12/19/peer-to-peer-ordered-search-inde... (which adds useful context about the above)

Sorry for just dropping these links, I should already be asleep :)

Re:Show HN: Stork: A customizable, WASM-powered full-text se... by jil on Sunday December 27, 2020 @11:10PM (#25556597)
I keep wanting to try that! Sharding the index would be really cool, and would definitely allow for a seemingly infinite index size. The barrier, as you were getting to, would be making sure index shard downloads could keep up with typing speed.

Definitely something I'll try experimenting with in the future, so... watch this space, perhaps?