nikhilgarg28

One of the authors of the post here. I've never heard of this approach but it sounds really cool. Will definitely try it out someday!

We hadn't considered that at all, will definitely check it out and explore further. Thanks!

Yes, it wasn't initially there mostly due to an oversight but we later edited the article.

What is the size and dimensionality of your corpus?

The article isn't meant to present a new library competing with Annoy or other state of the art approaches, but instead, describe how vector search works using short code snippets.

I've now updated the article to clearly mention this and also cite annoy.

We did some benchmarking and found that an index of 1M vectors using 15 trees can be built up in ~115 seconds (8 CPU, 8 GB RAM). https://github.com/fennel-ai/fann/pull/8.

Given this, around a second to build an index of 10K items sounds about right.

Apologies, that got omitted by mistake. I've now included the post to include that piece of code as well.

One of the authors of the post here. Happy to take any questions!

One of the post authors here. While building/writing this was fun, FWIW, we were really surprised by how fast the Rust version turned out to be with relatively little effort.

Fennel | Distributed System Engineer | Menlo Park (Bay Area) or Remote | Full time (https://fennel.ai/)

Fennel is an ML Infra company building realtime feature engineering platform. As part of this, we are building a streaming database with few ML specific layers of abstractions on top.

We are looking for a senior distributed system engineer. Preferable (but not necessary) to have prior experience in database internals and/or stream processing systems. Bonus if you have written Rust before.

Model deployment is an important but still tiny part of the overall ranking/recommendation systems. Bulk of the complexity stems from two key properties of recommendation systems (which are different from say computer vision models): 1. The system operates on user feedback. As a result, it needs to manage flow of lots of data, with at least some subset being managed in realtime.

2. For any single request, there are thousands of things to recommend from. As a result, a single request is not scoring a single ML model but thousands of models - one (or often more, see value modeling the post) for each candidate.

(Disclaimer: I'm the author of the post)

Good feedback, noted. Will get the next post focused on training within the next couple of days.

Curious - why did people not agree with you? Was it an ideological belief or were they betting on some assumptions that turned out to be incorrect?

That's very interesting. The estimate of 100ns came from here: https://people.eecs.berkeley.edu/~rcs/research/interactive_l.... and is probably not very precise (maybe because it is only capturing rough order of magnitude). I have now updated the post. Thanks for the feedback! Specific constant aside, the point about latency not improving much still holds.