Show HN: NNext.net – A Firebase-like managed vector storage for ML applications
As an ML engineer, I was frustrated by the lack of a datastore in which vectors are first-class citizens. As a result, most ML engineers, including myself, end up using awkward workarounds to store vectors such as arrays in SQL/NoSQL databases, stringifying vectors and storing them as text in in-memory-based caching systems such as Redis ETC. Furthermore, these systems don't allow for vector-based query operations such as nearest neighbor search. Consequently, engineers have to deploy additional approximate nearest neighbor search systems such as Facebook's FAISS or Spotify's ANNOY. These systems, while nifty and fast, are difficult to install and are costly to maintain. To address these issues, I built NNext, a managed vector datastore in which vectors are first class citizens. NNext, allows you to store vectors along with any json-blob metadata. Furthermore, NNext comes with a fast approximate nearest-neighbor (ANN) search capability.
I would love to get feedback on your experience as Data Scientist or ML engineer storing feature vectors and ANN systems. Please shoot me an email at [redacted].
Also check out this similar co I ran into: https://www.pinecone.io/ (the CEO, EL, has some classic sketches and feature hashing papers).
Depending on the topology of the data, different metrics are used and you can make different performance tradeoffs. And differing business use cases make it hard to find a one-size-fits-all. As I'm sure you're aware, there are dozens of fast ANN approaches, and the code implementing the ANN often is the tightest loop, making hard to be pluggable and performant.
TileDB is also quite interesting.
Best of luck though, I agree with you there's a "missing product" to be invented, I'm not sure this is exactly it, but I don't think I know any better.
Compliance is pretty obvious and even if you don't store the feature dictionaries and only the vectors, that's a hard conversation to have with the compliance team. I GET IT, without the feature dictionaries the vectors are useless, I KNOW this is how it works from a technical point of view, but the compliance team still won't sign off on it. That's just the way of the world.
And much much more important is performance, uncompressed high dimensional features are huge and even if you use run-length-encoding or sparse vector storage in the protocol plus some lossless compression, I have trouble keeping the GPU fed from disk, let alone over the network, it's going to be multiple orders of magnitude too slow. If the benefits is not claimed to be streaming but are fast vector similarity, keep in mind I can do cosine similarity on literally millions of vectors a second on a single CPU core using vanilla numpy, this was fast enough for me to implement realtime face recognition vector search for Dubai airport, so pretty high scale operations.
I've love a self hosted version of this, optimised for I/O throughput to the GPU. That would be great.
Compliance is hard to get around. There are products that have done this by sheer force of their cyber security budgets such as large cloud providers which are HIPAA compliant. As a result, companies that require thorough compliance measures are simply not our target audience.
Your second point about performance is critical to almost all of our target audience. First of all, there are self-hosted versions of this such as Facebook’s FAISS (has GPU acceleration), Spotify’s ANNOY, Elastic search KNN search and Milvus. We plan to optimize performance using the latest ANN algorithms, caching, being as close to the wire (Non-REST) transmission of vectors, dimensionally reduction e.t.c. At the end of the day, our stack will probably not be as fast as a self-hosted version but it’ll confer other advantages that we hope ML engineers will find useful similarly to AWS RDS or DynamoDB.
What advantages would this bring for a user like me? I guess it might make more sense for people working closer to production?
If that is nnext's only or main value prop, it will have to be changed.
Is your point that the market is simply too small/doesn’t exist or that the value prop is simply too weak / defensible?
If there is an OSS system that does everything you do, that isn't a big differentiator. As soon as you have any sort of traction, then the second competitor will swoop in, Lyft, Gitlab, Indiegogo.
You need to have defensible differentiation. Otherwise why not just run the OSS system on K8S and sell the service?
https://github.com/milvus-io/milvus