News for hackers. Stuff that matters.
Re:Show HN: Embeddinghub: A vector database built for Machin... by cyrusthegreat on Thursday September 16, 2021 @10:11AM (#28552035)
Hi everyone!

Over the years, I've found myself building hacky solutions to serve and manage my embeddings. I’m excited to share Embeddinghub, an open-source vector database for ML embeddings. It is built with four goals in mind:

Store embeddings durably and with high availability

Allow for approximate nearest neighbor operations

Enable other operations like partitioning, sub-indices, and averaging

Manage versioning, access control, and rollbacks painlessly

It's still in the early stages, and before we committed more dev time to it we wanted to get your feedback. Let us know what you think and what you'd like to see!

Repo: https://github.com/featureform/embeddinghub

Docs: https://docs.featureform.com/

What's an Embedding? The Definitive Guide to Embeddings: https://www.featureform.com/post/the-definitive-guide-to-emb...

Re:Show HN: Embeddinghub: A vector database built for Machin... by ypcx on Thursday September 16, 2021 @01:41PM (#28554717)
In the "Definitive Guide to Embeddings", in the figure "An illustration of One Hot Encoding", the "One Hot Encoding" table doesn't make any sense whatsoever. Am I wrong?
Re:Show HN: Embeddinghub: A vector database built for Machin... by make3 on Thursday September 16, 2021 @01:44PM (#28554765)
no you're right ahahah wth are these
Re:Show HN: Embeddinghub: A vector database built for Machin... by cyrusthegreat on Thursday September 16, 2021 @03:46PM (#28556491)
You are both right. I just realized this and would be embarrassed if I wasn’t laughing so hard. I gave an original drawing to our designer with the correct values and we didn’t inspect their final image. We’ll get this fixed, thanks for pointing this out and sorry for the confusion :)
Re:Show HN: Embeddinghub: A vector database built for Machin... by JPKab on Thursday September 16, 2021 @11:10AM (#28552794)
Holy shit, this looks amazing!

I see you've got examples for NLP use cases in your docs. Can't wait to read them. Embeddings are a constant source of complexity when I'm trying to move certain operations to Lambda, this looks like it would speed the initializations up big time.

Re:Show HN: Embeddinghub: A vector database built for Machin... by cyrusthegreat on Thursday September 16, 2021 @11:44AM (#28553276)
We're so glad to hear that! We'd love your feedback as we keep building. Please join our community on Slack: https://join.slack.com/t/featureform-community/shared_invite...
Re:Show HN: Embeddinghub: A vector database built for Machin... by localhost on Thursday September 16, 2021 @11:54AM (#28553392)
Curious about how your solution is different / better than nmslib which I've tried in the past?
Re:Show HN: Embeddinghub: A vector database built for Machin... by cyrusthegreat on Thursday September 16, 2021 @12:18PM (#28553683)
We actually use HNSWLIB by NMSLIB on the backend. NMSLIB is solving the approximate nearest neighbor problem, not the storage problem. It’s not a database, it’s an index. We handle everything needed to turn their index into a full fledged database with a data science workflow around it (versioning, monitoring, etc.)
Re:Show HN: Embeddinghub: A vector database built for Machin... by localhost on Thursday September 16, 2021 @05:09PM (#28557645)
That's great. I've been very impressed by the performance of nmslib in my scenarios. I'll definitely check out eh - thanks for sharing!