News for hackers. Stuff that matters.

A comment on Show HN: NNext.net – A Firebase-like managed vector storage for ML applications.

Re:Show HN: NNext.net – A Firebase-like managed vector stora... by KhoomeiK on Wednesday March 03, 2021 @12:04AM (#26324971)
What I've done for most of my research projects is just pickle a PyTorch dataset object that contains all my embeddings. The pkl file can then just be uploaded anywhere and becomes plug and play with any Torch model.

What advantages would this bring for a user like me? I guess it might make more sense for people working closer to production?

[ Parent | on HN ]
Re:Show HN: NNext.net – A Firebase-like managed vector stora... by wakahiu on Wednesday March 03, 2021 @12:11AM (#26325015)
Exactly, once you have latency-sensitive applications that read-write embeddings, then you have to rethink your vector storage system. Furthermore, when your dataset grows to millions of datapoints, having a system that scales according avoids downtime and costly query lookup operations.
[ Parent | on HN ]
Re:Show HN: NNext.net – A Firebase-like managed vector stora... by wakahiu on Wednesday March 03, 2021 @12:19AM (#26325067)
How would you handle an online ML application where the set of embeddings is changing such as an image recognition app where images are constantly being added and need to be deduplicated?
[ Parent | on HN ]