News for hackers. Stuff that matters.

A comment on VectorDB: Vector Database Built by Kagi Search. Parent comment

Re:VectorDB: Vector Database Built by Kagi Search by dmezzetti on Sunday November 26, 2023 @06:52AM (#38420935)
My assumption is that this gives you the ability to locally encode vectors. This is useful for those not using API services to build their vectors.
[ Parent | on HN ]
Re:VectorDB: Vector Database Built by Kagi Search by andy99 on Sunday November 26, 2023 @07:25AM (#38421096)
Transformer inference is ~60 lines of numpy[0] (closer to 500 once you add tokenization etc). It would be nice to just have this and not all of pytorch and transformers.

[0] https://jaykmody.com/blog/gpt-from-scratch/

[ Parent | on HN ]
Re:VectorDB: Vector Database Built by Kagi Search by dmezzetti on Sunday November 26, 2023 @07:31AM (#38421126)
What about models besides GPT? Most of the popular vector encoding models aren't using this architecture.

If you really didn't want PyTorch/Transformers, you could consider exporting your models to ONNX (https://github.com/microsoft/onnxruntime).

[ Parent | on HN ]
Re:VectorDB: Vector Database Built by Kagi Search by cherryteastain on Sunday November 26, 2023 @07:41AM (#38421175)
It's 60 lines for CPU only inference, which'll be slow. If you want GPU acceleration it'll be a lot more than 60 lines.
[ Parent | on HN ]