Re:VectorDB: Vector Database Built by Kagi Search
by dmezzetti on Sunday November 26, 2023 @07:31AM
(#38421126)
What about models besides GPT? Most of the popular vector encoding models aren't using this architecture.
If you really didn't want PyTorch/Transformers, you could consider exporting your models to ONNX (https://github.com/microsoft/onnxruntime).
Re:VectorDB: Vector Database Built by Kagi Search
by cherryteastain on Sunday November 26, 2023 @07:41AM
(#38421175)
It's 60 lines for CPU only inference, which'll be slow. If you want GPU acceleration it'll be a lot more than 60 lines.
[0] https://jaykmody.com/blog/gpt-from-scratch/