News for hackers. Stuff that matters.

samosx

Infinity embedding engine added to KubeAI

Just merged and released the [Infinity support PR](https://github.com/substratusai/kubeai/pull/197) in KubeAI, adding Infinity as an embedding engine. So you can get embeddings on your local Kubernetes clusters with an OpenAI compatible API.Infinity is a high performance and low latency embeddings engine: https://github.com/michaelfeil/infinity KubeAI is a Kubernetes Operator for running OSS ML...

( Read More... | no comments | on HN )
Re:Estimating required GPU memory for serving LLMs on Sunday November 19, 2023 @12:03PM (#38334664)
Having a hard time with estimating how much GPU memory that LLM needs to serve it? What kind of GPUs to use and how many?

Wrote a blog post to demystify the process of GPU memory usage estimating.

Re:Show HN: Kubectl Notebook on Friday August 25, 2023 @01:07AM (#37258284)
Thanks for the feedback! All resources are namespaced right now, except there is an issue the notebook plugin where namespaces are indeed broken: https://github.com/substratusai/substratus/issues/193 Will get that fixed very soon.
Very simple prototype that uses stackoverflow API to get questions and chatGPT to answer them and store the results in postgres or sqlite using Peewee ORM.
Re:Full Text and Vector Search for Firestore with Weaviate on Sunday November 06, 2022 @02:16PM (#33495997)
OP here. You're right the solution is based on the existing documented full text search solutions. You can technically only sent the data you need indexed and the ID instead of sending all data. There is no native mechanism and firebase functions makes it as seamless as possible.
Re:Open Source Free Website Speed Monitoring – Websu on Monday February 28, 2022 @04:34PM (#30505441)
Feedback welcome, what feature would you want so this tool is helpful for you?