Show HN: Open-source real-time talk-to-AI wearable device for few $

Re:Show HN: Open-source real-time talk-to-AI wearable device... by danielbln on Saturday October 05, 2024 @02:53PM (#41752042)

Any plans for being able to run the entire thing locally with local models?

[ Parent | on HN ]

Re:Show HN: Open-source real-time talk-to-AI wearable device... by _joel on Saturday October 05, 2024 @03:09PM (#41752156)

You can make an OpenAI compatible local server using LMStudio[1] and load any model you want. It'd have to be on another host though, the s3 has some inference capabilities with addons afair, but nowhere enough grunt to run locally at any usable token/s

[1] https://lmstudio.ai/

[ Parent | on HN ]

Re:Show HN: Open-source real-time talk-to-AI wearable device... by homarp on Sunday October 06, 2024 @08:19AM (#41756623)

or with open source tools like

llama.cpp - https://github.com/ggerganov/llama.cpp/blob/master/examples/...

mistral.rs - https://github.com/EricLBuehler/mistral.rs/blob/master/docs/...

lmstudio and ollama use llama.cpp underneath. cut the middle man

[ Parent | on HN ]

Re:Show HN: Open-source real-time talk-to-AI wearable device... by zq2240 on Saturday October 05, 2024 @08:43PM (#41753895)

Thanks for asking. We will launch local LLM, STT, and TTS models in the near future versions.

[ Parent | on HN ]