News for hackers. Stuff that matters.

A comment on /story/40247270.

Re: by tiomat on Monday May 06, 2024 @03:02PM (#40278264)
Interesting how long it takes to get this “assistant” functionality on a local running models with llama for e.g.? It’s seems like many use cases require access to user data.
[ Parent | on HN ]
Re: by timm37 on Monday May 06, 2024 @05:12PM (#40279579)
local running would be impossible as you would need your computer running 24/7 (since the widget is available on your website which is always online).

But building a simple RAG/Vector DB on top of llama3 is not very complicated.

[ Parent | on HN ]