News for hackers. Stuff that matters.

A comment on Dot: use of local LLMs and RAG in particular to interact with documents.

Re:Dot: use of local LLMs and RAG in particular to interact ... by pentagrama on Sunday April 07, 2024 @11:46AM (#39961524)
Not sure if install the Windows GPU or CPU app version [1].

I have:

Processor: Ryzen 5 3600

Video card: Geforce GTX 1660 TI 6Gb DDR6 (Zotac)

RAM: 16Gb DDR4 2666mhz

Any recommendations?

[1] https://dotapp.uk/download.html

[ Parent | on HN ]
Re:Dot: use of local LLMs and RAG in particular to interact ... by alexpinel on Sunday April 07, 2024 @11:55AM (#39961567)
With those settings I would recommend GPU. CUDA acceleration really makes it faster, but keep in mind the CUDA toolkit 12.2 install will be a some 3-4gb
[ Parent | on HN ]
Re:Dot: use of local LLMs and RAG in particular to interact ... by pentagrama on Sunday April 07, 2024 @12:59PM (#39962057)
Thank you!
[ Parent | on HN ]