Show HN: a desktop app to discover, download, and run offline LLMs

Re:Show HN: a desktop app to discover, download, and run off... by asselinpaul on Wednesday May 24, 2023 @03:55PM (#36063147)

This is great! CPU only for now or can it leverage GPU for sped up inference?

[ Parent | on HN ]

Re:Show HN: a desktop app to discover, download, and run off... by ybu on Wednesday May 24, 2023 @03:58PM (#36063174)

Focus is CPU for now - but it'd be very useful to have GPU support. As a baseline, this can support anything that llama.cpp. Relevant link: https://github.com/ggerganov/llama.cpp#blas-build

[ Parent | on HN ]