Llama.cpp guide – Running LLMs locally on any hardware, from scratch

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by explorigin on Friday November 29, 2024 @01:44PM (#42276001)

Privacy, available offline, software that lasts as long as the hardware can run it.

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by JKCalhoun on Friday November 29, 2024 @02:32PM (#42276396)

Yeah, that's me. nCapture a snapshot of it, from time to time — so if it ever goes offline (or off the rails: requires a subscription, begins to serve up ads), you have the last "good" one locally.

I have a snapshot of Wikipedia as well (well, not the whole of Wikipedia, but 90GB worth).

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by 3eb7988a1663 on Friday November 29, 2024 @03:05PM (#42276671)

Which Wikipedia snapshot do you grab? I keep meaning to do this, but whenever I skim the Wikipedia downloads pages, they offer hundreds of different flavors without any immediate documentation as to what differentiates the products.

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by severine on Friday November 29, 2024 @05:14PM (#42277593)

You can use Kiwix: https://kiwix.org/en/

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by JKCalhoun on Saturday November 30, 2024 @06:34PM (#42284863)

wikipedia_en_all_maxi

I guess that means English ... and maxi? As I say, was something around 90GB or so.

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by 3eb7988a1663 on Sunday December 01, 2024 @07:07PM (#42291776)

Was hoping you had more insight than "maxi sounds good" which is the also the best I have.

[ Parent | on HN ]