Llama.cpp guide – Running LLMs locally on any hardware, from scratch

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by chatmasta on Friday November 29, 2024 @05:30PM (#42277697)

Be careful running this on work machines – it will get flagged by Crowdstrike Falcon and probably other EDR tools. In my case the first time I tried it, I just saw “Killed” and then got a DM from SecOps within two minutes.

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by broknbottle on Friday November 29, 2024 @11:34PM (#42279459)

the irony, preventing and killing something that is actually useful, while we let crowdcrap hum along consuming tons of memory and bottlenecking IO so it can do snakeoil things...

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by yjftsjthsd-h on Saturday November 30, 2024 @12:13AM (#42279670)

Are they specifically flagging LLMs, or do they not like Cosmopolitan Libc / APE?

[ Parent | on HN ]

Re:Llama.cpp guide – Running LLMs locally on any hardware, f... by chatmasta on Saturday November 30, 2024 @12:59AM (#42279875)

Nah nothing to do with LLMs, it’s just because the method of Llamafile is very similar to malware - basically zip up an executable, concatenate it with some stuff, throw it in /tmp and execute it with a randomly generated high entropy name.

(That said, after I explained it to SecOps they did tell me I would need to “consult legal” if I wanted to use a local LLM, but I’ll give them the benefit of the doubt there…)

[ Parent | on HN ]