It's Time to Stop Taking Sam Altman at His Word

Re:It's Time to Stop Taking Sam Altman at His Word by henry2023 on Saturday October 05, 2024 @10:33AM (#41750154)

I’ve got a local llama 3.2 3B running on my macOS. I can query it for recipes, autocomplete obvious code (this is the only thing GitHub Copilot was useful for). And answer simple questions when provided with a little bit of context.

All with much lower latency than an HTTP request to a random place, knowing that my data can’t be used to trading anything, and it’s free.

It’s absolutely insane this is the real world now.

[ Parent | on HN ]

Re:It's Time to Stop Taking Sam Altman at His Word by nicolas_t on Saturday October 05, 2024 @12:13PM (#41750900)

What do you use to interface with llama for autocomplete? and what editor do you use?

Not wanting my data to be sent to random places is what has limited my use of tools like copilot (so I'd only use it very sparingly after thinking if sending the data would be a breach of nda or not)

[ Parent | on HN ]

Re:It's Time to Stop Taking Sam Altman at His Word by grahamj on Sunday October 06, 2024 @03:17PM (#41759445)

ollama + VSCode + Continue extension

[ Parent | on HN ]

Re:It's Time to Stop Taking Sam Altman at His Word by mark_l_watson on Saturday October 05, 2024 @12:40PM (#41751093)

+1 for sure running LLMs 3.2 3B is super fast and useful. I have been pushing it for local RAG and code completion also. I bought a 32B memory Mac six months ago, which I now regret because the small local models are now extremely useful and run fine on old 8B memory Macs, and support all the fun experiments I want to do.

[ Parent | on HN ]