M4 MacBook Pro - Hackdot

Re:M4 MacBook Pro by mrcwinn on Wednesday October 30, 2024 @12:43PM (#41997249)

Question without judgement: why would I want to run LLM locally? Say I'm building a SaaS app and connecting to Anthropic using the `ai` package. Would I want to cut over to ollama+something for local dev?

[ Parent | on HN ]

Re:M4 MacBook Pro by andrewmunsell on Wednesday October 30, 2024 @12:45PM (#41997276)

Data privacy-- some stuff, like all my personal notes I use with a RAG system, just don't need to be sent to some cloud provider to be data mined and/or have AI trained on them

[ Parent | on HN ]

Re:M4 MacBook Pro by gonesurfing on Thursday October 31, 2024 @06:50AM (#42005477)

I’ve been thinking about the local notes use case, but know zero about doing it. Do you have a good setup you could point me at?

[ Parent | on HN ]

Re:M4 MacBook Pro by jwitthuhn on Wednesday October 30, 2024 @01:35PM (#41997959)

For me it is consistency. I control the model and the software so I know a local LLM will remain exactly the same until I want to change it.

It also avoids the trouble of using a hosted LLM that decides to double their price overnight, costs are very predictable.

[ Parent | on HN ]

Re:M4 MacBook Pro by satvikpendem on Wednesday October 30, 2024 @10:50PM (#42002986)

Lack of censorship.

[ Parent | on HN ]