Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by emsign on Friday April 11, 2025 @05:25AM
(#43652018)
Running models at home seems like a waste of money while at the same time they are currently heavily subsidized in the cloud by dumb money.
[ Parent
| on HN ]
Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by chpatrick on Friday April 11, 2025 @06:50AM
(#43652467)
I use ChatGPT for most practical stuff but I really enjoy running local models. I find it really interesting and I think it's important for people to know how to run these without being beholden to big tech. If you have a used 3090 you can already run some really strong models. There are some really interesting local models as well, like the abliterated ones.
Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by borisk on Friday April 11, 2025 @05:57AM
(#43652208)
How is it a waste of money if a lot of people, including the author of the article, already have an nVidia GPU in their PC?
Running locally has a lot of advantages - privacy, getting to learn how to run LLMs, not having to deal with quotas, logins, outages.
Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by hu3 on Friday April 11, 2025 @08:19AM
(#43652969)
Home electricity bill of a decent GPU alone is usually more expensive than renting GPUs on demand.
Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by borisk on Friday April 11, 2025 @09:21AM
(#43653472)
Do you have any numbers to back that statement up?
E.g. I have solar panels and a home battery and pay less than a $100 a year for electricity.
Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by hu3 on Friday April 11, 2025 @11:47AM
(#43655188)
Solar panels aren't free. And aren't hassle-free either.
I wouldn't even bring that into the equasion since a large part of the population can't use solar panels.
Not to mention GPU value depreciation.
Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by sterlind on Friday April 11, 2025 @05:48AM
(#43652148)
it's nice to be able to have private conversations.
Re:Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
by calaphos on Friday April 11, 2025 @07:08AM
(#43652565)
There's also big efficiency increases when batching multiple requests, making clouds inherently more cost effective for normal use cases.
Way better utilization of expensive hardware as well ofc.