News for hackers. Stuff that matters.

A comment on Run LLMs at home, BitTorrent‑style. Parent comment

Re:Run LLMs at home, BitTorrent‑style by judge2020 on Sunday September 17, 2023 @06:37PM (#37550011)
It can cost a lot to run a GPU, especially at full load. The 4090 stock pulls 500 watts of power under full load[0], which is 12 kWh/day or just under 4380 kWh a year, or over $450 in a year assuming $0.10-$0.11/kWh for average residential rates. The only variable is whether or not training requires the same power draw as hitting it with furmark.

0: https://youtu.be/j9vC9NBL8zo?t=983

[ Parent | on HN ]
Re:Run LLMs at home, BitTorrent‑style by pavelstoev on Sunday September 17, 2023 @07:36PM (#37550454)
Imagine someone paid you 25c/hour for 4090 compute sharing.
[ Parent | on HN ]
Re:Run LLMs at home, BitTorrent‑style by judge2020 on Sunday September 17, 2023 @11:07PM (#37551760)
That's pretty much what Nicehash does, but after you pay for that electricity it isn't super profitable - especially if you use it for 1/3 or more of the day for your own purposes (gaming/etc).
[ Parent | on HN ]
Re:Run LLMs at home, BitTorrent‑style by namtab00 on Sunday September 17, 2023 @08:15PM (#37550731)
> $0.10-$0.11/kWh for average residential rates

you Americans don't know how good you have it...

[ Parent | on HN ]
Re:Run LLMs at home, BitTorrent‑style by throwaway20222 on Sunday September 17, 2023 @08:44PM (#37550906)
That’s a cheap rate for sure. Southern California is $.36/.59/.74 peak. Super expensive.
[ Parent | on HN ]
Re:Run LLMs at home, BitTorrent‑style by teaearlgraycold on Sunday September 17, 2023 @09:52PM (#37551297)
Southern California? Time to buy some solar panels!
[ Parent | on HN ]
Re:Run LLMs at home, BitTorrent‑style by judge2020 on Sunday September 17, 2023 @10:14PM (#37551445)
Only Cali and the most northeastern states seem to have these high rates. Every other continental state is under $0.14 https://www.eia.gov/electricity/state/
[ Parent | on HN ]