Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by Aerroon on Sunday February 22, 2026 @12:29AM
(#47108476)
Something to consider is that input tokens have a cost too. They are typically processed much faster than output tokens. If you have long conversations then input tokens will end up being a significant part of the cost.
It probably won't matter much here though.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by teo_zero on Sunday February 22, 2026 @01:33AM
(#47108777)
I think we use different units. In my system there are 3600 seconds per hour, and watts measure power.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by IsTom on Sunday February 22, 2026 @04:24AM
(#47109617)
OP probably means watt-hours.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by dotancohen on Sunday February 22, 2026 @09:45AM
(#47111411)
And 0.5 tokens/s should work out to 1800 tokens at the end of the hour. Not 3600 as stated.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by thatwasunusual on Sunday February 22, 2026 @05:18AM
(#47109857)
> Cost wise it does not seem very effective.
Why is this so damn important? Isn't it more important to end up with the best result?
I (in Norway) use a homelab with Ollama to generate a report every morning. It's slow, but it runs between 5-6 am, energy prices are at a low, and it doesn't matter if it takes 5 or 50 minutes.
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by xienze on Sunday February 22, 2026 @12:45PM
(#47112982)
> Why is this so damn important? Isn't it more important to end up with the best result?
You’re wondering why someone would prefer to get the same or better result in less time for less money?
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by qoez on Sunday February 22, 2026 @07:09AM
(#47110367)
Open router is highly subsidized. This might be cheaper in the long run once these companies shift to taking profits
Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by culopatin on Sunday February 22, 2026 @02:09PM
(#47113706)
But why not cross that bridge then. By that time you might have much more optimized local infrastructure. Although I do see that someone suffering through the local slowness now is what drives the development of these local options.
Great achievement for privacy inference nonetheless.