Re:Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-G...
by Aerroon on Sunday February 22, 2026 @12:29AM
(#47108476)
Something to consider is that input tokens have a cost too. They are typically processed much faster than output tokens. If you have long conversations then input tokens will end up being a significant part of the cost.
It probably won't matter much here though.