Re:Cost Per 1M tokens Of Self Hosting Llama-3
by jezzarax on Friday June 14, 2024 @04:42PM
(#40684905)
I checked some logs from my past experiments, the decoding went for about 400 tps over a ~3k token query, so about 7 seconds to process it, and then the generation speed was about 28 tokens.
[ Parent
| on HN ]
There are no comments on this comment yet.