Re:Ask HN: If you're burning tokens at scale, what are you u...
by akudha on Monday August 17, 2026 @03:33PM
(#49336427)
Interested in this kind of work. How did you get into this niche?
Re:Ask HN: If you're burning tokens at scale, what are you u...
by kingkongjaffa on Tuesday August 18, 2026 @09:36AM
(#49345455)
Is this through LLM providers on a metered API connection?
You're probably at the scale that having your own hardware to run local models is worth it, even if it's a couple of mac minis or mac pros in a cupboard in your office.
We use a routing approach where simpler tasks go to smaller, cheaper models and more complex tasks go to frontier models. We are also working on having our own model for some of these workloads.
Document processing is probably one of our biggest agent use cases for tokens at this scale.