News for hackers. Stuff that matters.

A comment on Outsourcing plus LocalAI will soon become more economical vs. Frontier labs. Parent comment

Re:Outsourcing plus LocalAI will soon become more economical... by zozbot234 on Wednesday May 27, 2026 @07:02AM (#48292352)
Whether something is "impractical" depends on your expectations. High-latency unattended inference is definitely viable, even though it doesn't align much with what's being run in hyperscale datacenters.
[ Parent | on HN ]
Re:Outsourcing plus LocalAI will soon become more economical... by dns_snek on Wednesday May 27, 2026 @08:23AM (#48293174)
I'd like to meet the person who's been using a 1 token/second system as their primary LLM for at least a few weeks. Anyone?

I think 1 token/second is optimistic here - and even then it's over 11 days per million tokens.

[ Parent | on HN ]