Another bonus: Here in Germany a huge chunk of the hardware cost can be deducted from the tax bill.
Let the market decide.
Seems like enterprises and individual users are slowly switching to local models.
Sucks to be Member of Technical staff at OAI/ANT.
I know a total of zero and individuals or companies using local inference for their workhorse inference.
Local models are just not up to most tasks for now at least.
Haven’t token prices been falling by like 75% per year? I guess you’re saying that after IPO they will stop trying to be competitive with all the cloud providers that host open models?
1. privacy (w.r.t. personal data, human agency, IP, whatever) – this is absolutely is the most crucial.
2. if the needs are absolutely sufficed with local model(s) runnable on a super-expensive mac with a whole lot of RAM and the equivalent cost on API/subscription would be at least few times (say 2-3x) considering machine repair/maintainence/etc
There's a 3rd case thoguh – the hobbylist case, which, imho and in my anecdotal evidence, is the majority of local-model-chasing outside of enterprise (which is usually not hobbyist to begin with).
So there of couse are three legit use-cases.