News for hackers. Stuff that matters.
Re:Token growth indicates future AI spend per dev by skybrian on Monday August 11, 2025 @02:37PM (#44867785)
Open source models could be run by low-cost cloud providers, too. They could offer discounts for a long term contract and run it on dedicated hardware.
Re:Token growth indicates future AI spend per dev by qingcharles on Monday August 11, 2025 @02:50PM (#44867951)
This. Your local LLM, even if shared between a pool of devs, is probably only going to be working 8 hours a day. Better to use a cloud provider, especially if you can find a way to ensure data security, if that is an issue for you.
Re:Token growth indicates future AI spend per dev by wongarsu on Monday August 11, 2025 @03:01PM (#44868104)
Exactly. There is no shortage of providers hosting open source models with per-token pricing, with a variety of speeds and context sizes at different price points. Competition is strong and barriers of entry low, ensuring that margins stay low and prices fair.

If you want complete control over your data and don't trust anyone's assurances that they keep it private (and why should you) then you have to self-host. But if all you care about is a good price then the free market already provides that for open models

Re:Token growth indicates future AI spend per dev by hkt on Monday August 11, 2025 @04:20PM (#44868975)
Hetzner and Scaleway already do instances with GPUs so this kinda already exists
Re:Token growth indicates future AI spend per dev by hkt on Monday August 11, 2025 @04:24PM (#44869026)
In fact, does anybody want to hire a server with me? I suspect it'll work out cheaper than Claude max etc: a server from hetzner starts at £220ish: https://www.hetzner.com/dedicated-rootserver/matrix-gpu/

It might be fun to work out how to share, too. A whole new breed of shell hosting.

Re:Token growth indicates future AI spend per dev by benterix on Tuesday August 12, 2025 @08:26AM (#44875368)
I have a couple of non-GPU servers with them and quite a few Heztner Cloud projects, but I never understood their GPU offering. They have just two (small) VRAM sizes and you pay per month, whereas the ones like Runpod have a large selection of whatever you need, and they are cheaper, and you can rent them for a shorter period like two weeks, no setup time. Am I missing something?
Re:Token growth indicates future AI spend per dev by hoppp on Tuesday August 12, 2025 @09:46AM (#44876144)
There are use-case for smaller models that fit in 20gb VRAM, like in database sentiment analysis and such.
Re:Token growth indicates future AI spend per dev by benterix on Tuesday August 12, 2025 @10:01AM (#44876312)
Sure, my point you can get these cheaper outside of Hetzner, so I don't really understand who are these for.
Re:Token growth indicates future AI spend per dev by hoppp on Tuesday August 12, 2025 @09:42AM (#44876099)
Hetzer already offers dedicated GPU servers from 180eur/month