News for hackers. Stuff that matters.
Re:Hetzner is working on LLM Inference by swiftcoder on Friday July 24, 2026 @07:30AM (#49034047)
It would certainly be interesting to have a highly respected EU-native inference provider, if only to make the regulatory gods happy
Re:Hetzner is working on LLM Inference by jonas_scholz on Friday July 24, 2026 @07:35AM (#49034102)
agree. I have a usecase for a bigger open-weight model hosted by an EU company and the selection isnt really great, hard to make the regulatory gods (and the devs) happy at the same time right now
Re:Hetzner is working on LLM Inference by sofixa on Friday July 24, 2026 @07:44AM (#49034191)
Scaleway have two separate (one fully managed one a bit less) services for that:

https://www.scaleway.com/en/generative-apis/

https://www.scaleway.com/en/inference/

Re:Hetzner is working on LLM Inference by jonas_scholz on Friday July 24, 2026 @07:45AM (#49034210)
yea, but no prompt caching right? This makes it unusable for my usecase at least, the cost would be insane
Re:Hetzner is working on LLM Inference by PaoloBarbolini on Friday July 24, 2026 @11:28AM (#49037103)
They confirmed on LinkedIn that they are working on it.

Also, they have a feature request that has been getting many votes recently: https://feature-request.scaleway.com/posts/1251/prompt-cachi...

Re:Hetzner is working on LLM Inference by hoppp on Friday July 24, 2026 @08:28AM (#49034609)
Scaleway does that already but competition is always good.
Re:Hetzner is working on LLM Inference by Aldipower on Friday July 24, 2026 @09:00AM (#49034933)
Does not scale in my pocket unfortunately.
Re:Hetzner is working on LLM Inference by atherton94027 on Friday July 24, 2026 @09:41AM (#49035458)
Never understood why hetzner gets brought up so positively here when there are dozens of EU competitors who are just as good and don't run 7 year old discounted servers
Re:Hetzner is working on LLM Inference by swiftcoder on Friday July 24, 2026 @11:57AM (#49037550)
> don't run 7 year old discounted servers

That's a pretty big part of why, honestly. How many of us are running server workloads that actually need the price/performance tradeoff of cutting edge server hardware?

Re:Hetzner is working on LLM Inference by atherton94027 on Friday July 24, 2026 @02:54PM (#49040113)
We're not even talking cutting edge. If you browse hetzner's auction page you can see they're renting machines with almost decade old CPUs like the Xeon W-2145 or i7 6700. That's just a lot of old hardware..
Re:Hetzner is working on LLM Inference by idiotsecant on Friday July 24, 2026 @04:25PM (#49041173)
So? If it's cheaper and gets the job done who cares?
Re:Hetzner is working on LLM Inference by atherton94027 on Friday July 24, 2026 @05:55PM (#49042077)
I mean if you run serious stuff on a non-ECC CPU from 2015, you'll probably have a bad time.

They're great for hobbyists but people on here keep bringing them up as the flagbearer of euro hosting

Re:Hetzner is working on LLM Inference by Yiin on Friday July 24, 2026 @10:56PM (#49044082)
serious stuff like what? majority of people don't work on scale you probably are thinking about.
Re:Hetzner is working on LLM Inference by swiftcoder on Saturday July 25, 2026 @03:28AM (#49045316)
I have several production services running right now on AWS t2 instances, which were released in 2016. For a service that isn't performance-critical, it's perfectly fine (and a significant cost savings) to run on decade-old hardware.

ECC vs non-ECC is also pretty irrelevant on a box with < 64GB of memory

Re:Hetzner is working on LLM Inference by atherton94027 on Saturday July 25, 2026 @11:57AM (#49048711)
A t2 instance is virtualized, AWS keeps its servers for 5 years (see their 10k https://www.sec.gov/ix?doc=/Archives/edgar/data/1018724/0001... ), so your t2 does get moved around regularly to newer servers.

And no, ECC is relevant with less than 64 gigs of ram, there's a reason it's been a standard on servers since the 90s

Re:Hetzner is working on LLM Inference by yjftsjthsd-h on Friday July 24, 2026 @05:53PM (#49042057)
Well. Yeah. The auction page is the leftovers; if you want current stuff then you're in the wrong spot on their site
Re:Hetzner is working on LLM Inference by MathiasPius on Friday July 24, 2026 @09:43AM (#49035488)
Because nobody ever mentions them by name, leaving all of us in the dark.
Re:Hetzner is working on LLM Inference by atherton94027 on Friday July 24, 2026 @09:47AM (#49035576)
Upcloud, Scaleway, Clever cloud, etc.
Re:Hetzner is working on LLM Inference by ffsm8 on Friday July 24, 2026 @10:26AM (#49036190)
These are all solid choices, but significantly less performance/€

Understandably too, because hetzner is running things on consumer hardware vs the enterprise hardware the others use

Re:Hetzner is working on LLM Inference by eliaskg on Friday July 24, 2026 @09:44AM (#49035515)
OVH already provides this → https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalo...

It works great.

Re:Hetzner is working on LLM Inference by someone4958923 on Friday July 24, 2026 @09:47AM (#49035577)
IONOS (1&1) also offers inference hosted in europe/germany: https://cloud.ionos.com/managed/ai-model-hub
Re:Hetzner is working on LLM Inference by npodbielski on Friday July 24, 2026 @10:41AM (#49036425)
No Qwen 3.6? Why?
Re:Hetzner is working on LLM Inference by rf15 on Friday July 24, 2026 @12:46PM (#49038261)
Because they're not serious about their offerings. They are an ISP and like to provide/bundle very basic additional services (hosting, mail, etc. and now 'AI')
Re:Hetzner is working on LLM Inference by rf15 on Friday July 24, 2026 @12:41PM (#49038191)
IONOS is established, but uh, 'respectable' is another question...
Re:Hetzner is working on LLM Inference by ghaering on Friday July 24, 2026 @12:54PM (#49038383)
STACKIT also offers model serving.
Re:Hetzner is working on LLM Inference by beernet on Friday July 24, 2026 @03:06PM (#49040299)
To all mentions of IONOS / StackIT / OVH:

All of these are lightyears behind US/China LLM offerings. None of them offer any model close to open-source SOTA. Runtime and reliability is a disaster and sure enough you have to pay a "sovereignty" mark up.

Re:Hetzner is working on LLM Inference by expedition32 on Friday July 24, 2026 @05:25PM (#49041766)
Freedom isn't free. Unless you want to live in a world were your children have to read the bible in class sacrifice has to be made.
Re:Hetzner is working on LLM Inference by frizlab on Friday July 24, 2026 @06:54PM (#49042610)
I live in the EU and want to live in a world where my children actually CAN read the Bible in class. However that’s not really possible anymore…

(Also no, modern science is (still) not incompatible with (the Christian) religion. I want my children to have both, as do I, or at least be taught both, so they can decide for themselves what they want when they are grown enough to do so.)

Who is sacrificing what, really?

Re:Hetzner is working on LLM Inference by beernet on Friday July 24, 2026 @05:40PM (#49041931)
Paying a premium for inferior service quality is not exactly freedom