Hetzner has nice dedicated servers for €33/mo:
I'm on an older one that just got bumped up to €29/mo due to increasing electricity prices; it was €21/mo until now, and I can't blame them for that one. The specs are E3-1245 V2 / 16GB / 2x3T, there's over 45 vhosts on it across ~25 wwwroots plus other random services, and CPU usage is basically nothing. The cores are really there just to handle bursty stuff. Most random side projects and small websites don't need almost any resources on modern hardware.
Previously I was on a Scaleway Dedibox, which go as low as €15/mo right now. It was €10 at one point even.
But the ISPs I know do not bill $$$ if you use the max bandwidth (max bandwith they did advertize to you btw) for a sustained amount of time: they'll just start throttling you.
Anyway GP ain't asking about "cloud vs hosting at home" but about "cloud vs dedicated server(s)".
Of course, the delayed sticker shock is a problem.. I think Google cloud actually lets you create a budget that turns services off if they go over, so there's a solution here if you run a hobby project that you suspect might take off and cost you more than it's worth.