One additional suggestion to people considering bare metal: Consider baking in a VPN setup from the start, and pick a service discovery mechanism (such as e.g Consul) that is reasonably easy to operate across data centres. Now you have what you need to do migration if you need to, but you also have the backbone to turn your setup into a hybrid setup that can extend into whichever cloud provider you want too.
A reason for wanting that is that one of the best ways I've found of cutting the cost of using bare metal even further is to have the ability to handle bursts by spinning up cloud instances in a pinch. It allows you to safely increase the utilisation levels of your bare metal setup substantially with according cost savings even if you in practice rarely end up needing the burst capability. It doesn't even need to be fully automated, as long as your infra setup is flexible enough to accommodate it reasonably rapidly. E.g. just having an AMI ready to go with whatever you need to have it connect to a VPN endpoint and hook into your service discovery/orchestration on startup can be enough.
Do you just use iptables? Or do you build out more complex solutions like software routers running on Linux/BSD?
I work in online gaming, and we're constantly seeing attacks on our infra.
You can add a magic header to traffic and drop anything that doesn’t contain the header.
Since this is done in hardware it operates at line speed. In our case 100GBit/s.
If there isn't a possibility for internal-networking (free), then I'd probably use the included iptables for a firewall on each machine. You should honestly have this running on the game servers anyway, if only to restrict communication to between the reverse-proxy and game server.
[0]: https://nimbusws.com
If we just put a LITTLE bit back into (like 5% of revenue for any of the big companies honestly -- but I'm not big yet so more for me) the F/OSS ecosystem, can you imagine what kind of world we'd be in??
I want to live in that world, so I'm trying to make it happen.
That sounds like the worst reason to change cloud providers: "because that provider bribed me to"
I think at some point Azure announced $X in free credits for YC members, and GitLab determined this would save us something like a year's worth in bills (quite a bit of money at the time). Moving over was rather painful, and I think in the months that we used Azure literally nobody was happy with it. In addition, I recall we burned through the free credits _very_ fast.
I don't recall the exact reasoning for switching to GCP, but I do recall it being quite a challenging process that took quite some time. Our experiences with GCP were much better, but I wouldn't call it perfect. In particular, GCP had/has a tendency to just randomly terminate VMs for no clear reason whatsoever. Sometimes they would terminate cleanly, other times they would end up in a sort of purgatory/in between state, resulting in other systems still trying to connect to them but eventually timing out, instead of just erroring right away. IIRC over time we got better at handling it, but it felt very Google-like to me to just go "Something broke, but we'll never tell you why".
Looking back, if I were to start a company I'd probably stick with something like Hetzner or another affordable bare metal provider. Cloud services are great _if_ you use their services to the fullest extend possible, but I suspect for 90% of the cases it just ends up being a huge cost factor, without the benefits making it worth it.