Like, if I'm configuring the foo service and it needs to know how to contact the bar service... that's not a problem? I just put bar-service-lb.mycompany.net in a config file and I'm done. I don't understand why people think they need complex "service discovery". Just give things names!
My cloud provider (or heaven forbid, wires in a data center! gasp!) already gives me perfectly good networking. Why layer anything on top of that?
No one really needs "ingress". Your server program can listen on a port! Like we've been doing for 50 years! This is a completely self-inflicted problem.
It's so depressing seeing the k8s generation completely forget the highly effective ways that we used to do things. All those things are still possible! You can choose to design, build, and run a simple system if you want to!
What K8s does is provide a simple and idiomatic rest API on top of all this architecture. No longer do you have to read a ton of disparate man pages, figure out what the sysadmin 10 years ago did for DNS, etc.. you just use kubectl or the API. It's the same interface to spin up more replicas of a frontend service as it is to open a port for ingress or even collect and review logs.
What about when you're given a few servers for your dev/test environment, but don't have control over the DNS server and adding records to it has an annoying amount of red tape?
Would this be a good option:
- FOO_URL=http://300.0.0.1:3000
- BAR_URL=http://300.0.0.1:8080
- BAZ_URL=http://300.0.0.2:80
- X_URL=http://300.0.0.2:5601
- Y_URL=http://300.0.0.3:5602
- FOO_URL=http://foo.dev.svc.cluster.local
- BAR_URL=http://bar.dev.svc.cluster.local
- BAZ_URL=http://baz.dev.svc.cluster.local
- X_URL=http://x.dev.svc.cluster.local
- Y_URL=http://y.dev.svc.cluster.local
I guess you can just run your own DNS server at that point as well, but that might raise some eyebrows in some places, vs just using whatever the orchestrator provides you with out of the box (and Kubernetes also has built in proxy functionality so people can connect to the cluster pretty easily and forward whichever services they want, which can take a bit of work otherwise, though i guess you'd get used to the proper order of arguments in the SSH tunnel commands too).
> No one really needs "ingress". Your server program can listen on a port! Like we've been doing for 50 years! This is a completely self-inflicted problem.
What if you want all of your services to have SSL/TLS certificates provisioned for them? What if you want to ensure that all of the inter-service communication is also encrypted? What about you wanting to add circuit breaking or rate limiting logic in there? Or maybe doing a MitM on yourself so you can debug the network traffic between the services better? Also, do you really want to configure the firewall rules for each of your internal services to be able to talk to others, or wouldn't it also be good to have it piggyback its traffic on some other overlay networking port that's open on all of them for cluster communications?
Oh, also to avoid weirdness with CORS or other technologies, it might be nice to route requests based on domains and paths, instead of ports (e.g. my-app.com/analytics might be better than my-app.com:9000 in some contexts), though of course that is easily achievable with any reverse proxy, just saying that ports aren't always the best option.
Personally, I think that there's a lot of benefit to using an ingress and a service mesh of some sort, as long as those don't add too much undue complexity. I mean, using our own Nginx/Apache/Caddy/Traefik instance instead of the Ingress abstraction was also perfectly passable, but there are certainly benefits to exploring new technologies and what they offer.
In my experience, all of these container technologies providing you with their own DNS implementation out of the box is very useful, especially because recognizing that "300.0.0.1:3000" is the wrong URL is harder than looking at "foo.dev.svc.cluster.local" and seeing that your service probably doesn't intend to connect to foo.
> It's so depressing seeing the k8s generation completely forget the highly effective ways that we used to do things. All those things are still possible! You can choose to design, build, and run a simple system if you want to!
I do agree with this, though! There should be more resources out there to show off how to do basic and useful networking without getting too deeply into some of the fancier technologies.
Might as well have dug up https://en.wikipedia.org/wiki/Reserved_IP_addresses#IPv4 but the arbitrary nature of it all hurts my head.
Furthermore, if you have 5 different projects that you need to work on and each of those would have their own separate DNS servers, you might find yourself in a slightly uncomfortable situation.
Most container orchestration solutions don't have that problem, because they embed their own server and provide the necessary proxying solutions so that you may connect to the cluster from outside as well.
Doing it yourself is probably only cheaper if you never need maintenance or upgrades, security patching isn't needed, or your time is cheap.
Backup recovery in a small, simple, robust system is very rare. Automated recovery is complex and there are quite a bunch of gotchas and perhaps project specific things to take into account.
But automated backups are typically straightforward. You need to know when and how to do them and where to put them.
I think there are tons of reasons to pay for an managed database if you need it. But it is not a baseline requirement for many projects.
For managed, I don't think I'd want to go lower than the $50 plan which would be roughly equivalent to what I have now.. would include storage but not backups I think.
But ya the data is valuable.. I'm running nightly backups. I want to do continuous backups someday or have some way to not lose up to 24 hours of data if something happens.
If I need to route to other things I just configure other dns entries with SSM Parameter store... or I just manipulate the local resolvers in the VPC.
Very straightforward.
But I'm generally running just my software, not other people's, and consuming aws datastores.
This is for work.
4 years later consul is gone in aws for us replaced by this simple dns/alb scheme. We haven't really needed all the features of advanced discovery.
The key in general to "discovery" seems to be load balancers and dns in our approach (service protect thyself vs well behaved clients). Easy to use LBs on aws is very powerful feature. We don't have that on our on prem and it changes how we approach things. There discovery is harder.
But if that's a feature that you want, you can even use Docker Swarm which I mentioned in another comment here, which basically gives you the same thing, so that you can reference other containers by their names.
Of course, it's a bit simpler than Nomad, so if you're already using Nomad then there's no point in moving away from it, though this still illustrates that there are plenty of simpler options than K8s!
You probably don't need to have a complicated microservices topology though. A load balancer->stateless app tier->managed RDBMS architecture should be more than enough until you're at a point where you can afford to scale a serious infrastructure team.
But... if you just use k8s you get things like basic service discovery, networking, ingress, etc. with it and don't have to figure out bespoke solutions (that you'll just chuck anyways once you move to k8s).
I do agree though I would be very hesitant to immediately dive in running stateful workloads like databases, etc. on my own k8s cluster. The cloud hosted database services that every provider has is such a significant time and complexity saver, especially if your databases are just getting started and small-ish.