News for hackers. Stuff that matters.
Re:Kubernetes Is Hard by ActorNightly on Monday March 27, 2023 @06:15PM (#35333506)
I disagree that production is hard in itself, I think the way people approach production makes it needlessly more difficult. It seems that when launching a service, everyone seems to try to get the entire stack in one shot, which is difficult.

A much better approach is "1. Make it work 2. Make it good 3. Make it fast". Your initial prototypes need to define the core functionality, and then you incrementally build stuff on top of that.

Re:Kubernetes Is Hard by YetAnotherNick on Monday March 27, 2023 @06:39PM (#35333789)
> 1. Make it work 2. Make it good 3. Make it fast

Disagree. The first two objective clashes very much with each other. I have worked in teams that did that. And generally if anything non trivial is done without planning to just "make it work", it would be more often than not, ridden with many design issues which doesn't allow it to be good.

Re:Kubernetes Is Hard by nerdix on Monday March 27, 2023 @07:02PM (#35334033)
I've never seen anything that couldn't be incrementally improved. There are definitely tons of situations where the additional effort to "make it good" is worth incurring upfront. But you can definitely make something work then make it better. And a lot of times, you'll find that your carefully planned out "good" solution ends up not being all that good and needs incremental improvement. It can be hard to get something right on the first try especially if you haven't done it before.
Re:Kubernetes Is Hard by jakupovic on Monday March 27, 2023 @07:08PM (#35334091)
Fully agree here. Not being abe to incrementally improve something that works would be a contradiction as the original thing never worked at all then.
Re:Kubernetes Is Hard by rektide on Monday March 27, 2023 @08:26PM (#35334144)
I see way more making-poor-decisions underbidding-complexity, then spending years hacking on shit cause it's kind of jank.

When folks have license to do good work, I usually see it going well. If they have some informed opinions & group wisdom at their disposal & which they leverage.

Re:Kubernetes Is Hard by YetAnotherNick on Tuesday March 28, 2023 @12:41AM (#35335990)
I have rarely seen bad but barely workable website become good with incremental improvement. After the product reaches some threshold of users, it is hard to change the code structure in a safe testable way if there is no good way to test it in production like environment, and so developers are rightfully afraid to make any changes. I have seen many examples both internally and externally.

- My bank's website which is a decade old and has millions of users, has tons of issues. I am not even talking about UI or look. It randomly hangs, session disconnects etc.

- ERP tool of one of my previous org sucked, even though used by 10s of 1000s of people and having dedicated team in maintaining.

- One team that I worked with has no proper staging environment, and it would have required synced deployment of dozens of services with many teams working with different language and config. And just this simple task couldn't be completed even with push from upper management, just because of complexity.

Re:Kubernetes Is Hard by ActorNightly on Tuesday March 28, 2023 @02:51PM (#35346004)
If you start designing into the future, you are now trying to make it good before making it work.

Making it work means you start with barebones, just to have a proof of concept that runs, is accessible, is secured, even if you have to click things together and manually ssh into EC2s to launch code. Once you have everything working, then the bigger design should be a lot clearer, in terms of what you have to do to make it good.

Re:Kubernetes Is Hard by lcw on Monday March 27, 2023 @06:36PM (#35333745)
> 1. Make it work 2. Make it good 3. Make it fast

Anecdotally, everytime someone tells me Kubernetes is overkill and then follows your approach. In a year they end up building the capabilities that come out of the box with Kubernetes, and of course they aren't as well thought out. Because they were done adhoc and as needed.

Re:Kubernetes Is Hard by dgroshev on Monday March 27, 2023 @08:55PM (#35334361)
Why is that bad? If the business survived for long enough to have that problem, that's a win, not a failure. Being killed or at least hobbled by unnecessary complexity, on the other hand, is a thing in way more businesses than we like to admit.
Re:Kubernetes Is Hard by Spivak on Tuesday March 28, 2023 @12:28AM (#35335919)
Yep, you can grow to be a huge company with just autoscaling groups and some terraform.

Absolutely painfully boring stack. I’m working at a place now with that stack and we haven’t had a single page in over a year now and we do 5 deploys a day ish.

Re:Kubernetes Is Hard by Veuxdo on Monday March 27, 2023 @06:20PM (#35333576)
Security as well. Now step 0 is "Make it secure". When using microservices connected via the internet, you have no choice.
Re:Kubernetes Is Hard by madeofpalk on Monday March 27, 2023 @06:27PM (#35333651)
> When using microservices

1. Make it work

Re:Kubernetes Is Hard by dgroshev on Monday March 27, 2023 @09:07PM (#35334460)
Security is a spectrum.

You can go from out of the box Debian listening on 22/80/443 directly on internet to WAF, intrusion detection, encrypted overlay networks with CA hierarchy and offline route keys, encryption at rest, elaborate IAM, hardware signed commits, full supply chain audits with every dependency vetted and vendored, multi-access-level logs, mandatory spyware on all corporate gear, and so on and so forth. For most businesses it's more of the former than the latter, and there's definitely no such thing as categorically "secure".

Re:Kubernetes Is Hard by yourapostasy on Monday March 27, 2023 @06:51PM (#35333918)
When I see developers use k8s as “programmable infra replacing on-prem functionality as a baseline”, some time after step 3 is, “make it as cheap as on-prem”. Unfortunately, many of the “move fast and break things” tenor that got the services built in k8s in the first place, baked in decisions that will cost as much to re-engineer into an elastic cloud-savvy form that also saves money compared to on-prem.

This is often from organizational or project management mis-coordination than technical choices the developers are fully aware of. Like a system’s owners not prioritizing putting events of interest into a message queue and insisting upon polling for the information on one end. And an offshore operations team saying learning how to support AWS Lambda/Azure Automation/GCP Cloud Functions is an added cost scope to the contract. So the developer is backed into a corner to run cron/SystemD-based polling off a small EC2. Thousands of decisions like this add up fast, and take a major executive committee commitment to root out tech debt and prevent new tech debt from accumulating to unsustainable levels to prevent cloud initiatives from sinking beneath a mire of expensive small decisions in pursuit of new accomplishments. It is a tough balancing act.