News for hackers. Stuff that matters.
Re:Please do not attempt to simplify this code by spullara on Friday December 28, 2018 @03:36AM (#18775566)
My take away from reading this code is that it is a huge mess that may be impossible to clean up. At some point they failed to introduce abstractions that would remove the need for all this complexity. They are probably right that now that it works that it will be hard to refactor it without leaving out some critical case. However, I pity anyone that works on this code base.
Re:Please do not attempt to simplify this code by healsjnr1 on Friday December 28, 2018 @03:53AM (#18775645)
Given where they are, it seems that now the major abstraction they are missing is type safety. Strict types that allowed a monadic structure would simplify this whole mess into a series of maps and folds.

Not only does this add the stricture of a compiler enforcing correct return types for all conditionals, it is also idiomatic of any language that supports this.

Having said all that, changing languages is rarely an option. Maybe this is the best solution given the tool available. If so, it demonstrates that maybe this tool isn't right for this job.

Re:Please do not attempt to simplify this code by andrewmcwatters on Friday December 28, 2018 @11:44AM (#18777793)
Yeah, I just don't buy this. It says nothing. This is the bottom-barrel suggestion most people bring up, and there's more to simplifying codebases than "strict types."

What in this <2k LOC file suggests strict types help anything whatsoever? It's not massive. It's large, but fold this and it immediately becomes more readable. Any attempts to simplify this would simply be reasoning about it differently.

Re:Please do not attempt to simplify this code by kierenj on Friday December 28, 2018 @03:45AM (#18775606)
Would you agree that sometimes, abstractions can just distribute and hide complexity whereby actually all of that context is needed to comprehend the algorithm or process at hand (some things just ARE complex)?

In this case right here, what's your counter-example, or what would you use instead of their specific approach?

Re:Please do not attempt to simplify this code by lmm on Friday December 28, 2018 @05:05AM (#18775920)
> Would you agree that sometimes, abstractions can just distribute and hide complexity whereby actually all of that context is needed to comprehend the algorithm or process at hand (some things just ARE complex)?

No, or at least not often enough to be worth thinking about. It is of course possible to use abstractions badly, but the problems that business software has to solve are always fairly simple because they're human problems; human business processes were never that complex. So if a program looks really complicated, the overwhelmingly likely cause is failing to use appropriate abstractions.

> In this case right here, what's your counter-example, or what would you use instead of their specific approach?

As others have said, a result type would greatly simplify this code without sacrificing any safety. No doubt after such a simplification made the code shorter and easier to comprehend, further simplifications would become apparent.

Re:Please do not attempt to simplify this code by femiagbabiaka on Friday December 28, 2018 @05:20AM (#18775976)
Your metaphor doesn’t really apply — this isn’t business software, it’s part of a distributed job scheduler. That is inherently complicated. There is of course an open argument of language choice, but I think there’s something to learn from the lack of software built in $BETTER_LANGUAGE.

It’s also an open source project, so if anyone wanted to throw up a branch with an example of simplification without sacrificing logic branch completeness that door is open. Code might even convince the k8s team to change what you think is misguided behavior.

Re:Please do not attempt to simplify this code by chrisdone on Friday December 28, 2018 @06:11AM (#18776133)
If you're saying they should have solved a distributed problem with a distributed language like Erlang, I'd probably agree.
Re:Please do not attempt to simplify this code by lmm on Friday December 28, 2018 @05:25AM (#18775991)
> That is inherently complicated. There is of course an open argument of language choice, but I think there’s something to learn from the lack of software built in $BETTER_LANGUAGE.

Sometimes the best software is no software. Kubernetes exists to solve problems that people using better languages don't generally have.

> It’s also an open source project, so if anyone wanted to throw up a branch with an example of simplification without sacrificing logic branch completeness that door is open.

The "Please do not attempt to simplify this code" comment suggests otherwise. In any case, the team have chosen a language in which good solutions are not possible.

Re:Please do not attempt to simplify this code by barrkel on Friday December 28, 2018 @08:47AM (#18776649)
> Kubernetes exists to solve problems that people using better languages don't generally have

This statement doesn't make sense, and the arrogant tone just stinks of ignorance.

Kubernetes isn't perfect, but if there's better provider-agnostic language-agnostic way to dynamically scale an application written in heterogeneous languages across a cluster or on a competitive choice of cloud platforms with dynamic provisioning, I'd like you to tell us, and explain why it's better than Kubernetes.

Language agnosticism isn't something to discard lightly. Different languages have different strengths and weaknesses.

Re:Please do not attempt to simplify this code by lmm on Friday December 28, 2018 @10:14AM (#18777149)
In my experience the costs of language agnosticism outweigh the benefits even without considering deployment. Good languages are general-purpose; a language that is drastically unsuited to some particular thing is probably not worth using at all, and if you're using a language that is decent at most things then the overhead of switching languages is higher than the benefit of using a language that gives a small advantage in some specialised area.
Re:Please do not attempt to simplify this code by geodel on Friday December 28, 2018 @11:56AM (#18777881)
What language you have in mind? All the languages I have known or used are drastically unsuited for one or other thing. So they all seem not worth using.
Re:Please do not attempt to simplify this code by lmm on Friday December 28, 2018 @12:59PM (#18778291)
I mostly work in Scala these days. But really any ML-family language is fine for everything. Garbage collection is not the barrier to writing system tools or soft realtime that it's reputed to be (the fact that people advocate Go for those use cases is proof of that).

(The JVM startup can be a bit slow for scripting one-liners, particularly if you're running them on every file in a directory or some such. There was a time when I would switch to Python or Bash for those, but I realized I was wasting more time making mistakes in those languages than I ever saved at runtime)

Re:Please do not attempt to simplify this code by dblotsky on Friday December 28, 2018 @03:14PM (#18779116)
> “human business processes were never that complex.”

That’s a falsifiable claim. Human problems / laws are some of the most difficult to code for, IMO. Examples: legal software, economics software, etc. Usually complexity increases as you delve out of the abstract and deal with real-life processes with humans.

Re:Please do not attempt to simplify this code by dilyevsky on Friday December 28, 2018 @04:13AM (#18775714)
+1. The funniest part is amount of tests that this code has (if i looked in the right place). Ofc there’s approx 0 chance of finding any regression!
Re:Please do not attempt to simplify this code by austincheney on Friday December 28, 2018 @05:25AM (#18775990)
Abstractions don't remove complexity. Abstractions instead hide the appearance of complexity behind layers of ever increasing code.
Re:Please do not attempt to simplify this code by xendo on Friday December 28, 2018 @05:31AM (#18776006)
You are right that they move the complexity somewhere else, but when abstractions are done right, you don't have to worry how they are implemented when you use them. That is huge benefit.
Re:Please do not attempt to simplify this code by fluffycat on Friday December 28, 2018 @05:42AM (#18776043)
I believe that premature abstraction is terrible. But in some cases good abstractions may help with handling complexity and may make testing easier. I have a feeling that because authors say that this class should not be changed / refactored, they failed to introduce a good abstraction. This also implies that their tests are inadequate.
Re:Please do not attempt to simplify this code by austincheney on Friday December 28, 2018 @07:31AM (#18776385)
I agree with the direction of your opinion, but would say it isn't a matter of helpfulness. In some cases is it is a necessity.

It must be understood that complexity is an economic consideration and not either a financial or technology consideration. It is always already present. Complexity is not something that is created or destroy; but retained, absorbed, or transferred.

Abstractions become necessary when they separate different functional layers to perform different respective responsibilities. In this case complexity is transferred both in and out of a system at a given layer, but you don't care so long as you aren't doing the jobs of the other layers. That is referred to as separation of concerns which results in the hardening of a system (risk reduction) which is the side effect of reducing costs due to restricting requirements available.

Many abstractions exist solely to provide a layer of convenience. In the case where an abstraction does the same job as the code it abstracts risks increase and costs increase. This is because the system continues to simultaneously absorb and transfer complexity like described above, but the requirements between the various layers isn't clearly separated. That results in fulfilling requirements, the same requirements, simultaneously at various layers. This has various names like scope creep, technical debt, and so forth. This is bad because risks and costs increase directly to the correlation of increased code and increased requirements. This is what makes the law of leaky abstractions valid.

It is easy to tell the difference between a necessary abstraction and a wasteful abstraction by the forcefulness of separation. If you can perform the same job in a lower level the abstraction isn't necessary and you are probably better off without it. Most JavaScript frameworks are unnecessary abstractions.

Re:Please do not attempt to simplify this code by mdpopescu on Friday December 28, 2018 @12:46PM (#18778204)
A file is an abstraction that hides an incredibly complex process, possibly even involving quantum effects. I have never found a reason to care about the actual details and managed to do quite a significant amount of work using just the abstraction.