> The example code is vey simplistic, so of course that linear code is more readable, but the idea doesn’t scale.

One of the best reviewed functions I wrote at work is a 2000 line monster with 9 separate variable scopes (stages) written in a linear style. It had one purpose and one purpose only. It was supposed to convert from some individual html pages used in one corner of our app on one platform into a carousell that faked the native feel of another platform. We only needed that in one place and the whole process was incredibly specific to that platform and that corner of the app.

You could argue that every one of those 9 scopes could be a separate function, but then devs would be tempted to reuse them. Yet, each step had subtle assumptions about what happened before. The moment we would have spent effort to make them distinct functions we would have had to recheck our assumptions, generalize, verify that methods work on their own... For code that's barely ever needed elsewhere. We even had some code that was similar to some of the middle parta of the process... But just slightly didn't fit here. Changing that code caused other aspects of our software to fail.

The method was not any less debuggable, it still had end to end tests, none of the intermediate steps leaked state outside of the function. In fact 2 other devs contributed fixes over time. It worked really well. Not to mention that it was fast to write.

Linear code scales well and solves problems. You don't always want that but it sure as hell makes life easier in more contexts than you'd expect.

I worked with an engineer that wrote the most clear and elegant linear code. It was remarkable, never seen anything like it since. I can't reproduce it but I do have an idea of what a well designed linear function looks like.. a story.

That is true of well designed nonlinear code as well.The code needs to tell a story or it will be a mess.

I will agree that it takes some skill, not that I am great at it. It's a different kind of skill than abstraction. Reading error handling in c code offered good insights for me to learn linearity better (c code that uses goto to jump to the end of a function for cleanup when an error occurs, for example).

However, if you screw up linear code, you screw up locally. If you write poor small functions, the rest of the team screws up because they barely ever read the contents of your functions that call other functions that call other functions. I've had way more problems with stuff being called slightly out of order, than with large functions.

I was just thinking that if I _needed_ to refactor this I might structure the stages as chapters in a book. One might be able to write an inner class or some such that had a “table of contents” function that called each stage in sequence as a void function with data managed out of line, maybe via cleverly designed singleton structs. Then the code itself can be written in order with minimal boilerplate between stage boundaries.

I think I’ve worked with some Python that looked and worked this way. I can’t place the details but probably in a processor pipeline running over a particularly hairy data format. Consider ancient specifications written by engineers talking on the phone encapsulated in relatively “modern” but still vintage specifications, sometimes involving screen-scraping a green screen mainframe terminal, wrapped in XML and sent over the internet. Anyway, point is I couldn’t agree more about stories.

I don't think the fact that a function works well is a good enough reason to write a 2000 line function. Sometimes there are long pieces of code that implement complex algorithms that are difficult to break into smaller pieces of code, but those cases are limited to the few you mentioned.

Computers execute code in a linear fashion, why on earth would you "need a reason" to NOT abstract something? Just because abstraction is often the right thing to do doesn't make it the base case.

It's like saying you need a reason not to add 4000 random jumps in your assembly code just to make it more difficult to read...

Source code isn't written to be executed by computers, it's written to be read by other humans.

Source code tends to be very far removed from how computers execute anything, so I wouldn't use that as a justification for any sort of code style.

> Source code isn't written to be executed by computers, it's written to be read by other humans.

It is pronounced "documentation".

I guess all we know is there were 2K lines of code and the commenter thinks that was the right way to do it. It would be necessary to see the code to appropriately critique it.

Not just the commenter, but his team as well. It passed code review with flying colors, apparently. The moral of the story is that there always exceptions and developers should not be ideologically committed to one approach above all else.

we know more than that: You could argue that every one of those 9 scopes could be a separate function, but then devs would be tempted to reuse them. Yet, each step had subtle assumptions about what happened before.

what we don't know is if it would have been possible to abstract those assumptions away so that functions could have been defined without them.

Even the contrived example in the post can be factored differently (and better imo). How do we know those 9 scopes are appropriate?

We do know that if we trust the poster, they said very clearly it could have been done but they didn't consider the value to outweigh the downsides.

yes, i meant we don't know if it would have been possible to extract functions in such a way that they are actually safely reusable.

> that implement complex algorithms that are difficult to break into smaller pieces of code

My longest code is always image processing. It's usually too hard to break up for the sake of breaking up. There's nothing to reuse between the calls to filters/whatever.

The default should be reversed, don't break into smaller pieces unless there's a really good reason.

>I don't think the fact that a function works well is a good enough reason to write a 2000 line function.

The fact that it works well and reads well (when it does, as in the parent's case), is.

Aside from those factors what else would be against it? Dogma?

I don't know if it is still like this, but the code for dpkg used to be like this, and it was amazing: if you ever needed to know in exactly what order various side effects of installing a package happened in, you could just scroll through the one function and it was obvious.

To this end, I'd say it is important to be working in a language that avoids messing up the logic with boiler plate, or building some kind of mechanism (as dpkg did) to ease error handling and shove it out of the main flow; this is where the happy path shines: when it reads like a specification.

>”but then devs would be tempted to reuse them”

Isn’t that the fucking point? Having a 2000 line function is a code smell so bad, I don’t care how well the function works. It’s an automatic review fail in my book. Abstractions, closures, scope, and most importantly - docs to make sure others use your functions the way you intended them. Jesus.

Agreed. I’ve written plenty of software of all kinds and have never had to write a 2000 line long methods (although I have had the joy of refactoring such messeses a time or two).

Just don’t do that. Your code doesn’t have to have abstractions out the wazzo, but if your class (or method) is getting bigger than 1000 lines that’s a great sign that it’s doing too much and abstractions can be teased out. Your future self will thank you, as well as your team.

I like this from Sandi Metz:

> You can't create the right abstraction until you fully understand the code, but the existence of the wrong abstraction may prevent you from ever doing so. This suggests that you should not reach for abstractions, but instead, you should resist them until they absolutely insist upon being created.

At least in the mobile world, I find that this “no abstraction” approach is the default one, and it usually leads to huge objects which do everything from drawing views to making network requests to interacting with the file system. These kinds of classes are quite hard to work in, hard to test, and also keep snowballing to get bigger and bigger. Things usually end with unmaintainable code and a full rewrite.

I am not saying you need to create complex abstract hiarchies right off the bat. But usually, it’s pretty easy to tease out a couple significant abstractions that are very obvious, and break down your classes by a factor of two or three. Just getting such low hanging fruit will prevent you from ever having a 2000 line long method.

And for the folks who are saying that they make sure to not add abstractions too early - are you disciplined enough to go back and add them later? I feel like if you’re the kind of engineer that busts out 2000 line methods, you’re also not going to refactor it as this method grows to 2500 or 3000 lines or beyond.

Probably most robust software you depend on is full of solid, quality abstractions. Learning to write code like this takes practice. The wrong abstraction might be wrong, but it’s one step closer on your journey to growing as an engineer. You won’t grow if you never try.

Some devs did find it a code smell... But each scope had a clear short high level comment describing what it did, there were end to end tests for the method, and very little state flowed from scope to scope (some did) - because that's what scoprs do... Prevent variables from leaking.

My point is the code smell isn't always accurate, and there are times and even for 2000 line monsters other devs agreed that there, it was the best way to hide complexity away from the rest of the codebase. If we ever needed to factor things out (we never did), we could spend some effort and do it.

A code smell means you should look into it, not that it's wrong.

Some things are genuinely 2kloc-complex. Maybe not that many. Do check! But some are.

Definitely not that many. Even for me this was an outlier, but it made me more comfortable with functions most people would consider long.

I'd like to clarify this was not necessarily 2kloc-complex, this was just 2kloc-long-and-not-really-meant-to-be-reused. It was a fairly long but linear process that was out of the ordinary for the rest of the codebase. It could easily have been split (hell, I had 9 fairly separate stages), but calling any of the intermediate stages out of order or without the context of the rest of the execution flow... would have been a foot gun for someone else. And, as time showed, we never needed those stages for anything else.

Have you tried reading code instead of smelling it?

>The moment we would have spent effort to make them distinct functions we would have had to recheck our assumptions, generalize, verify that methods work on their own

Why? Why can't the functions say "to be used by <this other function>, makes assumptions based on that function, do not use externally"? Breaking out code into a function so that the place it came from is easier to maintain... does not mandate that the code broken out needs to be "general purpose".

Specifically, in that place, there was no need. And prematurely splitting it would have caused us to overthink and over generalize. Having a long, linear and tested function was a better choice.

I understand your point, but perhaps that would have simply been an opportunity to refine your approach to code design. If such a situation leads to excessive deliberation and overgeneralisation, your code base must be riddled with unnecessary overthinking and overgeneralisation.

Or maybe it was just a long, sequential algorithm where breaking it up wouldn't have been an improvement.

I have been programming for more than 30 years. Except for code generated explicitly to be only consumed by machine, I've never come across a function consisting of 2000 lines of code that should not have been broken up. Something is wrong there, and if you show me the code, I'll tell you what's wrong with it.

I can imagine a new control statement with this type of syntax:

  code
  code
  uses (a, b, c, d) {
    code
    code
  }
  more code
  more code

It's a block that defines the variables it uses, with no other access to the outer scope. It would help break up a linear function into blocks with clearer dependencies.

Glad you can see that without even looking at the code.

Some things you don't have to see to know whats going on. Function with 2000 lines of code? Have fun rationalising this.

> You could argue that every one of those 9 scopes could be a separate function, but then devs would be tempted to reuse them.

Good thinking. Now they’ll just add 50 flags and ten levels of nested ifs instead which is much simpler.

You don't have to write tests to prove that private methods work on their own. Just test the public behaviour.

At first I thought how horrible, but basically you have sort of 9 functions within the same scope, each having a docstring. So I guess not too different from splitting them up.

I read you have "end to end" tests.

One question though: Wouldn't each part benefit for having their own unit tests?

Maybe, maybe not. For our particular case it would have been mostly wasted effort.

I found that I like to write tests at the level of abstraction I want to keep an implementation stable. I'd be totally fine if someone went in and changed the implementation details of that long process if needed. We cared that stuff got cleaned up at the end of the process, that the output matched certain criteria, that certain user interaction was triggered and so on... In that case it made more sense to test all our expectations for a larger scope of code, rather than "fix" the implementation details.

Tests usually "fix" expectations so they don't change from build to build. Tests don't ensure correctness, they ensure stuff doesn't alter unexpectedly.

>One question though: Wouldn't each part benefit for having their own unit tests?

Not necessarily, especially since this allows for the case where individual unit tests pass fine, but the combined logic fails.

Tests effectively freeze requirements; you should test those things which should be preserved throughout any changes, and not test those things which should be open to change. In this case, it seems that is no real requirements for any of these 9 steps - perhaps the implementer could figure out how to do the same outcome by skipping a step or merging two steps, and the existence of unit tests for these 9 functions somehow encodes the idea that these 9 functions each are inherently needed, which is not necessarily true.

2000 lines is like a small project. I cant imagine putting that all in one function.

If the sub-functions could be reused and people would be tempted to change them, then that’s what your tests are for. In fact, it’s often tricky to test the sun-function logic without pulling them out because to write the test you have to figure out how to trick the outer function to get into certain states. Follow the Beyoncé rule: if you like it: put a test on it. Otherwise it’s on you if someone breaks it.

So where's the proof that the function'd code scales? As the complexity of the overall code grows, so would something that gets chopped into dozens of functions to the point of being unreadable.

Suddenly, you realize that the dozens of functions __need to be called in specific orders__, and they are each only ever used once. So really what you're doing is forcing someone to know the magic order these functions are composed in order for them to be of any use.

In a decent programming language you can nest functions, so all the little functions that make up some larger unit of the program are contained within (and can only be called within) that outer function. They serve less as functions to be called and more just as names attached to bits of code. And since they can't be called anywhere else, other people don't need to worry about them unless they're working on that specific part of the program.

Dozens of functions need to be called in a specific order?

Oh my God.

The truth is that either one can be done wrong.

Unfortunately organizing your code along the right lines of abstraction is something that just takes skill and can't easily be summarized in the form of "just always do this and your code will be better"

If you organize your code into units that are easy to recompose and remix, well you get huge benefits when you want recompose and remix things.

If you organize your code into units that can't be easily recomposed, then yes you've added complexity for no benefit. But why make units that can't be treated individually?

"As the complexity of the overall code grows, so would something that gets chopped into dozens of functions to the point of being unreadable."

So the answer to this is, "don't chop it into functions in a way that leaves it unreadable, instead chop it into functions in a way that leaves it more readable."

That may be unsatisfying, but it gets to the point that blindly applying rules is not always going to lead to better code. But it doesn't mean that an approach has no value.

There's an easier approach that will also aid you in telling you how to precisely chop up your function.

Simply don't chop up your function until you need a slice of it somewhere else. Then refactor out the bit you need. You'll find out exactly which bits need to be replaced with variables and exactly where the slice needs to happen.

This is the correct answer right here if you have a good enough team. It is still the way I want to work. Unfortunately, I find that there are too many developers who haven't learned that you should always be considering to "refactor as you go". I'm trying to teach by example, but it's an uphill battle.

Exactly. Start with the straight-ahead linear approach and factor out once it's unwieldy.

Same thing for copy pasta funcs -- the first copy is fine, the second one may be too, but after that consider extracting to a parameterized func (a permutation of the Go Proverb "A little copying is better than a little dependency.")

A single use function absolutely makes sense - you are effectively naming a block of code in some way, documenting it.

The API shouldn't be that. Expose something easy to use. That is the point of abstractions. It doesn't matter if there are a dozen methods called in order if those dozen methods are called by a helper method, beyond maybe some implementation details.

Really the question should always come up when there are more than say two ways to do things. If I can make a pizza from scratch, reheat a chilled pizza, create a pizza and chill it, reheat a half dozen pizzas, or make three pizzas of the same kind and chill them suddenly the useful abstractions are probably something you can figure out between those helper methods.

Honestly that is the real fear of the left way of thinking. If you add a quantity, whether to cook and whether to chill parameters you end up with a hard API where certain combinations of parameters don't make sense.

Have a clean API and make the implementation as simple as is feasible. Reuse via functions when it makes sense but don't add them willy nilly.

Aka "it is a craft and you figure things out" as someone said in the comments here

> Suddenly, you realize that the dozens of functions __need to be called in specific orders__, and they are each only ever used once. So really what you're doing is forcing someone to know the magic order these functions are composed in order for them to be of any use.

That's where nested functions show their true utility. You get short linear logic because everything is in functions, but the functions are all local scope so you get to modify local scope with them, and because the functions are all named, it is easy to determine what is going on.

If you have dozens of functions that need to be called in specific orders, design and use a state machine and then use a dispatch function that orchestrates the state machine.

I'm very dubious of anyone resorting to "readability" as a justification.

What you're doing by breaking things into functions is trying to prevent it's eventual growth into a bug infested behemoth. In my experience, nearly every case where an area of a code base has become unmaintainable - it generally originates in a large, stateful piece of code that started in this fashion.

Every one who works in said area then usually has the option of either a) making it worse by adding another block to tweak it's behaviour, or b) start splitting it up and hope they don't break stuff.

I don't want to see the "how" every time I need to understand the "what". In fact, that is going to force me to parse extraneous detail, possibly for hundreds of lines, until I find the bit that actually needs to be changed.

> What you're doing by breaking things into functions is trying to prevent it's eventual growth into a bug infested behemoth

Not every piece of code grows into a bug-infested behemoth. A lot of code doesn't grow for years. We're biased to think that every piece of code needs to "scale", but the reality is that most of it doesn't.

Instead of trying to fix issues in advance you should build a culture where issues are identified and fixed as they come up.

This piece of code will be a pain to maintain when the team gets bigger? So fix it when it actually gets bigger. Create space for engineers to talk about their pains and give them time to address those. Don't assume you know all their future pains and fix them in advance.

> In my experience, nearly every case where an area of a code base has become unmaintainable - it generally originates in a large, stateful piece of code that started in this fashion

In my experience it gets even worse with tons of prematurely-abstracted functions. Identifying and fixing large blocks of code that are hard to maintain is way easier that identifying and fixing premature abstractions. If you have to choose between the two (and you typically do), you should always choose large blocks of code.

The great thing about big blocks of code is that their flaws are so obvious. Which means they are easy to fix when the time comes. The skill every team desperately needs is identifying when the time comes, not writing code that scales from scratch (which is simply impossible).

> but the idea doesn’t scale.

You are wrong here.

> this is one of those “programming is a craft” things, where experience helps you determine what is right in a situation.

You are right here.

The key insight on why giant linear functions are often more readable (and desirable) is because they allow you to keep more concepts/relationships simultaneously together as a single chunk without context switching which seems to aid our comprehension. An extreme proponent is Arthur Whitney (inventor of the K language) who writes very terse (almost incomprehensible to others) code so as to accommodate as much as possible in a single screen.

Two examples from my own experience;

1) I found reading/understanding/debugging a very large Windows message handler function (i.e. a WndProc with a giant switch statement containing all the business logic) far easier than the same application rewritten in Visual C++ where the message handlers were broken out into separate functions.

2) The sample code for a microcontroller showed an ADC usage example in two different ways; One with everything in the same file and another where the code was distributed across files eg. main.c/config.c/interrupts.c/timer.c/etc. Even though the LOC was <200 i found the second example hard to understand simply because of the context switch involved.

YEAH, but the moral that should be taken from that is not "it's always better to write huge, linear functions". Rather, "there are cases where huge, linear functions make sense because of the way the code needs to interact with things". Along the same lines, there are cases where breaking the code up into smaller functions, and calling them from the main function, makes more sense".

> Linear code is more readable

^ Wrong

> Linear code is sometimes more readable

^ Better

Not quite what i was trying to convey. Linear code actually has a lower burden on one's cognitive load and thus easier to comprehend. But of course it breaks down at certain sizes (variable) which is when it makes sense to partition it into smaller pieces and apply Abstraction etc.

See for example Cyclomatic Complexity - https://en.wikipedia.org/wiki/Cyclomatic_complexity

> Linear code actually has a lower burden on one's cognitive load and thus easier to comprehend. But of course it breaks down at certain sizes ...

I agree with this, but I think the combination of those two sentences winds up being "linear code is sometimes easier to comprehend, and sometimes not". The statement "linear code is easier to comprehend" is misleading. Your statement makes it seem like "at certain sizes" is the edge case; whereas, in my opinion, it's the only case that really matters. For a small enough block of code, "easier to comprehend" becomes a moot point.

> See for example Cyclomatic Complexity

I think that's only tangentially related. Cyclomatic Complexity deals with branching, which is somewhat orthogonal to refactoring out code to separate functions (though refactoring can make the branching easier to read, since it shows more in a smaller area).

> an extreme proponent is Arthur Whitney (inventor of the K language) who writes very terse (almost incomprehensible to others) code

But k has a small set of built-in commands and a built-in database; it was made for fast analysis of stock information, so with that you have everything you need and you use the same semantics. The only thing you need to know is the data structure and you can build whatever you need.

So in this way, it's very likely that, given two tables A + B and 'bunch of operations' X on A and 'bunch of operations Y' on B where Y depends on the result of X, and given the tasks to;

- create X' = X

- create XY' = X + Y

to implement XY without knowing X already exists rather than figure out X exists and reuse it.

The problem with not k (or programs written in similar style; it doesn't really matter what the programming language is), that we have learned to use the second style from the article, and, more extreme, to separate everything out in layers. You cannot even reach the data model without going through a layer (or more) of abstractions which makes it necessary not only to know the datamodel in detail but also find the matching findXinAandApplyWithYToB(). Where X & Y & A & B are often some kind of ambiguous and badly named entities. And then there is of course badly designed databases which is also quite the norm as far as we see, so there is a much lower data integrity which means that if you create something without checking all the code that touches it, that you might change something and the data becomes inconsistent.

I notice the same when working on systems built with stored procedures on MSSQL/Postgres; it is far quicker to oversee and (at least basically) understand the datamodel (even with 1000+ tables, which is rather normal for systems we work with) than it is to understand even a fraction of a, let's say Go, codebase. So when asked to do do a task XY', you are usually just not searching for X'; you are simply reading the data used in X & Y and whop up a procedure/query/whatever yourself. It's simply much faster as you have a restricted work surface; the model and sql (I know, you can use almost any language in postgres, but let's not here) and you can reason about them and the tasks at hand when you shut off internet and just use your sql workbench.

Nice post. If i understand you correctly; you are saying that K is specialized enough (my knowledge is only cursory here) that you can directly work with the data model easily rather than going through multiple layers of abstractions and hence linear code is normal. In other languages it may not be so easy to do and multiple layers of abstractions only make things harder to comprehend. True, IMO Abstraction should always follow Understanding of the Problem space and not some arbitrary dogma. What i find infuriating nowadays is "cargo culting" where people blindly follow something because they read it somewhere/listened to somebody without thinking through the motivations involved and whether it is applicable to their current problem. In other words "they don't think" for themselves. Examples are "OO is bad"(it is not), Agile/Scrum processes will magically solve all your PM problems(hell, no!), Using the latest library/framework/API/fad will magically make your system better (no!), etc. etc.

> If i understand you correctly; you are saying that K is specialized enough (my knowledge is only cursory here) that you can directly work with the data model easily rather than going through multiple layers of abstractions and hence linear code is normal.

Yes, it is often just easier to write the linear code than figure out if you can reuse anything because the space is small. I think a good 'feeling' for this is, if you need internet search/package managers/copilot etc for something or if you can just write working code sitting on a desert island, quite possibly on paper. For instance, for C, asm (arm/68k/z80/8080 and older intel), k and some others I can write working code like that for non-toy applications in specific domains. And, at least for me, those languages lend themselves very well for this linear programming. Incidentally, but not related, this is for me also the most enjoyable way of programming; I kind of really hate using libraries. That's also because we work in heavy regulatory areas where you cannot just add them; you have to inspect them and sign of on them and, of course, most of them are terrible...

Nice; you have found a niche work domain for yourself which you seem to enjoy.

May we all be so lucky :-)

PS: You might want to consider adding your contact info. to your profile.

> The key insight on why giant linear functions are often more readable (and desirable) is because they allow you to keep more concepts/relationships simultaneously together as a single chunk without context switching which seems to aid our comprehension.

The problem with giant linear functions is that those concepts get separated by sometimes thousands of lines. Separating out the high-level concepts vs the nitty-gritty details, putting the latter in functions that then get called to implement the high-level concepts, does in my experience in most cases a better job of keeping related things together.

See my other comment here : https://news.ycombinator.com/item?id=37522577

The issue is one of Policy vs. Mechanism - https://en.wikipedia.org/wiki/Separation_of_mechanism_and_po...

It is "Mechanism" which should be separated out and encapsulated while "Policy" (aka business logic) is what is better centralized as a linear (possibly large) function.

I have seen many instances where people just out of habbit factor out a lot of linear code that will never be reused into separate functions.

These pieces of code then often end up being private functions of a class. With state. Since they are private functions now, they are not really testable.

So now we got a lot of private functions that are only called once and typically modify side effect state. When these functions are grouped together with the caller, it is actually still a bit readable in simple cases.

But then after a while someone adds other functions in between the calling function and the factored out ones.

Now we have bits and pieces modifying different side effect state that no one knows if they are called from different places without getting a call graph or doing a search in the class file.

If you insist on making the code non-linear, I'd beg you to at least consider making these factored out private funcs inner funcs of the calling function if your language supports that. This makes it clear that these functions won't be called from anywhere else.

As with so many things in life, in a real codebase this is not an either/or, but an art of combining the two into something that stays readable and maintainable.

If the function was truly linear having a long function wouldn't be so bad. But it actually isn't, the example contains multiple branches!

Will people bother testing all of them? Or will they write a single test, pass in a pizza and just glance at it actually working? My guess is the latter, as testing multiple branches from outside is often tedious, vs testing smaller specialized functions.

> The example code is vey simplistic, so of course that linear code is more readable, but the idea doesn’t scale.

...that's basically why common sense and taste in programming is still required, it's not a purely mechanical task. That's also why I'm not entirely a fan of automatic code formatting tools, they don't understand the concept of nuance.