Finding the right level of abstraction is basically the one and only skill that matters for a software engineer IMO. I've got about 15 years of experience in programming and it's still something I'm actively working on.

I disagree with the general take that "adding complex abstractions to code do NOT make it more reliable". Good use of abstraction makes code easier to write, easier to maintain and easier to extend. Good use of abstraction can make code more concise and more regular while at the same time allowing better diagnostics when you do something wrong.

As a quick example: a generic JSON serialization library that can work with any type is probably a lot nicer to use that one that requires manual reimplementation for every class in your program. It'll be more complicated to write but it's probably well worth it in the end. Similarly a logging system that can abstract over several backends and log levels depending on the environment probably beats having a bunch of if/else every time you want to log a message.

But one needs to remember the old saying: debugging code is harder than writing it, so if you write code that's as smart as you're capable of producing then by definition you're not smart enough to debug it.

You are completely right: finding "THE RIGHT" level of abstraction is extremely important. Without abstractions we would be writing code in Assembly.

This is the reason I used the term "overly complex abstractions", not just "abstractions" in general, of course. Overly complex meaning it's a failure to use "the right abstraction" where you instead go for something much more complex than the proble called for... this is what I understand most people in this thread are referring to when they refer to previous Scala projects they've worked on.

When I was young I though that good writers are the ones who write such complex sentences that nobody can understand. Later I learned that good writers are clear and understandable.

I used to write complex code because I found it elegant. Now I write simple code because I enjoy breaking down a complex problem into its simplest form, which for me means I fully understand the problem.

I think the writing metaphor is a good one; we are writing source code for other humans to read after all.

We are also writing code so we can read it ourselves in a few months or years and still understand what it does!

I agree 100%, and I always bring it back to one simple observation that has been true of almost all the projects I've worked on: the limiting resource that programmers work with is their own time and brainpower. To my mind, much of the discipline of programming is centered around making frugal use of that scarce resource.

I still think programmers should work with powerful tools, because "dumb" tools often force you to express things in convoluted ways, and you don't need a powerful language to make a huge mess. (Java proves that you only need single inheritance to produce virtually unlimited amounts of spaghettiness in real-world projects. The difference between Java and Scala is not that Scala enables teams to produce epic piles of FP crap, it's that we long ago stopped being shocked when teams produce epic piles of OO crap.)

I think the programmers doing terrible things with Scala would be doing terrible things in any language, given the chance. What they need is hands-on technical management. When they have a wild idea, they need to be walked through an appropriate engineering decision-making process. They need somebody with the authority to tell them, "Ha ha, yeah, that design with etcd and Kafka and the robot space lasers would friggin' rock, how cool would that be, but on the other hand, we could just stick a REST API in front of a database and you'd be done in two weeks, so let's compare pros and cons."

That's a common falsehood though.

It does not matter how "simple" each line of code is.

What matters is how "simple" the whole application/solution is.

You can argue that you can write very simple code in Assembler - all instructions are very clearly defined and very simple (well, maybe not anymore in CISC...). Someone can learn them all in a day. The problem is just that now you end up with a lot of code and overall that will be hard to maintain.

Compare it to Javascript: certainly not the language where code is most maintainable or simple, but the end-result is easier to maintain then assembler.

In the end, the choice of programming language is like a the choice of a compression-tool like gzip. The more complex, the more difficult to learn and use (setting library size and various compression settings) but the output will be smaller compared to a simple tool or no compression (Assembler) while containing the same amount of information.

Counter-point:

I write lots of F#, and I find that computation expressions (a fancy version of do-notation) makes my code more "dumb" and "simple", despite being an advanced "FP" feature. This is because it pushes the complexity from my business logic and into library code.

With computation expressions:

    async {
      let! foos = fetchFoos

      for foo in foos do
        do! launchFoo foo

      do! clearFoos
    }

Without computation expressions (this is probably wrong but you get the idea):

    Async.bind 
      fetchFoos
      (fun foos -> 
        Async.bind 
          (
            foos
            |> Seq.fold (Async.bind (fun foo -> launchFoo foo)) (Async.just ())
          )
          (fun () -> clearFoos)
      )

Notice how my fancy non-blocking code reads like straight-forward blocking code?

I could not manage complexity without this feature.

You should always reach for the simplest solution first (and Haoyi has a great set of guidelines about that: https://www.lihaoyi.com/post/StrategicScalaStylePrincipleofL... ). The point of those complex FP abstractions is to let you keep writing dumb code even when you want to do something fiddly (such as async operations with error recovery). But don't cargo-cult the complex solution if you don't have the complex problem! A plain function whose result is a plain value should be expressed that way.

> adding complex abstractions to code do NOT make it more reliable.

Sometimes false. If the (more) complex abstractions are in well-tested libraries that haven't been written solely for your code.

> But [complex abstractions] do make [your code] much harder to modify,

Not necessarily. You can separate concerns; you can isolate changes; and you hopefully reduce the amount of code you've written yourself, significantly even.

> [complex abstractions] do make [your code] much harder to understand

This is as likely false as it is true.

> [complex abstractions] do make [your code] much harder to fix

Again, this very much depends. If you're using a well-tested, widely-used external library with those abstractions, it may well be easier to fix your own code.

> the other lesson from FP: overly complex abstractions are also very bad

Lisp, in its persistent failure to achieve world domination, has been teaching us this lesson for over 60 years now!

Infinitely powerful languages are fun and, yes, powerful. But as soon as the project has more than one or two people in it, they also result in runaway complexity that will hurt the product far more than any gains in expressibility.

“Complex abstraction” is a contradiction in terms since an abstraction is a relation A -> B where B needs to be simpler than A.

From the client's point of view that may be true; abstraction should be the process of pushing complexity down. But in this case I think we're talking about code that has inadvertently introduced unnecessary complexity in an attempt to be hyper-generic.

i think it depends if you need it, rust for example is quite complex, but it solves a class of issue that other language have trouble solving while maintaining speed