Most professions have a healthy respect for the base materials they work with no matter how high the abstractions and structures they build with it go. Artists know their paints, stone, metal, etc. Engineers know their melaterials as well. They build by taking the advantages of each material into consideration, not assuming that it's no longer relevant to their job because they get to just work in I-beams. Programmers would do well to adopt a healthy respect for their base materials, and it seems like often we don't.

I disagree with how your use of "just" here. It's common for programmers to dismiss the importance of syntax but syntax and notation are the interface and UX between the language semantics and your brain. It's no less important to get this right. There's a half-joke that Europe was able to rapidly advance in Calculus beyond Britain due to the superiority of Leibniz notation.

> healthy respect for their base materials

What's unique about computers is the theoretical guarantee that the base does not matter. Whether by lambda calculus, register machines or swarms of soldier crabs running from birds in specially designed enclosures, we're fine as long as we appropriately encode our instructions.

> bunch of branches and loops

You could also easily say it's just a bunch of state machines. We outsource tedious complexity and fine details to compiler abstractions. They track things for us that have analogues in logical deduction so that as long we follow their restrictions, we get a few guarantees. When say, writing asynchronous non-deterministic distributed programs, you'll need all the help you can get.

Even designing close to the machine (which most programs will not need) by paying attention to cache use, memory layout, branch divergence or using SIMD remain within the realm of abstractions.

This is a contradiction. Simplicity is obtained through abstractions. As an example, fifty years ago, 'goto' reigned supreme. Then structured 'if/else/for' took over, as the superior abstraction over 'goto'. Now use of 'goto', while being far simpler to implement and closer to the bare metal, is commonly derided by many programmers.

The long term trend in software is constantly increasing abstraction hiding mountains of complexity, which increases both simplicity and complexity. Writing print('hello world') is simple, but the compiler, OS, silicon, etc. that makes that simplicity possible is extremely complex.

It is a must, because decades of business requirement built on top each other without understanding the whole is complex. Writing a JIT-compiler that can routinely change between interpreting code and executing it natively, a database optimizing queries, a mathematical library using some fancy algorithm are all complex, in a way that is not reducible.

Complexity easily outgrowth even the whole of our mathematics, we can’t prove any non-trivial property of a program, halting problem, etc.

So all in all, no, we can only respect our “base materials” by finding the proper abstraction for the problem, as our base material is complexity itself. It might be for loops and ifs, but it very well be a DSL built on top of who knows how many layers, because at that abstraction level can we even start to map the problem domain to human consumable ideas.

In my experience programming programming with primitives and basic flow control operations frequently tends to be at least be order of magnitude faster than more complex state management paradigms. Compilers are very good at optimizing that style of code. Loops often get unrolled, the and the branch predictor is kept happy. A good compiler may use vector expressions.

In many cases with cold code it flat out doesn't matter, the CPU is plenty fast, but when it does matter, explicit if-and-for code absolutely mops the floor with the alternatives.

You can optimize one specific query (quite painstakingly, I believe) to beat a general db, but it is very far from clear that “for loops will beat higher level abstractions”, imo.

With my own set-up I can have my data in pre-calculated immutable tables that are memory mapped off disk. That means I can write lock-free code. My updates are large hour-long batch operations off a written journal.

For the lexicon I'm also rolling a special one-way compression scheme/hash function that guarantees uniqueness but only works due to quirks with the data.

General purpose databases can't do these things, because then they wouldn't be general purpose databases.

SQL databases perform really well on average given your query is relatively optimized, but they do not represent the pinnacle of performance. Any time you want the fastest most efficient code to deal with a particular use case, it pays enormous dividends to roll your own implementation that has domain awareness. It means you can cut corners general solutions simply can't.

I’m yet to see a secretary who could “return a new table state such that as if documents became a right fold as binding together a map of composition of signing and copy routines over documents” instead of “get these documents from the table, copy, sign and put them back in a binder”. This is a nonsense some of us want to believe in, but it is not true.

If the project had instead been designed to have less unnecessary state and “transitions” it would have been a lot easier to make changes.

All those ideas sound good by themselves but they are really bad for “defensive” coding. Good luck selling a project to refactor something when it’s going to take multiple people-years. Once you’ve made the mistake of committing to an inflexible design it’s either that, replace/ignore the thing, or deal with low productivity as long as the thing exists.

so was the chosen model the issue or choosing a state machine model at all?

Imperative code:

    Take A and B.
    Add C ml of Water.
    Stir for 2 minutes.
    If it thickened, add D grams of flour, else go back to stiring.

Imperative code quickly devolves to a point where it is no longer understandable by anyone but the developers who wrote it.

For problems where those are the right tools, sure. But they aren't the right tools for all problems any more than ifs and for loops are.