I'm an unabashed Clojure fanboy and will talk the language up to anyone who will listen, but one thing that I've been thinking about lately is that good judgment on abstraction use is necessary for using such powerful tools. I don't think the place to tackle it is by being "smart enough to understand," but rather by being "disciplined enough to not abstract sometimes." The exciting thing is that discipline doesn't require you to be super smart, just to think it's worth caring about.

Also half-baked, but I think it's something.

It's not that much harder to enforce than it is to enforce conventions in less powerful languages, and the convenience of working in a well-designed language is so nice.

I think there's a misunderstanding. What I'm calling decompression has nothing to do with digging into the implementation of a particular abstraction. What I mean is merely tracking down all the indirections in the public interface and breaking them down to their concrete meaning in a specific case.

Some languages allow for a lot more moving parts than others. For instance, in Java obj.otherName means that otherName is an instance variable (or theoretically a class variable) and access time is guaranteed to be constant (tbd: caveats). It's not redefinable as it is in C#, Python or Swift. So there's one less thing to track down but also less flexibility.

Or take an extreme example like Go's range loop. It's defined only for a handful of builtin data structures. When you see a range loop, you know what it does without following any further indirections. It can be extremely annoying and create a lot of friction when you have to work with custom data structures. But it can make other people's code easier to read than in other languages.

Or take something like this:

  a == b

So transforming an expression's possible meanings into its concrete meaning is what I call decompression. There may not be a need for looking anything other than public interfaces.

Also, you're talking about abstractions purely from the perspective of users. But we are often creators and maintainers of abstractions as well when we model a particular problem.

You might be interested in this book, Patterns of Software, which brings up the issue that inheritance in OOP isn't about reuse but about compression and how the implications of that are why it's not had that much success in following through on the "reuse" promise. http://dreamsongs.com/Files/PatternsOfSoftware.pdf

Consider JVM and Java: that you can write Java seemingly without any knowledge about JVM is an illusion - you already learned a lot about how things like JVM work under the hood when first learning programming[0]. Learning more about JVM itself lets you write more efficient code and debug better.

Consider functions: it's near-impossible to contain all the information necessary to use a non-trivial function in its header[1]. That's why good code contains comments and other forms of documentation describing the abstraction in more details. Even then, it's not always enough - sometimes it's really much easier to understand what you need by reading the source[2].

Consider any appliance - be it a car, or a radio, or a dishwasher. You can use it to it interface, to some extent at least. But knowing what's going on under the literal hood really does help with use, and especially helps when something goes wrong. If you don't know what's hidden under the abstraction layer, any failure will likely be incomprehensible to you and leave you helpless.

--

[0] - In theory, it might be possible to learn some programming without learning anything about hardware or hardware-emulating abstractions. In reality, I've never seen it or heard of it, and I suspect that the simplest mental model of code execution is isomorphic to somewhat simplified computer or virtual machine.

[1] - Function name + name of its arguments + types of its arguments and return value, if available.

[2] - Then again, implementation code may not capture the entire abstraction either. That's why good code often features comments inside the implementation, explaining the rationale behind some of the less obvious code parts.

[1]: Once a certain point has been reached; theory does predict diminishing returns.

I have read quite a few studies, and I always come away thinking that the methods used are inadequate to support any conclusions at all.

But does that mean there is no causal effect of programming language choice? I don't know and I'm not making any claims either way.

I was talking about abstraction in one person's mind vs abstraction shared between a group a people. And what I said could actually be the reason why the choice of programming language has little effect (if that is the case)

I'm also not making definitive claims, but we can't ignore observations, either. While small effects can appear or disappear depending on methodology, big effects are easy to find and hard to hide, especially in an environment with strong selection pressures. Therefore, the most likely explanation to why no big effect has been detected -- either in studies or in industry -- is that there isn't one, and that should at least be enough to make it the working hypothesis. So before explaining effects that have yet to be detected we should establish that they exist at all, something we have so far failed to do.

There may well be a small effect, and there could still be a large effect, although the chance of that is small unless we have a really good explanation to why we haven't found it.

1) It is inconsistent with the personal experience of most developers. Few would claim that C is just as productive and safe as some garbage collected language (provided that both are suitable for the task at hand), or that a SQL group by statement is not more productive to write than the equivalent procedural code.

2) The methods used to study the subject are completely unfit for purpose.

3) It is inconsistent with the few results that seem at least somewhat well supported because they are simple enough to measure, such as the constant bug count per line of code.

I disagree. I've been programming for about 30 years now, and most reports I've received -- both from programmers and managers -- are that language doesn't matter; certainly not much. Carefully selected personal "feeling" report are worthless (especially in biased forums), as that would produce even more evidence that homeopathy is effective than that programming languages make a difference.

> Few would claim that C is just as productive and safe as some garbage collected language

I agree, but that's where the many caveats come in. First, my argument is more one of diminishing returns (following Brooks's theory). I.e., languages make an increasingly small difference. C may have been a big improvement over assemby, and Java may have been a smaller but still quite big improvement over C, but now differences are quite small.

Indeed, the effect of GC and memory safety vs C was one that was almost immediately detected by industry (once mature, performant, etc.) and triggered a huge shift. We do not see such shifts now. Companies move among mainstream languages, among non-mainstream languages, and between the two classes in a way that does not suggest a big effect at all.

> or that a SQL group by statement is not more productive to write than the equivalent procedural code.

That's a completely different matter. I don't think anyone believes that writing a 1MLOC program in SQL is easier than writing a similar one in Java or Python, even if the Python/Java one is 1MLOC, and the SQL one is 200KLOC. When the program is small, there can be large differences, but that's a completely different problem (and also explained by theory).

> The methods used to study the subject are completely unfit for purpose.

I disagree, but it doesn't matter. Big effects are easy to find and hard to hide. And in an environment with strong selective pressures, they're found even when no studies at all are conducted.

> It is inconsistent with the few results that seem at least somewhat well supported because they are simple enough to measure, such as the constant bug count per line of code.

It is not. First, the difference in LOC is not as big as PL fans claim (it can be big for small programs, not large ones). Second, you may want to review those results.

I don't necessarily disagree, but I'm a bit skeptical about the way in which you use the concept of diminishing returns. If I'm not mistaken then Brooks used diminishing returns in relation to headcount. And this is in fact how economists use the term as well - adding quantitatively more of one production factor.

But that doesn't apply to qualitative changes like introducing new production methods and tools, which is what programming languages are. New programming languages are not simply "more programming language".

If we agree that there have been observable effects of new programming languages in the past, then we cannot exclude the possibility of the same happening in the future.

>First, the difference in LOC is not as big as PL fans claim

What PL fans claim is a bit of a fluffy benchmark, but there are undoubtedly significant differences in LOC, certainly significant enough to make an economic difference.

>I've been programming for about 30 years now...

Shockingly, me too :)

Oh, I'm referring to "No Silver Bullet" and diminishing returns in the sense that languages at best help with "accidental complexity", so the less of it there is, the less languages can help.

> New programming languages are not simply "more programming language".

But there are inflexible theoretical limitations on their utility (even without Brooks). We know the effect of different programming constructs on expressive compression and reasoning costs. The closer we get to the limit, the less benefit we can have.

> then we cannot exclude the possibility of the same happening in the future.

I'm mostly pointing out that it's not happening at present. I'm less sanguine about future advances because of various limitations (and, BTW, Brooks's predictions were called overly pessimistic by PL fans at the time and it turned out they were too optimistic), but I won't rule out another major breakthrough, or possibly two.

> but there are undoubtedly significant differences in LOC, certainly significant enough to make an economic difference.

I don't think I agree with your first assertion (although that depends on what we mean by "significant" here), and I certainly disagree with your second. We simply have not been able to detect or induce such an effect. Here, too, there are theoretical results showing that increased expressiveness cannot lead to cheaper reasoning, which is not an obvious result even in the worst case (because there are far fewer "compressible" programs than non compressible ones).

>Here, too, there are theoretical results showing that increased expressiveness cannot lead to cheaper reasoning, which is not an obvious result even in the worst case (because there are far fewer "compressible" programs than non compressible ones).

I find that very interesting. Do you have a source for it?

* http://www.lsv.fr/Publis/PAPERS/PDF/Sch-aiml02.pdf

* http://www.lsv.fr/Publis/PAPERS/PDF/DLS-jcss-param.pdf

* https://www.cis.upenn.edu/~alur/Zohar03.pdf

What's important to put those results in context for those who are not familiar with the subject is that, in the context of complexity theory, the "model checking problem" does not refer to the complexity of a particular model checker algorithm, but to the inherent complexity of the problem of deciding whether a program M satisfies some property 𝜑 (i.e. whether M is a model of 𝜑 in the formal logic sense). The "model checking problem" is the closest to a mathematical description of what we usually mean when we talk of "reasoning about a program."

I've summarized these results and more here: https://pron.github.io/posts/correctness-and-complexity

Do you mind sharing a source for this? It seems trivial to generate a programming language (Brainf*ck) that performs very poorly.

A more likely statement: all commonly used languages are more or less on par.