Part of your point still stands: the `for' loop is harder to read and write specifically because it is more general. `for' loops can do all kinds of wacky things.
The first kind is exhaustiveness. Being general by accounting for all special cases. That one is complicated, and rarely worth doing beforehand.
The second kind is genericity. Being general by ignoring all the special cases. That one is often simpler and more solid than any special case. Parametric polymorphism (or generics, or templates) are like that: ignoring the specifics of the parameters limits what you can do with them, making the generic thing simpler.
The layer at which you choose to apply exhaustiveness or genericity is irrelevant in my opinion. C++ often leans towards the exhaustiveness end of the spectrum despite being a language, for instance.
I think that maps to your distinction, except that I don't believe in 'exhaustiveness'. The state space of a program isn't some fixed thing for most programs. It evolves and grows in response to what we want it to do, which dimensions we choose to generalize along and where we stay stable. I think it's equally reasonable to view the evolution of special cases as equally exhaustive at every point, it's the boundaries of the state space (requirements) that is growing in strange ways.
Not that I'm trying to be all dogmatic about referential transparency- more like, without it, is there a large benefit to functional style vs. procedural style?
From all I've heard people in impure languages have begun to embrace the notion of restricting side effects voluntarily.
In fact, the more design you do, and the more you're exposed to, the better you can get at design. And the better the design, the easier it is to understand.
An "ugly generalized abstraction" is pretty much by definition not a good abstraction. Moreover, a good abstraction is simple. It's easy to understand and it's easy to manipulate. [1]
But you need to practice to get better, so it may take a few ugly monstrosities before you can start creating masterpieces.
[1] Granted 99% of the code I've encountered in Java qualifies as ugly abstractions; Java's nature seems to encourage poor abstractions. Or attract poor developers. Or both. But even in Java it's possible to create beautiful and easy to understand abstractions.
I was deliberately avoiding the d-word because it is vague enough that I'm not sure there's anything useful around there. The good designers are all good at designing within some domain. Jonathan Ivy, Christopher Alexander, Ralph Lauren, they all had their specialty. You wouldn't ask Ralph Lauren to design a rocket ship. Software is even wider a set of domains than meatspace. So I'm not sure you get better at design by 'practicing design'. That's just a set of vague words like 'simple', 'good', 'beautiful', 'abstraction', 'decoupled', 'coherent', 'cohesive', etc., etc. I think you get better at design by better understanding a domain. You get better at design by avoiding premature design.
(Now, the Forth way isn't the one I would choose to be easy to understand: http://yosefk.com/blog/my-history-with-forth-stack-machines..... But that's a separate story..)