What assertion, exactly? That the personal computing stack can be implemented in 20K LOC? They've already achieved the bulk of it, so that claim no longer qualifies as incredible.
Pooh-poohing this staggering achievement because they didn't reinvent personal computing while they were at it really takes the cake for moving the bar. If someone had gotten all the world's recorded music on a thumb drive in 1970, would you have complained that they didn't write the music themselves?
No, the assertion from the article that "99% or even 99.9% of the effort that goes into creating a large software system is not productive."
While it's very interesting and frankly astonishing that they've been able to reimplement things that are fairly well understood in 100x less code, this is not at all the same as proving the assertion that these things could have been originally done in 100x less code. I hope that it's true that they could have been -- it's exciting to think that it might be true -- but it doesn't seem to be a claim that STEPS proves, or tries to, from the explicit goal of reusing concepts and designs from other OSes.
Second, sorry, but I'm not buying this response at all. These guys have done something that all conventional thinking about software complexity would say was impossible. They've refuted something fundamental about universal practice. Seems to me the only open-minded response is to be flabbergasted and ask what else we need to question.
To say (I'm exaggerating how it sounds) "Oh, well, sure, they did that impossible, but this other impossible over here, that's the one that really matters" just seems like rationalization to me.
Oh and also, it's at least three orders of magnitude, not two.
"Alan Kay speculates in this talk that 99% or even 99.9% of the effort that goes into creating a large software system is not productive."
This is the claim.
Without in any way diminishing the astonishing accomplishments of the STEPS team, let's remember that they've spent (are spending) five years to write 20K lines of code. This doesn't sound like the level of effort is 1000 (or 100, or possibly even 10) times less for what they've accomplished than it would have been had they implemented exactly the same functionality in, say, Java. Would the Java code have been 1000 times bigger? It seems quite likely that it would have! But the bulkiness of the end product's source code doesn't determine the amount of effort that went into building it.
Could it be that they spent four and a half years figuring this new system out, and a few weeks writing the 20K lines of source? It could; I haven't read most of their papers. If that was the case, then that's what needs to be pointed out to refute the assumption that they're spending as much or almost as much effort on the end product (but with astonishingly little source code required to specify it).
What we do know from general research (at least, I've read this many times - anyone know the source?) is that programmer effort tends to be about the same per line of code. That's one reason everyone always cites for higher-level languages being better. Maybe that result doesn't hold up here, but it seems like a reasonable baseline.
If you can do a second draft with 10 times less code, then at least 90% of the first draft's complexity was accidental. By definition, so to speak. The question of avoiding that accidental complexity at the first try is a separate question.
The VPRI did find particular abstractions, embodied in their various Domain Specific Languages. Implicit parallelism (not concurrency) was tested in one of their languages (Nile), for 2.5 drawing. The problem was embarrassingly parallel to begin with, but they only had to modify the runtime to parallelize it (30 times faster with 40 cores). I also believe they have some form of implicit concurrency.
Also, it doesn't matter if they didn't invent some new ground-breaking abstraction. Because they just showed that applying the right old ideas in a systematic ways can have a tremendous effect. Now the industry just have to wake up.
But is that not the more important question? -- how much effort it takes to design software (which surely is the gist of Brooks' observation).
If the final code is much smaller, yet it took as long to design/write, what has been gained? Something, certainly (easier to manipulate, reuse), and furthermore there must be a fairly good relation between size and effort, yet it is the effort to create that is the key matter here.
If you accept the claims of
> People regularly say they translated their system from language A to language B and got 10-100 times code-size reductions.
then it's clearly false at least for first systems that
> there is essential complexity in software, and it is the dominant factor.
But there are various things involved so I am certainly not sure.
It does not square with at least some research and what Brooks most perspicaciously noted in 'No Silver Bullet' -- that there is essential complexity in software, and it is the dominant factor. That really seems very opposed to anything anywhere near 100-fold general improvements.
Looking at the VPRI Steps report of 2011-04, prima facie, overall, they seem to be producing a 'second-draft' of current practice, not a revolutionary general technique.
People regularly say they translated their system from language A to language B and got 10-100 times code-size reductions. This seems essentially what STEPS is doing: when you look at something again, you can see how to improve it and do a much better job. But the improvement comes not from the general aspect -- the language B -- but from understanding better the particular project because you already had a first-draft.
This can still be valuable work though, since it is about general things we all use. Second-drafts are valuable, and ones of important things even more so.
But we should probably not infer that they are coming up with a revolutionary general way to write new software.
Substantially because large scale software cannot be designed: it must be evolved. We only know what to focus a second-draft around because we muddled together the first draft and it evolved and survived. You cannot write a second-draft first.
Having said that, if they find particular new abstractions, they could be very valuable. If, for example, VPRI came up with a neat way to express implicit concurrency, maybe, that might possibly be comparable to inventing/developing gargage-collection in common programming languages -- certainly useful and valuable.