I agree that visual programming seems to be one of the great failed dreams - lots of academics and practitioners tried it, and it just doesn't seem to work. Although, the crop of XML-mappers (mapforce, biztalk mapper) do represent simple functions in a visual language, and seem to own that market. I agree it's not serious programming, but more "coding for non-coders" (and it's marketed as such). Yet, it has a place - this time, for business analysts. There are non-programmers everywhere who need to do some kind of coding - and creating a safe, useful tool for them is worthwhile. And maybe even for coders sometimes, if (and only if) the task really doesn't need serious programming. Examples: make; regular expressions; paths; printf/scanf; JSON; HTML (though these mini-languages aren't visual, they also aren't suitable for serious programming). I'd add SQL, bash, sed and awk, but they are powerful enough; just awkward.
I was intrigued by your suggestion that one can't abstract a visual language. While there are templates, maybe one can do better. The anchors and widgets changing the placement on resize are a kind of abstraction - but you're right, you can't make your own abstractions. No one's done it - it doesn't mean there isn't a way to do it. Maybe there is.
However, for interaction, I think you're pretty right. Although some standard styles of interaction, and standard ways of generalizing/abstracting them could be captured, you can't have create new styles or generalizations using this system. It isn't powerful enough.
> Maybe we could create a better visual version of abstraction, but I don't feel it's the most promising direction to take in order to make abstraction more powerful.
The thing about visual programming is that it helps people. I agree it's unlikely to make abstraction more powerful (though with pure research you don't know what you'll find). I think it would be really cool to create visual abstraction, that is usable and intuitive, when it seemed impossible.
PS: just a thought: if an "interaction" a kind of mapping where by if you do X, then Y happens, could that not be represented in the same visual way that XML mappers use (above)? Functional abstraction could also be represented as a visual mapping - but it certainly is not a natural or intuitive form for it.
What's important, though, is that these people don't build UIs. First and foremost they manipulate or analyse data. The data is at the center. The important thing is a simple data model that follows the closure principle (filter tables and you get tables, filter text and you get text). The UI is there to support this simple data model and it has to be similarly reductionist I think. Placing arbitrary UI widgets on forms is not the way to do this.
Yes, more fundamental research please :-) Maybe machine learning could be used to let users correct the system's guesses rather than initiating some action. That would go further in the direction that graphical UIs have begun, which is to let users select from what is offered to them and react instead of making them give commands to the system.
Or maybe some of the thinking that has gone into computer games would be useful, I don't know.
I like your point about users "reacting" to an limited range of choices, explicitly stated (and surely using domain specific terminology). It's a grammar you don't have to internalize, with libraries you don't have to know about, or know the search terms to find term. It's very "discoverable", and the limited and clear range of choices minimizes the information content (increasing the probability of obtaining the desired outcome). I'm sure it would also prune the tree by omitting choice combinations that don't make sense.
I think people don't like the system guessing for you, like Office's Clippy. Maybe it's OK if the user asks for the guess - like getting a list of search results; or vim/bash completion; or suggestions (google, Stackoverflow)
LabView is pretty powerful. Commercial, expensive, often annoying -- but pretty powerful. Integrates very cleanly w/ Visual Studio for text-based programming, too.
Yes there is abstraction in templates. Templates typically let you indicate (visually or otherwise) what the static parts look like and where the dynamic parts go. That's it. We have done that forever. There's no progress. Templates don't describe interaction and that's what UIs are there for and it's the growing part of UI design.
Tool vendors have tried for many years to describe the interaction parts visually as well. But interaction is not inherently visual. A condition is not visual. A data transformation is not visual. Even the difference between two visual states is not visual itself.
It's a logical abstraction that manipulates visual objects. Mixing logic that manipulates visuals with the visuals themselves is not useful, and in terms of established software design principles it violates the separation of concerns.
Painting logic with a GUI builder is like entering a sine function by drawing squiggly lines on the screen.
Maybe we could create a better visual version of abstraction, but I don't feel it's the most promising direction to take in order to make abstraction more powerful.