Bret Victor was definitely an influence when I sought to combine the visual graph-patching paradigm (for audio programming) with a repl-like experience on a more-powerful-than usual functional backend [1].

He does talk about real-time visualization of outputs. IDEs for building trees is definitely not where we want to go.

However, there's an aspect in there that I'm utterly convinced about: direct manipulation of program objects! The ability to grab hold of program objects, tweak and move them and see the response, almost like you had some cord or lever in hand that connects to the program, feels amazing. To me, it's the most amazing aspect of Sutherland's super early work [2]. Hancock's dissertation also deals with some of these things [3].

As you observed, we struggle with generalizing such direct manipulation. Especially the composition and abstraction of such components. That's the difference between a IDE to draw trees and a magic medium between thought and computing. Will we some day have the latter? I do not know.

1: https://kronoslang.io/ (It's all written by yours truly, including the website, WASM jit compiler and frontend, so please be gentle :)

2: https://blogs.commons.georgetown.edu/cctp-797-fall2013/archi...

3: https://dspace.mit.edu/handle/1721.1/61549