I haven't read the rest of the article, but I got to that phrase and my disposition instantly turned from "huh, this is interesting" to "snake oil", whether deserved or not.

Reminds me of the recent "One weird trick!" article [0] and lines like, "the simple solution dietitians don't want you to know!"

Content aside, that kind of thing immediately sets off alarms for me.

[0] http://www.slate.com/articles/business/moneybox/2013/07/how_...

For me it's the fact that I've seen this whole spiel a lot of times already, in EVERY DETAIL. Visual programming will save us all. Nonprogrammers should be able to program. Suppressed, blah blah blah. For something so revolutionary, it sure is some awfully-well-trodden ground. And I don't just mean that this is an old paradigm, I mean this spiel is well-trodden ground.

In a nutshell, it doesn't work. Using Fred Brook's venerable terminology, the essential complexity of programming is too difficult for nonprogrammers. Even if you entirely get rid of the accidental complexity... and visual layout does not do that, it's full of incidental complexity involved in the layout itself... you still can't get non-programmers into a programming situation, and trying to get them into a concurrent programming situation is just sheer insanity.

And we know it doesn't work because it has been tried. Over and over. Ad naseum, as in, yeah, I'm sick of hearing these ideas as The Answer as if nobody has ever heard of these before. Tell me what's different about your solution and all the other people who have tried this.

I agree with you about the visual-layout-makes-programming-easy spiel, but it does seem like there might be a little more to flow-based programming than that. At least a few smart people seem to take it seriously (e.g. on LtU), and dataflow as a computational paradigm has a lot of research behind it and hasn't seen a whole lot of implementation. It's not out of the question that someone could apply it in a new way that would make some classes of application easier to build. I'm open to that and would find it interesting. But after having looked repeatedly for an intelligible technical explanation of FBP, I'm starting to think that the idea is either a trivial repackaging of a well-known model (like Unix pipes, or CSP with directed graphs) or is better shelved with the snake oil. The fact that it's typically accompanied by rather sensational language is not a point in its favour.

Edit: it's too bad that the programming model is getting lumped in here with the visual programming schtick. Those are two different things. (Of course the former would get no PR without the latter.)

To be clear, I fine with their attempt to make any product they intend to make. I'm very skeptical about it due to the fact that it's been tried a lot, but hey, their money, their choice, maybe they'll make it stick, maybe they'll make a niche market work (LabVIEW, etc), who knows. It's the rhetoric I'm objecting to. Which bodes poorly, IMHO; if they really believe this gibberish then it's more likely they'll make some very bad business decisions, such as resisting finding a good niche in favor of trying to take over too much stuff. There's a lot of prior art to learn from here, and if they've learned as little of it as that marketing spiel implies, their odds of success are very, very low.

I consider this an instance of stream programming.

Do you think of stream programming as a kind of dataflow programming? If not, what's the difference?

I suppose "dataflow" is an overloaded term, and the fact that its various meanings have quite a bit in common doesn't help.

Definitely, yes. In section 2 of this paper, I tried to explain how our language is related to what are typically called "synchronous dataflow" languages, or just SDF languages: http://www.scott-a-s.com/files/pact2012.pdf

Slide 11 of this talk makes the same point, but with less words: http://www.scott-a-s.com/files/debs2013_tutorial_slides.pdf

Data flow when used as a converse of control flow is quite well defined. Are you thinking in terms of how data flows across the system or in terms of control? Stream programming usually uses data flow at the top level and control flow in the atomic components connected together (we can have compound components that involve more data flow). Once you add control lines to your system, however, you are basically re-encoding explicit control flow as data flow.

I might have been unclear; what I meant was that the different programming languages or models that people call "dataflow programming" (spreadsheets, Lucid, Oz, FBP, etc.) have a lot of differences between them; and on top of those there is dataflow analysis in compilers, dataflow hardware architectures, and so on.

Data-flow is a broad term that doesn't mean one specific thing, but it is usually used correctly in what it means; e.g. data-flow analysis contrasts nicely with control-flow analysis. I don't find it confusing.

I believe FBP primarily deals with data flow, not control flow, but the "flow" in FBP seems to be left intentionally ambiguous.

This reminds me of my experiences learning programming and web development. For a while I used Drupal for everything, because it allowed me to 'program without programming'. The views module (now in core drupal, or slated to be, I think), is essentially a visual query builder.

What I discovered after a while, was that there was a tiny sweet spot where it was nice to do complicated things without writing code, but that it had huge drawbacks. Queries became ridiculous, for one.

And very soon, the stuff I'd build using the Views module was 'complicated' enough conceptually that a normal SQL query would not only be much faster, but easier and quicker to write. The fact that this query could be version-controlled and didn't live in a database was nice too...

It's interesting that business processes are often described as input, process, output. Whether a human or a machine accomplishes the process is another discussion. But what is not unique is that designing the process, it's all still another process. The downside is that not knowing how to effectively scale a process is the harder problem that may require different, more scalable tools later.

Move fast and break things. Replace what breaks with stronger pieces until it doesn't.

Agreed. It's easier to teach someone python than it is to teach a programmer visual programming.

> Reminds me of the recent "One weird trick!" article [0] and lines like, "the simple solution dietitians don't want you to know!"

Computer scientists hate him!

Flow based programming has a funny way of being suppressed... this has been tried many, many times. Some examples off the top of my head: Scratch, Windows Workflow Foundation, Arc Model Builder, etc. All of the marketing for NoFlo seems to be targeted at non-programmers who think "I could code anything if I could just understand the syntax!" Their main video basically says "Don't you wish you could finally stop paying those sneaky developers who act like everything takes longer than it really does? Now you can!"

> Their main video basically says "Don't you wish you could finally stop paying those sneaky developers who act like everything takes longer than it really does? Now you can!"

That one point really gave me pause as well. As I posted the other day:

-----

"If your bottom-line is dependent on code, you know the angst of asking a programmer, “How long will {{ insert feature }} take?” You know programmers can jargon-speak their way to your consent and when everything comes crashing down, you are to blame. NoFlo frees your businesses from a labyrinth of text files so you can get-a-grip on what-the-hell-is-going-on!"

If you tell people that NoFlo is the solution to prevent programmers from "jargon-speaking" their way to your doom then... Sorry, that's just a lie with a topping of polarization.

The way to prevent programmers messing up your business is to not hire shit programmers and manage good programmers properly. Not hiring the programmers with the fanciest visualization.

https://news.ycombinator.com/item?id=6198748

Flow based programming is actually heavily used in music technology, where it actually is a good paradigm for the domain. Max/MSP and its open source cousin Pure Data are interesting, and have dedicated if small communities of not-exactly-programmers using them for real, creative expression.

But most of the proposed use cases are clearly bullshit, and using these languages/programs makes it clear why: organizing code in a single dimension is already hard... organizing it properly and sensibly in two dimensions much more so, as well as a number of other issues with order and tracking race conditions. For anyone who wants to really evaluate this technology, I would give these products a try.

Really though, functional programming and OOP both have sufficient support for data flows that it isn't really an issue for programmers.

Ha ha ha, if only :).

But on serious note, with all the brains we have, we should've created some tools to automate things, progress is too slow. I think we can do way more.

> we should've created some tools to automate things, progress is too slow

Most developer software is abstractions and tools to automate things. Tasks that were hard in 1995 are trivial today; most web developers have probably never had to think about the vast majority of things that web developers in 1995 concerned themselves with, because our libraries and frameworks and tools take care of them.

The complexity of the trade is ever-increasing, though, and so we need new tools and new libraries and new frameworks to deal with that increasing complexity. This is the biggest place where visual programming falls apart - it's fine at dealing with programming as it existed when it was released, but programming is not a static discipline, and evolves wildly and rapidly. Non-sentient tools just simply can't keep up.

True. But it was not all linear progress. In 1995 I had Turbo Pascal for example, that had ide with decent editor, fantastic debugger and profiler. And don't even start me on context sensitive help. What I have today is way crappier or non-existant.

LightTable is tool that one guy started as a futuristic tool, yet it is what we had before.

Complexity of our work is ever increasing and fun definitely never stops, I wish tools were better so we can enjoy languages more. As tools augment our ability to do work, having excellent tools would allow you to do more and be easier.

Plenty of tools do what you describe, and far better than Turbo Pascal did in the 90s. Try Visual Studio or IntelliJ, for example.

When you can't perform static analysis (which is very difficult to do effectively in dynamically-typed languages), many of those tools you are talking about are just not possible.

Programming languages evolve slowly because they're more like mathematical notation than technology, so said someone smart whose identity I can't recall. Lots of smart people have been thinking for more than a half century about the following problem: How to elegantly tell a computer what to do? With all that brainpower invested in this problem, the low-hanging fruit is gone. It's understandable that progress is slow now. I doubt there are great programming paradigms waiting in the wings, waiting on technological advances like faster CPUs, more memory, or better graphics.

After staggering expenditures of effort by some of the world's sharpest minds over more than half a century, I doubt there are any more major breakthroughs awaiting us in the field of programming languages. (Unless AI starts to play a major role, in which case I couldn't predict how programming would change.)

Just what exactly do you think programming _is_?

Scratch is not Flow-based programming. It's semi visual programming for kids, it only eliminate syntax error for kids but still produces code-like output.

Here's one example of reimplementing Jekyll (static site generation) using NoFlo: http://bergie.iki.fi/blog/noflo-jekyll/

Thank you for providing an example.

I look at that and I see a couple of things. First, this is procedural programming straight out of my old, old COBOL books. Batch processing - read some files, process the data, output it. That graph is pretty, but no different from the structured analysis and design that was forced on me in the mid 80s.

That stuff was a nightmare to work with. No real search, endless 2D layout issues, no way to reuse, a very little bit of information conveyed in a lot of space, no way to debug, and so on. And, if your program is anything but batch/data flow oriented, good luck.

I see maybe 100-300 lines of JSON in the image. Or Lua. Whatever. A modest amount of paste code to tie already written components together in in/out chains. Now, I get that your diagram is more easily 'groked' than JSON, and that is a benefit. But what if I want to search for things? Use awesome tools like sed to batch change stuff? Perform static analysis? All that graphical stuff just falls apart. Oh, perhaps there is some DOM, and I access that, but then I'm right back into the incredible power and expressiveness of text.

More importantly, I'd like to see the flow diagrams of all those nodes you are connecting together. I am prepared for egg on my face here, but are those written in NoFlo, or text? I'm guessing the latter.

So, how is this different than LabView? I completely understand the value of being able to quickly plug together pre-existing components when you have a tiny interface (you just have in/out nodes, basically, in that diagram). A small solution for a small problem, and I don't mean that dismissively.

Despite the article's claim, and as many have already said, this stuff has been around for decades. There are definitely places for it. But having lived through it, I tell you SA/SD died a very deserved death.

Sorry, this isn't directed at you, you were just the person kind enough to provide an asked for example.

The other major thing that sticks out for me is that that graph is just a high level, block diagram of the program. Well, "just" not meant to be dismissive. The original article talks about the horrors of programming where changing one thing means changing 10 other components. They are not describing programming, they are describing hacking spaghetti code with no design. If that is the way you program, or the way the code base you are working on is coded, then I can see why these flow diagrams seem so revolutionary - a design is being done, and software is broken up into discrete components. That's pretty fundamental SW engineering. But the magic is not in the 2D visual display, but in the idea of small components that do one thing and that are not intimately tied to other components. So hey, if the tool helps you learn or use that method, good, I guess. How you reasonably scale it up to anything large is beyond me (I've done entire avionics systems in data flow diagrams, and it ain't pretty at that scale, believe me).

> this is procedural programming straight out of my old, old COBOL books. Batch processing - read some files, process the data, output it.

Modularizing systems into separate, standardized units has a tendency to look like this (or like shell scripting). Is this a huge problem?

Not at all. The point is that the visual code doesn't differ much from the textual code in terms of architecture and complexity.

I think there is some broad confusion over "flow-based programming" implying "non-programmers using a graph editor to connect things". So I'm going to ignore that completely in favor of talking about flow-based programming for programmers.

Here's an example I have experience with:

The GRL research language [1] is a dataflow language for behavior-based robotics. It's a real programming language that batch compiles down to C code that runs in O(1) time and space (it's not Turing complete). It escapes from the "spaghetti code" problem you mention by providing an abstraction of a procedure as a compile-time rewrite rule that transforms the dataflow graph (for Lisp hackers, every procedure is basically a macro that can be passed around by value at compile-time).

(FWIW, graphical editors can actually be a useful and pleasant way to construct throwaway flow graphs, but aren't particularly useful for developing new abstractions (i.e. dataflow procedures or data types). They run into the spaghetti code problem you mention, by which the only way to reuse a chunk of functionality is copy-paste)

The system has been used for things like programming actual robots, Half Life bots [2], and motor control for interactive game-like thingies [3]. What it's really great at is managing stateful information over time. Things like moving averages, low pass filters, and other types of state machines (flip-flops, "just true" constructs, etc) all become first-class values in your programming language.

So how could this be used in a more flexible environment that isn't ideological about dataflow programming? While dataflow graphs may be limited in their expressiveness, we can use them to encapsulate many types of state machines, particularly those that change over time or in response to stimulus. Under this model, every self-contained flow graph becomes a thunk whose input values are determined by evaluating arbitrary expressions in the host language (and can be dynamically instantiated). For implementations of this technique, see [3]. Finally, [4] is an implementation of the stateful time-based dataflow constructs I wrote for a game in C#, which power things like motor behaviors, input processing, collision avoidance, animations, and interpolations. As a bonus, you can add events on top of these for a simple implementation of functional reactive programming.

So do you want to build a web app in a dataflow language? Maybe not. Do you want flow-based constructs in your programming language? Depending on what you're building, the answer could range of "Not really" to "I can't live without them."

[1] http://www.cs.northwestern.edu/~ian/grl-paper.pdf

[2] http://www.cs.northwestern.edu/~gdunham/flexbot/manual/grl_f...

[3] http://www.aaai.org/ocs/index.php/AIIDE/AIIDE11/paper/view/4...

[4] https://bitbucket.org/leifaffles/chad/src/d6a13c2315cce41dba...

I also work with a distributed embedded system that uses dataflow. The Von Neumann architecture as a revolution of course. But, in a way, it forgot about the natural concurrency of the electronic circuit and how powerful and simple that can be. Dataflow programming can bring some of that back. I remember reading Alan Kay saying something about how programmers should go back and work with a plugboard for a while. Maybe that is what he meant. Its possible to 'program' something with just wires and simple discrete components and that can liberate your thinking.

Yes, I think this is the main insight of the "revolutionary" part of FBP. It's not there to make everything look like a circuit board, but to be the template for the parts of software that are well-suited for the paradigm - which, if factored out properly, could be a lot of them, more than we use today.

The tricky part isn't in whether the abstraction works, but how we go about introducing it. And the downfall of most visual languages is in burdening themselves with too much power, which I don't think FBP is guilty of. It's evolved from production systems that were still coded in textual forms, but architected - on paper - with the diagrams.

Yup, that's because the NoFlo way isn't going to be the paradigm-shifting development; it's just graphical sugar on the same old thing.

If it'll ever become big then it'll be more like what Javelin does by adding FRP to Clojure's persistent data structures [1]

[1] http://www.infoq.com/presentations/ClojureScript-Javelin

Completely agree - I see links to frameworks and tools in the article but none to projects and products made with them.

Ah, the secret cabal of computer scientists is at it again.

I read up to that quote and decided it was bullshit.

NoFlo is basically a clone of Windows Workflow Foundation but implemented in a worse language

WWF: http://bit.ly/16YNGFK

Meh that looks like some UML class diagram to me (at first glance). Just need to have a "generate code from diagram" in this case. :?

>release something that uses this paradigm and let's me get something done and I'll use it

>This allows programmers to write individual functions that execute one step at a time, but route data between these functions asynchronously and concurrently

These concepts are not new. Concurrent programming is all about data pipelining, this is basically describing a gimped version of Erlang.. with pictures. The obvious practical advantages of a textual language make it laughable.

edit for clarity.