A state transfer diagram would probably be more useful as a view on the code (similar to how the Smalltalk browser is a simple graphical representation/layout of code. Or indeed any IDE with advanced code folding (show me the class name and public methods only).

But there's ways to mix the two: Python doctests is one. The lp approach is to "escape" the code, not the comments. I really do think some richer data structure than flat text files is needed - simple multi-/hyper-text seems like a minimum. Not only links (most ides have this: hoover to display help, click to goto definition etc).

Why so few seem to successfully add vector graphics and diagrams I don't know. I guess jupyter / ipython might be a rare breath of air. But who really wants to maintain 10k locs of code and 15k locs of tests in ipython notebooks?

We need something more (lively kernel might be more interesting here - webdav for persistence Web browser for run/edit/view - I'm not sure if the required power Canberra realised any simpler).

I personally only see the utility of the notebook as a demonstration tool of a finished product. Kinda goes to your comment on the notebooks and code maintenance. It's a good teaching or presentation tool but not (IMO) a development tool.

The main question is if there's any fundamental reason for why we can't have both: a rich, versioned, distributed data store for our logic and data - and a number of views that allow us to inspect and modify it.

The best example I'm aware of is probably Smalltalk coupled with monticello version control and a solid object database like gemstone/s (I'm not aware of a real open/free workalike for the last part).

That's not not say relational or logic databases aren't useful - but it's more difficult to manage the data, query code and program code in one integrated system if the database takes the form of an external server. That said, Ms visual studio and the SQL db gui does a pretty good job of presenting a somewhat coherent environment.

I used to be strongly in the Linux/Unix camp - thinking that the datastore we call the filsystem is a good abstraction. But I've come to believe that even the strong legacy of user familiarity isn't a good enough reason to stick with it. Even if we were a little bit more serious and at least went all plan9 - rather than the half-hearted state of Linux/Unix (everything is a file, or a database file, or a binary file or an archive of a filsystem or...).

That said a "filsystem" might very well still be a decent building block for a higher level system (with decent search, for example).

But yeah, I don't think jupyter is the be all, end all of development. But it is an interesting way to move legacy programming environments forward in a low friction manner.

For the OODB, perhaps Squeak's Magma? http://wiki.squeak.org/squeak/2665

I think the literate in literate programming is too formal. Maybe it should have been conversational programming.

The comments should be the things that you would tell a new team member during a pair programming session. You assume she knows what the syntax of the language is, but you don't assume she knows the business intent behind the code or the history of trial and error that led to its current state.

> Now, if someone lets me put images in comments, that would be amazing. Good for state transition diagrams and random scribbles

That's exactly what JetBrains MPS (https://www.jetbrains.com/mps/) does. Even better: the state transition diagram is the code. Some of my colleagues are playing with it - not sure if it's used in production yet.

You didn't solve the OPs problem, you just moved it around.

In his case, his algo said X, and his code did Y.. very easy to see the mistake.

In your case, lets add a comment

// Note: PCI-DSS requirements apply below

Now 3 years later, the law changes and the requirements do not apply. So you are at the same situation. Code does X, comments say Y. Which is right?

Not a very easy to keep comments and code in sync

That's sort of a different problem. I often think of comments as "why" and code as "how" (or "what" if you're declarative). If how breaks, then the code breaks, so you have immediate feedback. But if why breaks - like if your reasons for writing it that way no longer apply - then it's impossible to recognize immediately. If there were instead a way to codify the assumptions behind why, such that the why statements would break when the assumptions become false, that would be interesting.

But at any rate, why is relevant, and code doesn't express the why.

My approach to this is that the comment can almost always be turned into a text. A test class called FooPCIDSSCompatibility with tests for all the bits both defines things more strongly and will start failing if you ever break it (either accidentally or because it is no longer required). Either way you need to update either code or test to get through the build process.

Comments can be useful for annotating algorithms or referencing stack overflow though.

That comment I'd interpret as the requirements must be considered in this region; so if the law changes, the requirements still apply, it's just that the code needs to be updated.

Requirements are usually exogenous to the code.

"PCI-DSS requirements apply" isn't adequate in a literate program for the same reason that "must work correctly" isn't adequate. Explaining the relevant requirements in detail is a crucial goal.

What about this:

1. Specs&design docs and code should be in separate files, because I believe the separation of concerns should be applied there. That's indeed the opposite of literate programming.

2. There should be two-way links between documentation and code: in the code, one should have links to the spec; and from the spec, one should have links to the code.

3. If the specs or the code changes, those links should be displayed in a different way to warn the reader that things are potentially not in sync. How to do that: check if the links point to the latest version. The maintainers have to update the links to remove the warnings.

With respect, I disagree.

Specification and design/implementation are not separate concerns. They are dual.

A sufficiently detailed specification is an implementation. Prolog does this (and Eve has a very similar feel).

As engineers, we traditionally work declaratively at the top of the "V" and imperatively at the bottom of the "V" -- but the reasons for this are largely historical/cultural.

We could (in theory) carry out the analysis/refinement process using entirely declarative notation.

The problem domain has primacy. Analysis separates problem-domain concerns and the duality takes care of the translation between problem and solution domains.

(OK -- so this is basically just a reiteration of the thesis of good old-fashioned AI -- that with a sufficiently powerful theorem prover and a sufficiently large and detailed knowledge base -- solutions will just pop out of a largely mechanical analysis process -- and I'm pretty sure this isn't at all trendy right now ... so I should relegate this to the "thinking out loud" bucket ...)

I'm fairly sure he was attempting to agree with the OPs problem.

Most coding editors (even the "deprecated since the 70ies" Vim) shall let you follow links you put in comments. For someone who likes to have as much code as possible on a single screen, this is the best option.

+1 for images. It's been my main reason for looking into tools like doxygen

Agreed, so then most programming systems are wrong if things are not obvious?

Well, it's just a code smell. Smelly code isn't necessarily wrong, it's just more likely to be wrong.

Sometimes a piece of code needs to be highly optimized to the point of being basically unreadable, at that point it's probably worth adding explanatory comments that you would normally avoid.