Edit: added the parenthetical comment
A comparison would be how desire paths form and are often turned into real paths.
I like your parenthetical observation too!
Sometimes I've got a specific goal because I'm trying to solve a problem. There have been exactly two times, I think, that I was not able to fix a bug in my TeX macros by reading The TeXbook and it was necessary to go ahead and read TeX: The Program to find out exactly how a feature worked. In each case I was able to find my answer in 15 minutes because TeX: The Program is so well documented and cross-referenced. That, in itself, is an eye-opener--the fact that a program can be so organized and so documented, so indexed, that you can find something quickly.
The other thing I learned from it is how a master programmer organizes data structures, how he organizes the code so as to make it easier to read. Knuth carefully laid out TeX: The Program so you could almost read it as a novel or something. You could read it in a linear pass. You'd probably want to do some jumping back and forth as you encountered various things. Of course, it was an enormous amount of work on his part, which is why very few programs have been done that way.
[1] link to the page (337) in Google Books https://books.google.ca/books?id=2kMIqdfyT8kC&pg=PA337
I really think there's something in the 'jumping back and forth' of the reader that could be leveraged, like if there was a way of recording, distributing, aggregating, analysing it etc.
Computers & Typesetting, Volumes A-E Boxed Set by Donald E. Knuth -- I'll read anything that Don Knuth writes, and he has written quite a bit, including Surreal Numbers and 3:16 Bible Texts Illuminated as well as the famous Art of Computer Programming series. But this five-volume set is my favorite. The two volumes titled (and sold separately) as The TeXbook and The METAFONT book are well-known, but what I really recommend to you are TeX: The Program and METAFONT: The Program, because these are simply the best-written, best-documented, best-debugged programs of their size ever published. They reward careful study.
I've tried to read the source for TeX: The Program (I actually have the boxed set he mentions), but was impeded by my lack of knowledge of TeX. I keep meaning to get back to it.
[1] Guy's list in archive.org: http://web.archive.org/web/20080422183814/http://www.bookpoo...
a) Each layer puts more important stuff up top. So you can start skimming from the first layer, but you don't have to read all the way down each layer, just get a sense of what it provides and why.
b) There's not much emphasis on lengthy comments (which would get linear). Instead just the order in which code is presented does a lot of heavy lifting.
c) The tangled code is intended to be readable (unlike Knuth's original tools), so I often jump down into it when I feel the need. Error messages and debugger support do show lines in the original literate sources, however.
Tangential, but reading your website ("I'm working on ways to better convey the global structure of programs"), I've been talking a lot with a visually impaired researcher who works in Auditory Display. We've been talking about designing interactive audio displays to offer overviews of things like webpages and code.
I'll definitely keep your work as a reference if that turns into a project!
Yes, I write all my personal projects in human order. To me human order is when the flow control of human consumption and computer execution are most closely aligned. I attempt to achieve this with depth and order.
1) Have a giant library that defines your library or application.
2) Inside the giant function nest child functions for the primary tasks in the order with which they will execute. In the case of the following example the options evaluation is first, followed by the lexer/parser is the second child task, followed by various code presentation tasks, and finally by an analysis task. http://prettydiff.com/lib/jspretty.js
3) Break the child tasks down, as necessary, into reusable components.
4) I also believe a reference should always be declared before it used, which can dictate the order of reference declaration.
My opinion is that when the flow control of the application is unclear you are wasting time during maintenance. You may not know where a problem is occurring, but if the flow of the application is immediately clear from reading the code you immediately know where to start, where to step to next, and where to stop. There is minimal guessing and it doesn't require breakpoints to figure it out.
Source code:
Is it possible to use the vis tool on other codebases?
https://godoc.org/e8vm.io/e8vm/dagvis
it is possible to layout any DAG (directed graph with no circles).
The dependency graph of golang programs are generated by this package:
https://godoc.org/e8vm.io/tools/godep
I have not very well packaged the drawing part (sorry), but the essentials are here:
https://github.com/e8vm/shanhu/blob/master/web/js/dag.js
it is using d3.
" Peter Norvig 6 July 2016 at 11:47 I think the problem with Literate Programming is that assumes there is a single best order of presentation of the explanation. I agree that the order imposed by the compiler is not always best, but different readers have different purposes. You don’t read documentation like a novel, cover to cover. You read the parts that you need for the task(s) you want to do now. What would be ideal is a tool to help construct such paths for each reader, just-in-time; not a tool that makes the author choose a single pth for all readers."
Has anyone attempted something like this?
I've heard you can do transclusion in org-mode which might be a starting point
Edit: Some initial ideas:
- Code can be deconstructed into blocks, and where the code is not self-documenting, prose can be added. Or even visualisations etc if a block is conceptually tricky to grok. You could even have MOOC-style validations to verify reader understanding for each block.
- Some kind of topology of the blocks should be generated based on how they interact and how they are conceptually related
- The author creates a few 'starting points' for different audiences, e.g. 'if you've used X before, start at Y'
- From there next blocks to read are auto-suggested to the user. A map or network diagram of all blocks is also provided so the reader can chart their progress and see where the 'big ideas' lie.
Edit 2: This also really reminds me of Bret Victor's 'Humane Representation of Thought' lecture (https://vimeo.com/115154289, 45:40) where he says there is a conflict between code being an engineering specification and an authored work meant to be read by humans.