In my devops work, I've been using literate programming as part of my personal documentation. Org-mode gets a lot of hype here on HN, but I really get a lot of value out of Org, Org-Roam, and Org-Babel (org-babel is the bit that underpins the literate programming part of org)

Being able to see the code that generates certain tables in my older documentation really helps me to more quickly remember and use stuff I did months or years ago.

Example: The command to get a list of S3 buckets that have a particular attribute set. The command to do that might not make much sense by itself (say in a history grep) but with extra text describing what and why, I understand it faster.

I also use org-babel to do the same thing at work, most recently with convoluted AWS CLI commands with lots of JMESPath query syntax that I can never remember. I use it for personal documentation, but it can also can make for very nice looking documents that you can later share with your team during Sprint retrospectives, or perhaps convert to a Confluence page.

Yes exactly, export to HTML and paste into Confluence.

Although it is nice also to show off my Emacs setup during a screen share to a couple dozen people who don't know what it is. They treat the org-mode headlines, rearranging, todo items, schedules, etc, as some sort of magic. Org-babel with the results in a simple table kinda blew their minds. The number of project managers who want "my emacs setup" created for themselves was kinda funny, since they're all normally so focused on Microsoft Project or Jira.

It turned out that going with Org Mode was an excellent idea! I wrote another post about the changes I made last night: https://sachachua.com/blog/2020/06/having-fun-kerning-using-...

Basically, tables rock, especially when you can use rectangle commands to copy and paste kerning values between classes you have to split up, and generating a test webpage programmatically is fun too. :D

Awesome! I remember your website from way back when, it was nice to see it come up on HN again with some really cool content.

Hahaha, I'm only just now emerging from the fog of raising a small child. She keeps telling me not to stay up late at night, I keep staying up at night because I want to do stuff...

I'v always liked the idea of literate programming, but I was always wondering how it actually feels in practice. Do they usual editor/IDE features still work like in a non-literate environment? – I assume it might work in Emacs, but are there other editors that support this approach?

Thinking about it, is Jupyter Notebooks (and the like) a form of literate programming?

Thinking even more about it, I think I actually used (or tried to) literate programming in a course about Machine Learning using R and – if I remember correctly – LaTeX to generate the actual application and a PDF.

You could use https://github.com/dzop/emacs-jupyter to combine Org mode + Jupyter.

Here's an example emacs configuration:

  (use-package jupyter
    :after org
    :config
    (org-babel-do-load-languages 'org-babel-load-languages '((jupyter . t)))
    ;; default args for jupyter-python
    (setq org-babel-default-header-args:jupyter-python
          '((:results . "replace")
     (:async . "yes")
     (:session . "py")
     (:kernel . "python3"))))

After that, you could execute jupyter code block (Org Babel):

  #+begin_src jupyter-python
  1+1
  #+end_src
  
  #+RESULTS:
  : 2

You can use latex in org-mode, generate pictures, get nice git diff, export it to pdf, etc.

Just curious: That code could be run by just plain `#+begin_src python`. What does using Jupyter gain you?

Nothing inherent to Jupyter (it is not like Jupyter notebooks vs. Python REPL), just some niceties: - low latency (it feels more responsive) - interactive output (tqdm works) - how Python objects are displayed by default

Essentially a good implementation of Python code blocks would reimplement half of Jupyter anyway.

It’s extremely widely used in the R world. The RStudio IDE has extensive tooling for the R Markdown format. https://rmarkdown.rstudio.com/

We don’t use the term “literate programming” much anymore, the concept of “reproducible reports” resonates much more with R users. But the experience is much the same.

(Disclosure: employed at RStudio)

It was disappointing to me when I recently found out that I couldn’t create Jupyter notebooks with vim. The on-disk format is JSON.

It sounds silly, but when there’s a lot of work do in terms of writing the literature part, and then adding the actual code part, I’m much happier with vim than I am with the browser based editor.

(This is for producing literate programs for the purpose of teaching others, not for my own benefit.)

You should look into Jupytext, it might be just what you're after.

In my view Jupiter notebooks can be, but must not be, a form of literate programming. Many tutorials take the form of a single juypter-notebook exaplaining concepts with code interleaved, which i would argue is a form of literate programming. But my data-exploration notebooks, mostly code and an abundance of plots with sparse comments interleaved, don't really resemble anything i would call literate programming.

Do you mean "need not be" rather than "must not be"? "Must not be" means they shouldn't ever be.

Most of my Jupyter notebooks are just code, but I've got some with long Latex mathematical derivations of formulas that appear in the code. I would actually like to be able to have that kind of thing in comments within the code itself. I don't do literate programming really though.

ah sry, i am german and frequently do this wrong since it's the exact opposite in german.

> I'v always liked the idea of literate programming, but I was always wondering how it actually feels in practice. Do they usual editor/IDE features still work like in a non-literate environment? – I assume it might work in Emacs, but are there other editors that support this approach?

In Org Mode, you get all the syntax highlightings and keybindings for whatever language's mode if you do literate programming with it.

What you won't get are things like being able to integrate your compilation failures, etc. As an example, if you write something that fails to compile, Emacs will not quickly be able to take you to the line that caused the failure. Similarly, things like linting, etc typically break.

Frustratingly, if you get an exception (e.g. "KeyError" in Python), it will not show the result of any print statements, so debugging is a bit painful.

I'm sure someone can solve these problems, but I don't know that anyone has.

I figured out that if you use the : session keyword, you can still see the prints before the exception. I tend to use something like :session "argh".

They can in theory but making it work in practice seems a little harder. For example I tried this

https://github.com/jingtaozf/literate-clojure

but ran into this

https://github.com/polymode/poly-org/issues/20

Either some IDE like functionality doesn't work or syntax highlighting doesn't work after you go through additional complexity compared to simply starting a project.

It was really nice being able to break the program down into steps like generate the template, cut glyphs, etc. and be able to execute parts without worrying about how to reevaluate all the relevant parts of the code or what kind of functions I want to define. I'm beginning to really like noweb.