News for hackers. Stuff that matters.
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by lifthrasiir on Wednesday April 10, 2019 @02:43AM (#19622402)
After considering two or three dozens of different "lightweight" markup languages (i.e. not flavors) and designing a few (plus at least three failed attempts to improve Markdown), I'm now almost convinced that Scribble [1] or Perl POD [2] have one of the best possible designs. Not that they are optimal, but they are definitely heading at the correct direction.

They are extremely generic: every Scribble syntax is prefixed `@`, every POD construct is either `letter<...>` or `=word`. And yet they are surprisingly unobstructive and readable enough. Compare with BBCode, which is also pretty generic but too annoying.

They are readily extensible and need no special one-off syntaxes for most things. Yes, using asterisks or underscores for emphasis is handy, but how about strike-throughs? Underlines? Non-emphasis italics (e.g. species name)? Language tags? Abbreviations? Do they nest? Do they work in any context? Why not adding some harmless options to the existing constructs? Syntax extensibility should be one of the foremost concerns of markup language design and they got it right.

Edit: I should stress that these concerns are only valid for writings. For casual comments, less is better (ideally no-follow links and verbatim texts should suffice).

[1] https://docs.racket-lang.org/scribble/

[2] https://perldoc.perl.org/perlpod.html

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by astrobe_ on Wednesday April 10, 2019 @04:52AM (#19622927)
Reading the comments here, it feels like people miss the whole point of markdown.

Markdown's raison d'être is, "if you follow these conventions when doing ASCII art formatting, the my program can transform this to HTML or whatever other format".

The conventions being mostly the ones used in the pre-HTML email era, like underlining headings with a bunch of ======.

Orgmode is completely different to that. Orgmode is an "outliner/GTD" at heart, but this aspect has been buried under a ton of features.

Then if you want "prawfishunal qvalitat" advanced formatting, typesetting and layout control what you really want is something like Latex. Yet another whole different thing.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by hyperion2010 on Wednesday April 10, 2019 @05:41AM (#19623135)
The thing about org mode is that it has everything you need to do the full fancy latex typesetting without loosing the ability to export to other formats the way you do if you write raw latex. Of course there are times when you have no choice but to duplicate a block if you are doing something really fancy, but that is a fairly rare occurrence.
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by lifthrasiir on Wednesday April 10, 2019 @04:57AM (#19622963)
But Markdown was advertised as "a text-to-HTML conversion tool for web writers" [1]. The raison d'être as you stated is not an end goal but a possible benefit. Gruber may have thought that the extensibility of Markdown is done by raw HTML; now we know that's not enough.

[1] https://daringfireball.net/projects/markdown/

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by omaranto on Wednesday April 10, 2019 @11:44AM (#19625810)
Of course Org Mode is completely different, because it can do tons of other stuff. But among it's many features, it also has lightweight markup syntax about which you could claim "if you follow these conventions when doing ASCII art formatting, the my program can transform this to HTML or whatever other format". And the conventions in Org Mode are also email-like in the sense that they don't look out of place in something meant to be plain text. (Maybe finding some large email corpus could answer which syntax is more email-like between Org Mode and Markdown, and maybe Markdown wins on that front, but my point is both syntaxes are lightweight for the most commonly needed markup.)
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by klibertp on Wednesday April 10, 2019 @03:57PM (#19628216)
> Orgmode is completely different to that.

Org has just much larger scope than Md. The whole feature set of Markdown covers a tiny fraction of Org features.

NOTE: my use case is writing high-quality, good looking technical and user manual-like documentation.

Org-the-format is very close semantically to ReST - most of Org constructs have a direct equivalent in ReST, and vice versa - rather than to Markdown. And Org-the-system is very similar to Sphinx (the documentation/article/paper creation tool in Python) or Scribble. They are functionally equivalent - headless Emacs in Docker with Org/Babel and Sphinx - and it's unfortunately very hard to justify choosing the former over the latter. I know - I just tried this past half a year and failed, even though I could count on a lot of goodwill from my co-workers :(

(I'm ignoring the agenda, timelog, notes, spreadsheet, and all the kitchen sinks which also come with Org because they tend to be of private use, and I'm focusing on professional context)

The situation changes drastically, of course, if you do use Emacs. The Emacs' UI for Org, which is yet another thing from Org-the-format and Org-the-(headless)-executable-exporting-to-html/pdf, is incredibly powerful. Examples of features: storing, pasting and inserting and following links, footnotes, images. Structural modification of the document without hassle (no more missing text after reordering a list!). Ability to edit all code and examples in their corresponding modes (with syntax highlighting, autocomplete, etc.) Ability to run a single chosen code block with a key-press, inserting or updating (or not) the results into document. Support for editing tables - rearranging columns and rows. Latex, PlantUML, and dozens of other languages and tools integration... These are all very nice, very powerful features - and that's on top on whatever conveniences you have already configured in your Emacs, too.

Unfortunately, without the UI, Org is no more useful than Sphinx. On the other hand, no one's going to switch to another editor just to edit the docs, nor should they be forced to.

I like Org. I store my notes and some todo lists in it, I enjoy Babel (support for IPython/Jupyter notebook style development but polyglot), I maintain some parts of my blog in Org (long and unwieldy otherwise lists nested to a few levels). I cannot, however, see any reason to introduce Org as a tool for my team at work, unless they all see the light and come over to Emacs side :)

PS. a bit off-topic but I need to vent a bit: the API of Org is just awful! Functions for traversing the structure of the document are both in outline and org modes, and it's hard to know which is where and why. There are myriads of ways to get the current element, none of which is simple. A lot of the processing is still based on text search and substitution, instead of AST. It's not trivial to go from source to AST, either. I found at least 2 bugs in macro expansion code. I wanted to syntax-highlight the results of a code block (JSON was returned) and I had to hack around in header/keyword parsing for this. To be honest - some of my enthusiasm vanished after the experience :(

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by fouc on Wednesday April 10, 2019 @06:42AM (#19623399)
Those are not lightweight markup languages at all?

The main point of markdown is that it is supposed to look like a normal & readable text document in its own right. The transformation to html isn't absolutely necessary for readability.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by neilv on Wednesday April 10, 2019 @07:55AM (#19623722)
I'm leaning towards agreement with you, especially for embedded API docs in code.

I've been using Scribble for embedded API docs for years,[1] and the formatted result looks great, and is very powerful and extensible (Scribble is actually a better LaTeX), but it's not nearly as readable in code as it could be.

I think I actually preferred my earlier three-comment-character-Texinfo format I had before, and I even wrote a converter to Scribble for it (so that it would look idiomatic), except Texinfo format just couldn't handle Racket types.

Now that I've put Racket on hold while I Rust a bit, in the interests of job-hunting, I'm happy to see Rust is using Markdown (coincidentally, also with the three-comment-character prefix).

[1] Here's one formatted example, everything coming from the code files and package metadata, no separate documentation files: https://www.neilvandyke.org/racket/roomba/

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Crinus on Wednesday April 10, 2019 @05:01AM (#19622982)
I do not have anything online at the moment to show, but a small document processor i wrote some time ago used a @-based syntax which was essentially a tree of named nodes with attributes - essentially the same thing as XML, but with a simpler syntax that was something like:

    @Foo; is a standalone empty node named Foo
    @Foo[aa] has a "aa" attribute without a value
    @Foo[bb=cc] has a "bb" attribute with "cc" value
    @Foo[dd="hello, world",ee] has "dd" attribute with "hello, world" value and "ee" attribute without value
    @Foo{blah} has a "blah" text child
    @Foo{blah @Bar;} has "blah" and Bar children
    @Foo Bar shortcut for @Foo{Bar}
    @Foo{!{...}!} has ... as unprocessed content
So you can type something like @P{This is a @I nice @Link[target=foo]{example} of @TT{code} @Smile;.} which would be sort of equivalent to XML <P>This is a <I>nice</I> <Link target="foo">example</Link> of <TT>code</TT> <Smile/></P> (but with less repetition, especially for longer documents).

In my processor the tree is passed to a script that does the actual handling by going through the nodes, children, etc and creating the output text (HTML or whatever) - the processor itself has no knowledge of specific node names, any meaning lies in the scripts.

My biggest issue with the above though is that i do not like editing markup by hand, so at some point i want to replace all the above with a GUI tool that does the same job but in a more graphical way without the need for (visible to the user) markup syntax (so the above example would be edited as just text but with the "nice", "example" and "code" bits having representational styling).

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by smartmic on Wednesday April 10, 2019 @06:36AM (#19623366)
Looks a little bit like the Lout typesetting system on a first glance.

https://en.wikipedia.org/wiki/Lout_(software)

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Crinus on Wednesday April 10, 2019 @06:55AM (#19623447)
Yes, it was mainly inspired by Lout (although i think there is a much older system that used a similar syntax - and GNU TexInfo also uses something similar, but i didn't knew about that at the time), however IIRC Lout doesn't use a generic syntax.
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by neilv on Wednesday April 10, 2019 @08:02AM (#19623768)
In a case of great minds think alike (you, Eli Barzilay, and Matthew Flatt), this (especially the square brackets and curly braces purposes) is remarkably similar to what Racket ended up using, for the @-reader syntax that's used by Scribble document processing: https://docs.racket-lang.org/scribble/reader.html#%28part._....

I made an Emacs mode for it: https://www.neilvandyke.org/scribble-emacs/

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by enriquto on Wednesday April 10, 2019 @06:25AM (#19623311)
having to close paragraphs is useless. It moves to the user of the markup a concern that is only of interest to the implementor. That should we exactly the other way round.

I prefer html5, where you can leave tags open as if there's no tomorrow.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Crinus on Wednesday April 10, 2019 @06:59AM (#19623458)
This is just the way i have the scripts setup since it makes more sense for me for paragraphs to be full nodes that contain their children content, but it is trivial to make empty P nodes act as paragraph separators.
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by enriquto on Wednesday April 10, 2019 @08:42AM (#19624023)
It is not a trivial matter when you are writing the markup manually. If you require to open and close paragraphs it becomes very cumbersome for no real gain (just a tiny amount of convenience for the programmer of the parser).

The ideal case is TeX, where paragraphs are separated by a blank line.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Crinus on Wednesday April 10, 2019 @10:18AM (#19624861)
With trivial i meant that it is trivial to have a script that makes @P; a separator (or start marker) if you prefer to work that way. I made the system script-driven with a generic syntax exactly because i didn't want to hardcode any specific way of working with the nodes.

As for TeX, i personally dislike markup languages where whitespace is treated in a special way since this is source code, which often needs some style of its own and sometimes that depends on context.

To me seeing where a paragraph - or any other "long" element - begins and ends can be helpful and if anything, i consider the lack of syntax for this in my markup as a negative (but i couldn't come up with a clean and unambiguous syntax for adding an optional "closing" bit). I compensate for that using comments (which look like @# Blah until the end of line), but it is a workaround.

And also FWIW i prefer to have the markup map to the underlying tree of nodes clearly than introduce "magic" that in some cases can make things harder even for writers.

Keep in mind though that this is for long(ish) documents, like a large tutorial, guide, manual, etc not short comments like this one here. For anything that is around the size of a common blog post or shorter, something more adhoc like Markdown is probably better.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by enriquto on Wednesday April 10, 2019 @10:26AM (#19624950)
> As for TeX, i personally dislike markup languages where whitespace is treated in a special way since this is source code

just to nitpick: except for APL, white space is treated in a special way in all programming languages to separate tokens :)

Using it to separate text (words and paragraphs) makes perfect sense in this context.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Crinus on Wednesday April 10, 2019 @10:55AM (#19625292)
Perhaps i used a wrong description, but i meant having special cases/multiple uses and specifically about text markup languages. If every other element of the document tree, like sections, links, emphasis, references, images, figures, etc is marked using explicit syntax then using whitespace for paragraphs is inconsistent and can often be misleading since this will also need to be context sensitive as you do not want paragraphs to be inserted in every node that can have children.

But this is obviously a stylistic preference as to me that sounds too much trouble for no gain at all - which is basically your response above :-P.

But as i wrote in my original message, i'm not a big fan of these plaintext formats and i'd rather work using a "WYSIWYM" editor that uses these nodes behind the scenes instead of having the user write the markup by hand. But that is obviously much harder to implement (and if the tool handles the nodes for you the syntax doesn't matter at all anyway). Still, to me that is the ideal:-).

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by enriquto on Wednesday April 10, 2019 @11:13AM (#19625490)
> But as i wrote in my original message, i'm not a big fan of these plaintext formats and i'd rather work using a "WYSIWYM" editor

you will pry plain text editors from my cold, dead hands!

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by perlgeek on Wednesday April 10, 2019 @04:24AM (#19622813)
I'm a big fan of Perl myself, but when writing POD, I occasionally stumble about the C<...> for inline code.

The problem is that you have to take care that there is no > in the inline code, and if there is, you have to switch to C<< ... >>. With Markdown's `...`, this happened far less often to me.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by lifthrasiir on Wednesday April 10, 2019 @04:32AM (#19622844)
You are right that it is suboptimal. (I do think that Scribble's choice of braces is not too bad in this regard.) On the other hand, I can't readily type backquotes in my mobile phone or recent macOS [1] so there is a trade-off :-)

[1] Annoyingly enough, macOS Sierra and later maps a backquote to a Won sign in Korean keyboard layouts.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by elcritch on Wednesday April 10, 2019 @05:02AM (#19622986)
Never read scribble before but I kinda like it. It reminds me of the good parts of latex. As in @action vs \action. Though I don’t like the table syntax as much from a WYSIWYG level (E.g. it doesn’t look table-ly) but for complex tables it don’t matter much. Can you readily extend scribble with Racket and how’s the install/usage story?
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Eugeleo on Thursday April 11, 2019 @06:31PM (#19639942)
I know we're talking about docs right now, which is where Scribble really shines. And when it comes to blogging/prose, I'd recommend Pollen[1], which is a writing system based on scribble. It is fairly lightweight and extensible with Racket (which is by the way actually a pretty awesome language).

I'm writing my college notes in it and couldn't be happier. The dev is very responsive on mailing lists and would gladly help you out if you have some trouble initially. There is a pretty thorough getting started guide on the linked page. There's a few blogs written in Pollen and even a few books (Ever heard about Flatland? Or Practical Typography?).

[1] http://pollenpub.com

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by lifthrasiir on Wednesday April 10, 2019 @05:06AM (#19623004)
I think it is in the default Racket install. And yes, that's a problem; it is actually a Racket code behind the scene so it is fundamentally tied to Racket (that can be good or bad depending on your use case, and some strength points can be attributed to that).
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Ultimatt on Wednesday April 10, 2019 @08:13AM (#19623832)
You might want to check out the Perl 6 extensions to POD https://docs.perl6.org/language/pod
Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by fxfan on Wednesday April 10, 2019 @12:36PM (#19626359)
I haven't tried dozens but I really really wanted to like asciidoc and rst. But they both require me to type n number of '=' for title string of length n. I confess I fail to see the utility in this kind of design.

I'm still looking (Scribble seems anything but lightweight!). Perlpod probably looks perfect but I have to see more.

Is there a markup that can do numbered lists?

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by ben509 on Wednesday April 10, 2019 @05:56PM (#19629172)
I feel the same way about RST; really wanted to like it, fought with it, gave up because Sphinx consistently seems to make simple things hard.

Regarding section titles, I think the theory is it makes it easier to stick new header levels into a document.

(For those who aren't familiar with RST: If it sees underlines like ====, ----, ^^^^ or ~~~~~, it decides what the header levels are based on the order in which it sees them.)

In practice, it means a typo makes it completely ignore your header, you have to learn a set of acceptable characters for headers, and there's no way to start a document with anything less than shouty H1.

Nice idea, though.

> Is there a markup that can do numbered lists?

Correctly and usably? NOPE.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by earenndil on Wednesday April 10, 2019 @04:22PM (#19628415)
Why do you need markup for that?

1. first item

2. second

3. and so on

Markdown does parse that, but it seems superfluous to me tbh.

Re:Org-Mode Is One of the Most Reasonable Markup Languages t... by Ovid on Wednesday April 10, 2019 @10:57AM (#19625312)
As a huge fan of Perl and very comfortable writing POD, I despise POD. I would much prefer Markdown for documentation. It's easier to write and easier to read. Just try creating lists in each.