News for hackers. Stuff that matters.
Re:The Future of Markdown by dgreensp on Thursday October 25, 2012 @10:29PM (#4700558)
Wow, I wasn't expecting my email to Jeff to end up as a front-page blog post!

The point here is that Markdown doesn't have a spec, nor do any of its variants to my knowledge, so I was proposing to come up with some Markdown-like language that does have a spec. Under discussion here is the more ambitious (but also appealing) plan of writing an official spec for Markdown, the same way JavaScript got a spec in the form of ECMAScript that we now identify with JavaScript itself.

A spec is a long, tedious, human-readable document that explains the behavior of a system in unambiguous terms. Specs are important because they allow us to reason about a language like Markdown without reference to any particular implementation, and they allow people to write implementations (Markdown processors) independently that behave identically. The Markdown Syntax Documentation is not a spec (it's highly ambiguous), nor is any implementation (not human-readable; some behaviors are probably accidental or incidental and difficult to port perfectly). The hard part of writing a spec is codifying the details in English, and secondarily making decisions about what should happen in otherwise ambiguous or undefined cases.

My motivation for working on a Markdown spec is first and foremost avoiding "bit rot" of content, which happens when we write content against one Markdown implementation and then later process it with another. We don't have this concern with HTML, JSON, or JavaScript, or at least we know what bounds to stay within to write code that will work on any implementation. This is achieved through specs, even if only implementers ever read them.

I would love pointers to Markdown processors that are implemented in a more principled way than the original code, for example using standard-looking lexing and parsing passes, but that still handle nested blockquotes and bullet lists together with hard-wrapped paragraphs.

Re:The Future of Markdown by greggman on Friday October 26, 2012 @12:09AM (#4700840)
Specs are important but I'd argue a conformance test is equally or even more important.

With a conformance test people can test their implementations and when ambiguities arrise new tests can be added or old tests fixed. Without, conformance tests different interpretations of a spec lead to divergent behavior. Once that behavior is out there long enough it becomes difficult to fix as people are depended on it and/or its quirks.

Re:The Future of Markdown by mistercow on Friday October 26, 2012 @03:13AM (#4701199)
I'm confused. How are you supposed to have a conformance test without first having a spec? To what would the implementations be conforming, if not a spec?
Re:The Future of Markdown by greggman on Friday October 26, 2012 @03:39AM (#4701247)
Sorry, What I meant to really say is specs are not sufficient. Sure specs are important. But even with specs, without comprehensive conformance tests implementations will diverge.

I work on the WebGL spec and tests based off the OpenGL ES spec and tests. Even though the OpenGL ES spec is long and detailed, every edge case for which there is no test is broken or different in one driver or another.

Re:The Future of Markdown by jamesrom on Friday October 26, 2012 @09:21AM (#4702019)
In an ideal world, the spec would be the test. A spec is just a human readable test. A test is just a machine readable spec.
Re:The Future of Markdown by ZeroGravitas on Friday October 26, 2012 @07:29AM (#4701741)
It's entirely possible to build a conformance test suite around an existing 'black box' of code and accept that whatever it does, even stuff that might be considered a bug, is what any compliant implementation should also do.

(You could also fix some of the more egregious bugs, and then expect implementations to follow your new behaviour)

Re:The Future of Markdown by SideburnsOfDoom on Friday October 26, 2012 @04:49AM (#4701408)
> Specs are important but I'd argue a conformance test is equally or even more important.

I agree, but the article mentions tests prominently. They didn't miss that out.

Re:The Future of Markdown by bergie on Friday October 26, 2012 @05:02AM (#4701438)
Here are the tests we wrote for HTML-to-Markdown conversion. Could be useful when switched the other way around :-)

https://github.com/bergie/to-markdown/blob/master/test/tests...

Re:The Future of Markdown by draegtun on Friday October 26, 2012 @07:05AM (#4701666)
Also worth looking at the tests for Text::Markdown which is the canonical version of Markdown on CPAN.

https://metacpan.org/source/BOBTFISH/Text-Markdown-1.000031/...

Re:The Future of Markdown by th on Friday October 26, 2012 @03:39AM (#4701246)
> Without, conformance tests different interpretations of a spec lead to divergent behavior.

I think you mean either:

> Without conformance tests incorrect interpretations of a spec lead to divergent behavior

Or

> Without conformance tests different interpretations of an ambiguous spec lead to divergent behavior.

Conformance tests should test specific behavior defined in the spec. Tests will never (or rarely) prove spec conformance but spec conformance should always prove test conformance.

Re:The Future of Markdown by lmm on Friday October 26, 2012 @05:26AM (#4701491)
I have yet to encounter a spec that was not in some way ambiguous.
Re:The Future of Markdown by jjr on Friday October 26, 2012 @08:20AM (#4701843)
1 + 1 = 2 Is fairly unambiguous until you introduce operator overloading (and why i despise overloading).
Re:The Future of Markdown by _pferreir_ on Saturday October 27, 2012 @04:45AM (#4705753)
> 1 + 1 = 2 Is fairly unambiguous until you introduce operator overloading (and why i despise overloading).

It is only unambiguous if you specify the numbering system you are using ;)

</pedantic>

Re:The Future of Markdown by cynicalkane on Saturday October 27, 2012 @12:06PM (#4706412)
Is there actually a notation and + operator for which 1 + 1 = 2 is ambiguous? Only being semi-rhetorical, I'd be interested to know if there is one. (Modular arithmetic is not considered ambiguous, at least not in the math neck of the woods.)
Re:The Future of Markdown by josch on Saturday October 27, 2012 @05:09PM (#4707246)
Not really answering your question, but more to the point of the discussion in my opinion, a() + b() could be ambiguous if a() and b() are functions with side effects and the evaluation order of the +-operator is not specified.
Re:The Future of Markdown by ramses0 on Saturday October 27, 2012 @03:40PM (#4707009)
1 + 1 = 10b is what he's saying.
Re:The Future of Markdown by _pferreir_ on Monday October 29, 2012 @02:34PM (#4713732)
Yes, that's what I meant - even "1 + 1 = 2" can be ambiguous if no context is provided.
Re:The Future of Markdown by JadeNB on Friday October 26, 2012 @02:29PM (#4703522)
> 1 + 1 = 2 Is fairly unambiguous until you introduce operator overloading (and why i despise overloading).

What is it specifying—or, perhaps a better question, what would an implementation look like? It might be more precise to say "I have yet to encounter an implemented spec that was not in some way ambiguous."

Re:The Future of Markdown by richbradshaw on Saturday October 27, 2012 @05:13PM (#4707262)
1 + 1 != 2 if the 1s are magnitudes of vectors - if the directions are opposite, then they can be 0. In fact, 0 <= 1 + 1 <= 2 in that system.
Re:The Future of Markdown by Dylan16807 on Wednesday October 31, 2012 @02:31PM (#4724364)
That's not how addition works. You're either adding the vectors or you're adding the magnitudes. You don't get to sleight of hand by writing one and adding the other.
Re:The Future of Markdown by nollidge on Friday October 26, 2012 @11:10AM (#4702452)
Overloading is a feature of natural language though, and therefore completely unavoidable in writing a spec.
Re:The Future of Markdown by anthonyb on Friday October 26, 2012 @07:18AM (#4701704)
Hee hee, your interpretation of his sentence is introducing ambiguities :)
Re:The Future of Markdown by eslaught on Thursday October 25, 2012 @10:40PM (#4700589)
You should look at Pandoc[1]. It's a Markdown-to-everything converter, and though I'm not super familiar with the code I believe it's well written. One of my favorite tools for writing.

[1]: http://johnmacfarlane.net/pandoc/

Re:The Future of Markdown by kenko on Friday October 26, 2012 @12:33AM (#4700884)
Pandoc is fantastic, the parsing code is clear, and its extensions to markdown are well thought out.

It's not just a markdown->everything converter, though. My install understands these:

Input formats: native, json, markdown, markdown+lhs, rst, rst+lhs, textile, html, latex, latex+lhs

Output formats: native, json, html, html5, html+lhs, html5+lhs, s5, slidy, dzslides, docbook, opendocument, latex, latex+lhs, beamer, beamer+lhs, context, texinfo, man, markdown, markdown+lhs, plain, rst, rst+lhs, mediawiki, textile, rtf, org, asciidoc, odt, docx, epub

Re:The Future of Markdown by Loic on Friday October 26, 2012 @04:40AM (#4701379)
You also need to add that the parser of Pandoc is not regular expression based. It is a very robust parser, parsing the document in an internal abstract tree you can easily modify. You can also adapt the parser to your needs with simple extensions. You can then get very nice outputs in HTML or PDF[1]. As it is a nice command line tool, it is easy to generate everything with a simple Makefile too.

[1]: http://notes.ceondo.com/mongrel2-zmq-paas/

Re:The Future of Markdown by intractable on Friday October 26, 2012 @12:52AM (#4700913)
+1 for Pandoc. It's awesome.

Also, it needs to be mentioned that Pandoc supports its own set of (IMO) sensible extensions / defaults to Gruber markdown [1].

[1] http://johnmacfarlane.net/pandoc/README.html#pandocs-markdow...

Re:The Future of Markdown by evolve2k on Friday October 26, 2012 @11:23AM (#4702537)
Yes I came here also to add a +1 to ensuring that whatever standards discussions would include Pandoc and it's awesome implementation and converter.

From the link, scroll down to 'Pandoc’s Markdown' about a third of the way down the page for all the details. http://johnmacfarlane.net/pandoc/README.html

Re:The Future of Markdown by 178 on Friday October 26, 2012 @02:37PM (#4703568)
+1 for Pandocs extensions. I can verify that is sufficient for producing a doctoral dissertion (in German; not my own…), via LaTeX. The way Pandoc handles citations, footnotes, tables(!), etc is just delightful.
Re:The Future of Markdown by mongol on Friday October 26, 2012 @04:27PM (#4704118)
Yet another "Yay!" for pandoc. It is just so good.
Re:The Future of Markdown by buro9 on Friday October 26, 2012 @02:29AM (#4701104)
I agree with all of this.

I'd also like to note that some parts of Markdown from the user perspective are non-intuitive and clumsy.

Such as links and images (inline).

Markdown works so well because it is intuitive and appeals to those who once saw old word processors. They don't have to worry about syntax, and can just enter their text into a textarea (free from JavaScript WYSIWYG interference and the inherent troubles of running that on old and new mobile phones, their playstation, web browsers, etc)... and it just works.

Yet some parts of markdown are simply not intuitive. Links and images are two places where I see in usability testing that the end user will constantly refer to help documentation to figure out how to do it.

Beyond getting the code consistent, maintainable, and testable I'd love to see the language itself solve some of the papercuts that trouble the lay end user.

Realising that what I was planning to do for my project (discussion forums, tumblr for forums) was to create an alternative to markdown that would resolve some of these user issues as well as parser issues, I had already decided that I would not call it markdown and that I would educate my users in something new that hopefully solves their and my needs and would remain very stable due to a well-thought out and documented design in the first place. If what you're proposing is in this vein then consider my hat thrown in, what help I can give I will... take me to your git repository.

Re:The Future of Markdown by ams6110 on Friday October 26, 2012 @07:27AM (#4701731)
This is sort of my problem with the whole genre of "light" markup formats.... ultimately they are still a bunch of arbitary conventions to remember, and at that point you might as well just learn/write HTML. YMMV.
Re:The Future of Markdown by zmj on Friday October 26, 2012 @10:37AM (#4702295)
That's not the problem being solved by Markdown. You don't want to be serving HTML written by users.
Re:The Future of Markdown by halostatue on Saturday October 27, 2012 @03:00PM (#4706896)
Strictly speaking, that isn't the problem being solved by Markdown, either.

Remember that Daring Fireball does not have comments, so this isn't a concern. Markdown was something originally created to solve an authorial problem, not something for forum creators to use for comments.

It's flexible enough to do that, but it isn't the purpose of Markdown.

Re:The Future of Markdown by georgemcbay on Saturday October 27, 2012 @06:51AM (#4705902)
Why not? You have to protect against malicious HTML injection whether or not the user is using markdown, plain text or html, so why not let them use a carefully restricted subset of html?
Re:The Future of Markdown by nsmartt on Sunday October 28, 2012 @10:50AM (#4709049)
While this is not wrong, markdown involves much less typing than HTML. Also, should one need to use HTML, markdown does allow one to switch between HTML and markdown.
Re:The Future of Markdown by wwweston on Saturday October 27, 2012 @01:40PM (#4706683)
They're arguably less arbitrary than HTML; most of them are based in things that people have done/tend to do with plain text when they're trying to create a formatted document. That yields two benefits: the source document is inherently formatted -- transparently so, since even people who don't know what Markdown or the like is will be able to read it -- and creates something of a visual mnemonic for the formatting convention.

Not saying any one implementation is perfect, just that they have these nice properties.

Re:The Future of Markdown by alexchamberlain on Friday October 26, 2012 @02:34AM (#4701114)
What is your problem with the link syntax? I think it's one of the best bits.
Re:The Future of Markdown by fudged71 on Friday October 26, 2012 @03:18AM (#4701211)
Having wasted a lot of my life reading miles of reddit comments, there are some people who don't understand the link syntax, or switch the brackets ()[]. It happens. But not that often, when the alternate way of posting a link is to just paste it in plain text with no title associated with it.
Re:The Future of Markdown by jacobr on Friday October 26, 2012 @03:10AM (#4701194)
You have to remember 1) which comes first, url or link text? 2) is it () or []?
Re:The Future of Markdown by uvtc on Friday October 26, 2012 @01:08PM (#4703140)
I look at it like this: [this] looks a bit like a button you might press (or a link you might click). It looks like what it does.

(this) looks like something you'd whisper off to the side --- to add to the conversation. For example, "That picture is nice (but it needs more trees).".

So links go like: [this is the thing you click](and btw, this is where it leads).

Re:The Future of Markdown by 178 on Friday October 26, 2012 @05:27AM (#4701495)
I struggled with that until I looked at it from a parser perspective: links start with [] because () are much more commons in normal language.
Re:The Future of Markdown by spion on Friday October 26, 2012 @06:46AM (#4701623)
There is actually an even easier method.

In normal text, we usually link to things by first citing them then providing the link. An example would be Google (http://www.google.com). Its natural that we surround the link in () in normal text.

Now the bit we add to help the parser is to tell it where the thing we're linking to begins and where it ends. [ and ] are not usually found in normal text and are therefore used for this purpose. Thus the modified example would be [Google](http://www.google.com).

Re:The Future of Markdown by SiVal on Friday October 26, 2012 @03:51PM (#4703935)
I would much rather see a more mnemonic approach for links, where you would write either _click here_ [http://wherever.com] or _click here_ (http://wherever.com). The underscores look like an underline, which is the traditional appearance of link text in HTML. It's easy to remember. Whether the URL is in square brackets or parentheses (maybe either one would be acceptable as long as they are a matched pair) doesn't matter, but underlining the anchor text is the way to go, IMO.
Re:The Future of Markdown by daigoba66 on Friday October 26, 2012 @09:07AM (#4701971)
Beautiful explanation. And one that I can remember while writing in markdown.
Re:The Future of Markdown by alexchamberlain on Friday October 26, 2012 @03:40AM (#4701251)
I'm just wondering whether a better solution to this is a polite warning in the UI... Did you mean []() instead of ()[]?
Re:The Future of Markdown by ars on Friday October 26, 2012 @03:47AM (#4701261)
I like the wikipedia syntax better for this [url text]. It's much easier to remember.
Re:The Future of Markdown by lloeki on Friday October 26, 2012 @07:12AM (#4701686)
> It's much easier to remember.

How so?

Markdown has this brilliance in that it's not just random markup that produces html or something output, it's own human-targeted plain text output is its own source. A such the link syntax is extremely easy to remember. When I write text, I want to read the label, whose reference is a link, hence the order is label then link. If I wrote this link in pure plain text I'd simply naturally write:

    Lorem ipsum dolor sit amet (http://www.amet.com), consectetur adipiscing elit.
So the order is the same is markdown since it's its own output. Still the world is not ideal so we need a hint to tokenize a little, and since parentheses surround the link, let's use square brackets. This also fits very well with the array/hash/dictionary syntax, where you associate a key sitting between square brackets with a value.

If I want to make a more remote reference so as not to interrupt the reading flow with a big link (remember that markdown source is its own plain text output) I can simply do what I'd do if I inserted a reference, that is annotate the label.

    Lorem ipsum dolor sit amet[1], consectetur adipiscing elit.
The footer naturally follows, as we again assign a key to a value. And since there can be an optional title to the link, it comes afterwards so that urls vertically align.

    [1]: http://www.amet.com "optional title"
    [2]: http://adispicit.org "optional title"
Again, I find brilliance in Markdown in that it leveraged long (as in decades) established conventions (e.g headers, italics, lists, quotes, code blocks...) that apply directly to plain text in order for the source to be its own output, so that even for someone who doesn't have the first clue about Markdown it's readily readable as if it were not markup.
Re:The Future of Markdown by alexchamberlain on Friday October 26, 2012 @04:26AM (#4701344)
The problem is that this is a discussion about standardising the Markdown syntax, not changing it.
Re:The Future of Markdown by buro9 on Friday October 26, 2012 @05:45AM (#4701528)
The fact that Meteor wish to call this Rockdown suggests to me it is to change it.

By creating a specification and tests, they will be modifying Markdown even if just subtly enough to remove bugs and ambiguity. Doing even that is to create something that isn't Markdown, existing Markdown parsers may do something different with the same text... so clearly this isn't Markdown.

It may share 90% (or more, or less) similarity with Markdown, but some areas would certainly need tweaking and changing to make the goal (specification, testability, consistent implementations) possible.

To my understanding this means that Rockdown is Markdown inspired/derived.

And as such, that there is a hope that a good specification and the ability to be implemented in a testable and consistent way... will, by mass use (on the sites mentioned), eventually result in Rockdown superseding Markdown.

In my universe, I'm simply looking to get rid of bbcode as the markup syntax of choice. And if you were at the support end of trying to tell people how to insert images on forums then you'd understand that the Markdown syntax is a real difficulty for a lot of people.

The path that I prefer is to use Markdown, but the areas that cause the greatest difficulty in usability testing, to modify those to make things more implicit and frictionless. Worrying about []() (or vice versa) is an obstacle to the user that shouldn't exist. But if everything else is good from a user perspective then I definitely want to keep that stuff.

Re:The Future of Markdown by ars on Friday October 26, 2012 @05:29AM (#4701500)
Wikipedia also uses markdown. So since there is no standard format for URLs, let's pick a good one.

Also, it could support both of these syntaxes at once - they don't conflict.

Re:The Future of Markdown by tripzilch on Friday October 26, 2012 @11:28AM (#4702563)
I still get those the wrong way around [text url] often enough. I only mess up MarkDown slightly less often because I write more of it.

The "() is more common than [] in regular text, so it makes parsing-sense to start a special markup code with []" is going to help me with that, though. That particular mnemonic only works for coders, though :)

Also, doesn't MediaWiki syntax sometimes also have a pipe | between the url and the text in a link? Or is that only for internal links?

Re:The Future of Markdown by buro9 on Friday October 26, 2012 @12:49PM (#4703035)
A parser should not be an interactive application.
Re:The Future of Markdown by bitcartel on Friday October 26, 2012 @02:44AM (#4701139)
Why are people still interested in Markdown - what problem is it solving and for whom?

It's 2012 and we have HTML 5 compliant WYSIWYG text editors. We no longer have to write plain text littered with special codes, for the purpose of running through a parser, to produce HTML which looks nice on a web page. Maybe it made sense a decade ago when web forms had terrible editors, but not anymore. I think Joe Internet writing blog posts and forum replies would agree with me.

For developers, a README file in plain text looks great everywhere, and avoids any Github vs Bitbucket vs Assembla display issues. If you need to write structured documentation for a system, there's probably already a designated markup language you're supposed to use, so Markdown doesn't help there.

Re:The Future of Markdown by ChiperSoft on Friday October 26, 2012 @03:15AM (#4701203)
I think you are grossly overestimating the quality of current rich text editors... None of them produce reliable clean code. Some produce less shitty code than others, but they do so through tremendous amounts of post-processing.

Markdown just works and always produces clean, easy to style code.

Re:The Future of Markdown by jdbernard on Friday October 26, 2012 @05:15AM (#4701469)
I am in love with plain-text files. I edit them easily no matter what device I am on, whether it is my desktop, my phone, over SSH, in a browser. I can use my more powerful text-editing software instead of being stuck in a weak WYSIWYG editor.

My blog is just a directory of Markdown files that get compiled into HTML files. No cumbersome WordPress or other blog software installation I need to maintain, just copy the HTML to the server. Simple.

Same thing with documentation. Write once, from anywhere, output clean code to multiple formats: HTML, LaTeX (and PDF, DVI, etc. from there). No need to worry about making sure complex documentation toolchains are installed, just a Markdown compiler. A huge advantage of Markdown is that the source is as readable as the output. JavaDoc, for example, sucks in this regard if you try to do any formatting (lists, tables, etc.). If I am reading the source code--and I prefer to read the source if I can--then the javadoc is often useless for complex functions where it is most needed, because the HTML formatting is unreadable. I am actually building my own documentation system because of this (see https://github.com/jdbernard/jlp if you are interested).

Re:The Future of Markdown by keithpeter on Friday October 26, 2012 @03:05AM (#4701185)
"Why are people still interested in Markdown - what problem is it solving and for whom?"

I understand the question and wonder why it attracted down-votes. The executive summary of my answer: light markup is used in places other than text areas on Web sites.

Many of us use Markdown to keep formatted text in, and, ultimately, to generate valid html markup from. Markdown is not just for textareas in Web pages. Whole books have been marked up in markdown.

Markdown files are just text files with 'explicit' markup within them and so will always be accessible/editable/processable in the future. Markdown is 'flexible' in the sense that it allows alternative markup for the same output, and is 'light' in the sense that a limited range of styling and block formats are available. Compare that with LaTeX which shifts with each tex-live release, and which means that markup written some years ago may not format without hand editing on a modern LaTeX release.

There is the reference implementation available from Gruber's site, and, as others have mentioned, pandoc implements markdown in a way that can be extended. I'm dubious of the need for an 'official' specification; I rather like the scrappy streetwise nature of Markdown.

Re:The Future of Markdown by JadeNB on Friday October 26, 2012 @02:26PM (#4703514)
> Compare that with LaTeX which shifts with each tex-live release, and which means that markup written some years ago may not format without hand editing on a modern LaTeX release.

What? Certainly details of individual packages may change, including addition or removal of functionality, over time, but I don't even know what it means to say that "LaTeX shifts". The underlying markup language for LaTeX, which is the same as that for TeX, is very flexible, but has had the same flexibility since at least 1982, if not 1978.

I understand the idea that built-in extensibility leaves a platform, to some extent, to blame for its plug-ins; but many people in this discussion have proposed building some sort of plug-in architecture for Markdown, too.

Re:The Future of Markdown by bitcartel on Friday October 26, 2012 @03:15PM (#4703754)
Sure, but why are we using the lowest common denominator, plain text, and then adding formatting?

Why don't we start with something like a visual HTML or DocBook editor, and then apply parsers to generate plain text, PDFs, etc?

Is it because we as programmers live in a plain text environment and are happy with the status quo, or is it because the tools aren't good enough?

Re:The Future of Markdown by dspillett on Friday October 26, 2012 @05:14AM (#4701467)
> Why are people still interested in Markdown - what problem is it solving and for whom?

For techie types it is familiarity: as a programmer I've used mark-down style syntax in code comments (and emails back when plain text was the norm) for years so it feels natural. I can type and edit text with mark-down faster then with full WYSIWYG control (even if the editor has good keyboard shortcuts). Also I can use the same text+markdown in code, emails and documentation that is desired to look a little fancier - I don't need to reformat for a second audience. Markdown isn't perfect, but for me it is a better solution for some things than WYSIWYG.

Simplicity and resulting cleanliness of mark-up can be an issue sometimes. I've seen WYSIWYG editors get tied in knots over bad HTML that they generated themselves.

Simplicity is another bonus for sites taking in user content, particularly sites like the StackExchange family. By effectively having a while-list of formatting options you can keep a consistent look on your site more easily. If you aren't sure what problem this solves, think Geocities.

It makes security a little easier to: you don't have to worry so much about potential injection attacks that are very possible if trying to generate/accept full HTML and filter it for the bad stuff afterwards (you still have to be careful to think about these issues of course, but your available attack surface is much smaller and therefore easier to manage).

> Github vs Bitbucket vs Assembla display issues

This is the very problem that people are actually discussing - competing interpretations make mark-down less useful then it could otherwise be, and a global standard might help that issue if it gets widely adopted. Of course many incompatible interpretations already exist and any new standard will face adoption friction, so we may end up in a situation like parodied in http://xkcd.com/927/ (it worked well enough for Javascript in the long run, but I think there were fewer common and slightly incompatible implementations of that at the time than there are of markdown now).

Re:The Future of Markdown by silvestrov on Friday October 26, 2012 @07:17AM (#4701697)
Markdown is fantastic because it restricts the formatting.

When users copy/paste from MS Word into HTML editors the paste often keeps the font from the Word document which is not the proper font for the html page. That results in a messy page layout.

In my experience it's easier for users to understand Markdown syntax than to understand the "hidden codes" in HTML editors when they want to fix the layout.

Re:The Future of Markdown by bitcartel on Friday October 26, 2012 @02:56PM (#4703669)
Sure, but if we could paste HTML between editors and browsers without any rendering or layout issues, there would be no reason for users to look beneath the surface. I do agree that we're not quite there yet.
Re:The Future of Markdown by dspillett on Monday October 29, 2012 @06:47AM (#4711733)
There is still the need for destination system to fully validate the input to catch badly formed content which may cause unpredictable issues and (more importantly) to screen out injection attacks, which may not be a trivial task. Relative to validating markdown decorated content it is far from trivial.
Re:The Future of Markdown by Gravityloss on Saturday October 27, 2012 @06:18AM (#4705868)
You can use markdown as source for formatting, but the source is very readable by itself.

This is extremely important for writing and also for fall back purposes. Any time there is any kind of a problem in writing or reading or interpreting, just fall back to source.

People can say html is human readable but the difference is huge. I didn't even know that markdown is parsed on hacker news, I just used stars as underscores as natural formatting.

There's also potentially much much less overhead. Ultimately, browsers and your cell phone apps etc should just format markdown natively, meaning huge performance boosts, as html, css, js etc has just become so complex for many tasks as a consequence of do-everything.

My phone's email client struggles browsing back and forth of what should be just a few small text files and file attachments. It uses way too many cycles since it's also a jpeg and css and js renderer with scaling and everything. As someone who's used text email and news groups, lightweight formatting is very natural and informative. Those services would not have been possible in the past if they had been bogged down with large amounts of excess complexity.

I guess reducing overhead is not sexy because it doesn't involve new features. In the future everybody will just create their typed static oneliner email as 10000x10000x10000 pixel 100 fps 3D movies of 2 minute length (you see, that's the time it takes to read it), and if some supposedly slightly technically literate person asks that perhaps they should use a word 2020 .doc instead for reduced file size, the answer is "but you can't do a space station flythrough in it".

Re:The Future of Markdown by thenomad on Friday October 26, 2012 @05:29AM (#4701498)
The sheer popularity of Markdown implies that there are some use cases for it. For example, Reddit uses Markdown exclusively, and I don't believe ALL their users are highly tech-savvy.

Personally, I don't trust or use (if I can avoid it) any on-the-web WYSIWYG text editor, because a) they tend to be slow and clunky and b) it's difficult to predict what HTML they will turn out a lot of the time, and I will often end up needing to edit that HTML.

Also, for common blogging / commenting tasks, Markdown is as fast or faster to use than WYSIWYG editors. I've been faffing around with a forum that won't let me use Markdown recently, and forces me to use its clunky WYSIWYG editor to add lists into my posts. It didn't take long for me to start yearning for the simplicity of linebreak - asterisk - item rather than a multiple-click WYSIWYG process.

Re:The Future of Markdown by bitcartel on Friday October 26, 2012 @03:04PM (#4703711)
If we assume WYSIWYG editors keep on improving, at some point, they'll become good enough. What then for Markdown?
Re:The Future of Markdown by falcolas on Friday October 26, 2012 @06:06PM (#4704492)
Personally? I would still prefer to write plain text with a bit of simplistic markup.

YMMV, but I've never been fond of WYSIWYG editors. I'd rather say what I mean, instead of say what this other guy thinks I mean.

Re:The Future of Markdown by vidarh on Friday October 26, 2012 @04:34AM (#4701361)
I find it more comfortable to write Markdown than to use a WYSIWYG text editor. That the WYSIWYG editors tends to produce really shitty output doesn't make things better.

Markdown and similar formats can also easily be used to produce other output than HTML.

And Markdown is plain text, and that's another appealing factor: Even if you read it somewhere that won't render the Markdown, it is still readable. The same can't be said about HTML.

Re:The Future of Markdown by fudged71 on Friday October 26, 2012 @03:25AM (#4701222)
For some people and some uses, plain text is just more comfortable.

I love the ability to scratch something down in Markdown really quickly, and being able to convert it to LaTeX if I want to, without any hassle.

I'd love to write blog posts in Markdown. Google+ uses the italic and _bold_ syntax in their own posts, which is very useful.

I don't exactly know why it's better, to be honest. Maybe WYSIWYG editors are inherently dependant on their own implementations and rely on people learning the interface.

Text-As-An-Interface is beautiful in it's simplicity. Markdown is the perfect mix of simple markup without the insanity of BBCode.

Re:The Future of Markdown by sopooneo on Friday October 26, 2012 @10:34AM (#4702285)
But is it plain text? If it is, then we can make an argument that html documents are as well.
Re:The Future of Markdown by fudged71 on Friday October 26, 2012 @03:29PM (#4703823)
I think that's the difference, though. You can read it easily without being compiled. HTML is very markup-intensive.
Re:The Future of Markdown by omaranto on Friday October 26, 2012 @07:15AM (#4701693)
1. Could you recommend some of these good WYSIWYG editors you claim exist? The ones I've seen are all slow or awkward (basically not having enough keyboard shortcuts) or get confused when I delete part of the text (and usually have all of these problems).

2. It sounds like you are only concerned in producing HTML. I like using markdown for everything. Using the excellent Pandoc I produce both HTML and pretty PDFs from markdown text, with nice looking formulas too. (PDFs are produce through LaTeX, for HTML Pandoc offers several options, I use MathJax).

Re:The Future of Markdown by bitcartel on Friday October 26, 2012 @03:24PM (#4703802)
As an end-user I find the Wordpress editor to be pretty good, and Google Docs ok. The Aloha editor looks great: http://aloha-editor.org/index.php

Pandoc looks great. So why not write in HTML or Docbook and then convert to plain text / RTF / PDF etc?

Re:The Future of Markdown by tripzilch on Friday October 26, 2012 @11:40AM (#4702655)
It's not just for comments and blog posts.

I often find myself writing short plain text files, usually quick personal notes. Occasionally they grow larger than what I'd call a "short" plain text file, and at this point two extra requirements pop up:

1) they are going to need some structure.

2) odds are, that I'm going to want to show this document to someone else at some later point in time[1].

Markdown offers (to me) by far the easiest transition from "basic plain text note" to "slightly longer formatted document", because most times, all I have to do is continue writing the way I was doing already.

[1] corollary: as programmers know, when reading their own code, "someone else" also includes you in three months time.

Re:The Future of Markdown by Adrock on Friday October 26, 2012 @05:12AM (#4701460)
In addition to all the other great answers here:

1. You can pry my {Emacs,vim} from my cold, dead hands.

2. Version controlled markdown is much easier to deal with.

Re:The Future of Markdown by gordonguthrie on Friday October 26, 2012 @11:54AM (#4702749)
Yeah, but whereas git and other version control systems are great with lines of code - and diff works well on that - for text we need a diff that works on lexical units within a (much longer) line or structure.

I know git can be configured to use an arbritary programme for diff - just not sure where to find one...

Re:The Future of Markdown by chokma on Friday October 26, 2012 @04:38AM (#4701372)
Many users like their WYSIWYG text editors (even though they are often problematic between older browser versions or with "paste-from-word" or conflicts with other JS-includes). And the expectations are rising - one customer of ours would like a WYSIWYG-editor "with Apple style interface" in the browser, with all the bell's and whistles of a full blown publishing solution including complete layout control, input validation, CMS support and so on. The old TinyMCE fields of his legacy app will no longer cut it.

Meanwhile, I write the documentation for my software in Markdown, because it is the fastest way for me to get structured text from my mind to my machine. And with the help of pandoc, it's convertible into all other text formats, from HTML to PDF and LaTeX etc. (Although the html-with-images to PDF support of wkhtmltopdf is even better, so it's Markdown => pandoc[for HTML] + wkthmltopdf [for HTML to PDF]).

Re:The Future of Markdown by krsunny on Friday October 26, 2012 @02:48AM (#4701145)
Because in 2012 there are still lots of plain textareas, like the one I'm writing this reply in.
Re:The Future of Markdown by sweetdreamerit on Friday October 26, 2012 @04:32AM (#4701357)
using Markdown you can: - track versions with tools like git - using pandoc, you can transform it in html, pdf, latex, beamer, rtf, slidy - you are sure you are separating style from content - it can be used everywhere: using console editors like vim (or the simple nano, that I love), text editors like notepad+, gedit. I found out a very nice android app, Epstile, that uses markdown and syncs the data using dropbox. - just this morning, in the office where I'm working, there was a discussion like "this doc was saved using office 2010 or office 2007?". Using markdown, you can avoid such a mess.
Re:The Future of Markdown by city41 on Friday October 26, 2012 @11:53PM (#4705291)
Most tech-type people don't like WYSIWYG. I know I don't. The product that my company makes has a very nice WYSIWYG editor built into it. For a hackathon I replaced it with a Markdown editor and all the other developers went nuts (sadly, for various reasons, it never made it into the real product).

The main appeal is Markdown lets us write formatted text without ever having to use the mouse or figure out what the current WYSIWYG editor's key commands are. Often times -- like making nested bulleted lists -- there just plain aren't any key commands. I'd much rather type a few square brackets and create a nice hyperlink on the fly than invoke some silly menu, paste in my link in the window that pops up, create the display text, etc etc.

Re:The Future of Markdown by alexchamberlain on Friday October 26, 2012 @04:34AM (#4701362)
Safety.

If you exclude HTML tags, Markdown is safe. People can't inject code.

I know this can be done for HTML as well, but it's harder.

Re:The Future of Markdown by blasdel on Friday October 26, 2012 @07:12AM (#4701681)
Unfortunately, that is absolutely not true in any implementation derived from the original Markdown.pl without a whole lot of bugfixes

Because of the chickenshit way he attempted to escape things by replacing them with their MD5 hashes and then switching them back, you can encode anything you want to be output by the markdown processor. Reddit was owned by a viral XSS comment because of this in 2009, it's been exploited a few other times since.

Re:The Future of Markdown by bitcartel on Friday October 26, 2012 @03:39PM (#4703872)
Found the info, interesting. Easy to think plain text is safe and forget about the parser.

http://www.f-secure.com/weblog/archives/00001777.html

http://blog.reddit.com/2009/09/we-had-some-bugs-and-it-hurt-...

Re:The Future of Markdown by Too on Friday October 26, 2012 @12:43PM (#4703005)
WYSIWYG always means either 1. proprietary file format and/or 2. Shitty code generation.

Neither works very well with version control or scenarios where you need compatibility between different system.

Plain text ALWAYS works.

Re:The Future of Markdown by tomp on Friday October 26, 2012 @01:02PM (#4703103)
Most WYSIWYG require the use of a mouse.
Re:The Future of Markdown by peteretep on Friday October 26, 2012 @04:46AM (#4701398)
I can't be bothered to learn LaTeX to mark up my Haskell assignments, so I've been using Markdown + LHS
Re:The Future of Markdown by halostatue on Thursday October 25, 2012 @10:38PM (#4700583)
https://github.com/jgm/peg-markdown and https://github.com/fletcher/peg-multimarkdown

I haven't looked at the implementations of these, but they are most certainly grammar-based, not regex-based.

Re:The Future of Markdown by someone13 on Thursday October 25, 2012 @11:12PM (#4700671)
Another good parser is python-markdown2 [1], which has a very extensive set of test cases, and the code is very readable and well-commented.

[1] https://github.com/trentm/python-markdown2

Re:The Future of Markdown by arnarbi on Friday October 26, 2012 @05:29AM (#4701499)
I do not agree that a spec has to be written in English. Like most natural languages, it is very easy to be ambiguous and to underspecify. Resolving this can make English just as human un-readable as anything else.

Specs like this should be written as executable reference implementations in a well defined programming language. This can very well be human readable, and should be done without regard for efficient execution. It's less ambiguous, amenable to automated conformance testing, and is easier to evolve than a natural language document.

Re:The Future of Markdown by chc on Friday October 26, 2012 @05:33AM (#4701505)
If that's the spec, how do we know which aspects of the executable are really part of the specification and which are implementation details? For (a somewhat extreme for the sake of illustration) example, if the spec is written for Ruby 1.8, do C implementations have to intentionally slow down to match its execution speed?
Re:The Future of Markdown by arnarbi on Saturday October 27, 2012 @11:40PM (#4708069)
Of course not. Such a spec would state what are its inputs and outputs, either as natural language descriptions or as types (say, if the reference language supports it).

Everything outside that (e.g. timing) would be fair game to optimize. Moreover, as I hinted at, a spec should not be optimized for speed or efficiency, but rather readability and unambiguity.

Re:The Future of Markdown by chc on Sunday October 28, 2012 @03:10AM (#4708351)
> Of course not. Such a spec would state what are its inputs and outputs, either as natural language descriptions or as types (say, if the reference language supports it).

So you agree it wouldn't work for specs to be "written as executable reference implementations in a well defined programming language"?

> Everything outside that (e.g. timing) would be fair game to optimize.

But sometimes you want to specify performance characteristics.

The simple fact is that without just embedding a traditional spec within it, an implementation cannot tell you what aspects of its behavior are "specified" and which are just implementation details. This makes reference implementations inadequate specifications. That's all I was getting at.

Re:The Future of Markdown by arnarbi on Sunday October 28, 2012 @06:58PM (#4710277)
> So you agree it wouldn't work for specs to be "written as executable reference implementations in a well defined programming language"?

No, then I would be disagreeing with myself. :)

> But sometimes you want to specify performance characteristics.

For a Markdown processor, hardly. But...

> ... cannot tell you what aspects of its behavior are "specified" and which are just implementation details.

That is a fair point indeed. I was merely disagreeing with the linked article in that a spec should be in a natural language, which I still hold to. Such specs generally have problems with under specifying, leaving too much up to implementation, which often ends with a few main implementations sort of becoming the actual practical spec. Look at browsers over the history for many examples. Your point seems to be that reference implementations might overspecify instead. This is true, but is easier to compensate for IMO. For example with clear types, natural language parts or even by conformance test suites being part of the specification.

Re:The Future of Markdown by Too on Friday October 26, 2012 @12:45PM (#4703015)
Use of well established standards. For example BNF or railroad diagrams can cover a majority of a language specificaton without using English.
Re:The Future of Markdown by chc on Friday October 26, 2012 @01:26PM (#4703224)
I think you're unintentionally moving the goalposts here. The idea I was questioning wasn't "Specs should be written in something other than English," it was "Specs should take the form of an executable reference implementation." AFAIK railroad diagrams are not executable. I'm also not sure how BNF is supposed to specify things like "the map function applies a function passed in to each argument in a list".
Re:The Future of Markdown by blasdel on Friday October 26, 2012 @07:15AM (#4701692)
IETF standards are required to have multiple interoperable implementations of the usually post-hoc english language spec.

One implementation can never be enough.

Re:The Future of Markdown by fiddlosopher on Thursday October 25, 2012 @11:20PM (#4700701)
[lunamark](https://github.com/jgm/lunamark/tree/master/lunamark) is another PEG-based implementation.
Re:The Future of Markdown by uvtc on Friday October 26, 2012 @11:47PM (#4705277)
I'm not so sure what's needed here is a spec.

What people really want is for all Markdown implementations to be basically the same so they don't have to learn any implementation-specific ideosyncrasies to switch from one to another.

The problem though, as I see it, tends to go away under certain circumstances. And the circumstances, I think, are these:

* If you've got a strong implementation, robust, actively-maintained, and runs fast,

* is easy to obtain, install, and use,

* is well tested and well documented, and

* has just the right blend of sensible additions to the syntax (for example, tables, def lists, LaTeX math, etc.) --- done tastefully,

then folks will just use that, model their own implementations after that, and just overall start considering that to be the standard.

I think this has been slowly and steadily happening with Pandoc.

And, aside from all that, two additional "killer features" that Pandoc seems to have over other implementations:

1. it can convert to/from other doc markup formats, thus making it easy to just convert your existing docs to pandoc-markdown and then use that as your master source format to generate other formats you might need; and

2. with its carefully-chosen set of additional features, it has been slowly proving itself capable of being a replacement for raw LaTeX for certain types of longer technical documents.

My understanding is that there's even some features in the works (for the next release) for converting between markdown dialects --- which would make it even easier to convert markdown files of various flavors into plain standard pandoc-markdown.

So, if you're looking for a standard, I'd suggest that it's for the most part already here. :)

Re:The Future of Markdown by draegtun on Friday October 26, 2012 @08:24AM (#4701849)
>I would love pointers to Markdown processors that are implemented in a more principled way than the original code, for example using standard-looking lexing and parsing passes...

Have a look at Markdent - An event-based Markdown parser toolkit.

https://metacpan.org/module/Markdent

Re:The Future of Markdown by gordonguthrie on Friday October 26, 2012 @12:00PM (#4702782)
I wrote a manual parser with look-aheads in Erlang (making extensive use of Erlang's pattern matching.

https://github.com/hypernumbers/erlmarkdown

No regexps.... except for url's and emails :(

Re:The Future of Markdown by aleemb on Friday October 26, 2012 @01:37AM (#4700994)
I hope some more thought goes into the usability as well. I find the syntax for links confuses a lot of users for whom I have enabled mark down editing. Wiki style [[link text]] works great if you assume links have no space or maybe something similar. Similarly the syntax for images has also always bothered me.
Re:The Future of Markdown by vidarh on Friday October 26, 2012 @04:39AM (#4701376)
I have no problems remembering the Markdown link format, but I never remember the expected order in what you describe as "Wiki style" (which is really a style employed by some wiki software - there are plenty of wiki software that uses other syntaxes, including Markdown)
Re:The Future of Markdown by ExpiredLink on Friday October 26, 2012 @06:04AM (#4701561)
IMO, for the spec the most important point is to add mandatory versioning information on the first line of markdown scripts, e.g.

    _markdown_version_ Rockdown_1.0 
This would allow processor implementers to support more than one markdown version (for a transition period or in general).
Re:The Future of Markdown by jdbernard on Friday October 26, 2012 @11:55AM (#4702762)
Ugh, no. This defeats the simplicity of Markdown. I do not want to have to remember some obscure implementation versioning string whenever I want to start a new document or comment. Maybe as an option, sure, but I will probably never use it.
Re:The Future of Markdown by Helianthus on Friday October 26, 2012 @01:10AM (#4700942)
I don't know if Reddit's implementation of Markdown is principled, but as a syntax it's pretty ubiquitous so you might have the traction to make it standard.
Re:The Future of Markdown by jamii on Friday October 26, 2012 @01:52PM (#4703345)
> I would love pointers to Markdown processors that are implemented in a more principled way than the original code

This is almost what you want:

http://news.ycombinator.com/item?id=555153

EDIT: wrong link :(