With a conformance test people can test their implementations and when ambiguities arrise new tests can be added or old tests fixed. Without, conformance tests different interpretations of a spec lead to divergent behavior. Once that behavior is out there long enough it becomes difficult to fix as people are depended on it and/or its quirks.
I work on the WebGL spec and tests based off the OpenGL ES spec and tests. Even though the OpenGL ES spec is long and detailed, every edge case for which there is no test is broken or different in one driver or another.
(You could also fix some of the more egregious bugs, and then expect implementations to follow your new behaviour)
I agree, but the article mentions tests prominently. They didn't miss that out.
https://github.com/bergie/to-markdown/blob/master/test/tests...
https://metacpan.org/source/BOBTFISH/Text-Markdown-1.000031/...
I think you mean either:
> Without conformance tests incorrect interpretations of a spec lead to divergent behavior
Or
> Without conformance tests different interpretations of an ambiguous spec lead to divergent behavior.
Conformance tests should test specific behavior defined in the spec. Tests will never (or rarely) prove spec conformance but spec conformance should always prove test conformance.
It is only unambiguous if you specify the numbering system you are using ;)
</pedantic>
What is it specifying—or, perhaps a better question, what would an implementation look like? It might be more precise to say "I have yet to encounter an implemented spec that was not in some way ambiguous."
It's not just a markdown->everything converter, though. My install understands these:
Input formats: native, json, markdown, markdown+lhs, rst, rst+lhs, textile, html, latex, latex+lhs
Output formats: native, json, html, html5, html+lhs, html5+lhs, s5, slidy, dzslides, docbook, opendocument, latex, latex+lhs, beamer, beamer+lhs, context, texinfo, man, markdown, markdown+lhs, plain, rst, rst+lhs, mediawiki, textile, rtf, org, asciidoc, odt, docx, epub
Also, it needs to be mentioned that Pandoc supports its own set of (IMO) sensible extensions / defaults to Gruber markdown [1].
[1] http://johnmacfarlane.net/pandoc/README.html#pandocs-markdow...
From the link, scroll down to 'Pandoc’s Markdown' about a third of the way down the page for all the details. http://johnmacfarlane.net/pandoc/README.html
I'd also like to note that some parts of Markdown from the user perspective are non-intuitive and clumsy.
Such as links and images (inline).
Markdown works so well because it is intuitive and appeals to those who once saw old word processors. They don't have to worry about syntax, and can just enter their text into a textarea (free from JavaScript WYSIWYG interference and the inherent troubles of running that on old and new mobile phones, their playstation, web browsers, etc)... and it just works.
Yet some parts of markdown are simply not intuitive. Links and images are two places where I see in usability testing that the end user will constantly refer to help documentation to figure out how to do it.
Beyond getting the code consistent, maintainable, and testable I'd love to see the language itself solve some of the papercuts that trouble the lay end user.
Realising that what I was planning to do for my project (discussion forums, tumblr for forums) was to create an alternative to markdown that would resolve some of these user issues as well as parser issues, I had already decided that I would not call it markdown and that I would educate my users in something new that hopefully solves their and my needs and would remain very stable due to a well-thought out and documented design in the first place. If what you're proposing is in this vein then consider my hat thrown in, what help I can give I will... take me to your git repository.
Remember that Daring Fireball does not have comments, so this isn't a concern. Markdown was something originally created to solve an authorial problem, not something for forum creators to use for comments.
It's flexible enough to do that, but it isn't the purpose of Markdown.
Not saying any one implementation is perfect, just that they have these nice properties.
(this) looks like something you'd whisper off to the side --- to add to the conversation. For example, "That picture is nice (but it needs more trees).".
So links go like: [this is the thing you click](and btw, this is where it leads).
In normal text, we usually link to things by first citing them then providing the link. An example would be Google (http://www.google.com). Its natural that we surround the link in () in normal text.
Now the bit we add to help the parser is to tell it where the thing we're linking to begins and where it ends. [ and ] are not usually found in normal text and are therefore used for this purpose. Thus the modified example would be [Google](http://www.google.com).
How so?
Markdown has this brilliance in that it's not just random markup that produces html or something output, it's own human-targeted plain text output is its own source. A such the link syntax is extremely easy to remember. When I write text, I want to read the label, whose reference is a link, hence the order is label then link. If I wrote this link in pure plain text I'd simply naturally write:
Lorem ipsum dolor sit amet (http://www.amet.com), consectetur adipiscing elit.
So the order is the same is markdown since it's its own output. Still the world is not ideal so we need a hint to tokenize a little, and since parentheses surround the link, let's use square brackets. This also fits very well with the array/hash/dictionary syntax, where you associate a key sitting between square brackets with a value.If I want to make a more remote reference so as not to interrupt the reading flow with a big link (remember that markdown source is its own plain text output) I can simply do what I'd do if I inserted a reference, that is annotate the label.
Lorem ipsum dolor sit amet[1], consectetur adipiscing elit.
The footer naturally follows, as we again assign a key to a value. And since there can be an optional title to the link, it comes afterwards so that urls vertically align. [1]: http://www.amet.com "optional title"
[2]: http://adispicit.org "optional title"
Again, I find brilliance in Markdown in that it leveraged long (as in decades) established conventions (e.g headers, italics, lists, quotes, code blocks...) that apply directly to plain text in order for the source to be its own output, so that even for someone who doesn't have the first clue about Markdown it's readily readable as if it were not markup.By creating a specification and tests, they will be modifying Markdown even if just subtly enough to remove bugs and ambiguity. Doing even that is to create something that isn't Markdown, existing Markdown parsers may do something different with the same text... so clearly this isn't Markdown.
It may share 90% (or more, or less) similarity with Markdown, but some areas would certainly need tweaking and changing to make the goal (specification, testability, consistent implementations) possible.
To my understanding this means that Rockdown is Markdown inspired/derived.
And as such, that there is a hope that a good specification and the ability to be implemented in a testable and consistent way... will, by mass use (on the sites mentioned), eventually result in Rockdown superseding Markdown.
In my universe, I'm simply looking to get rid of bbcode as the markup syntax of choice. And if you were at the support end of trying to tell people how to insert images on forums then you'd understand that the Markdown syntax is a real difficulty for a lot of people.
The path that I prefer is to use Markdown, but the areas that cause the greatest difficulty in usability testing, to modify those to make things more implicit and frictionless. Worrying about []() (or vice versa) is an obstacle to the user that shouldn't exist. But if everything else is good from a user perspective then I definitely want to keep that stuff.
Also, it could support both of these syntaxes at once - they don't conflict.
The "() is more common than [] in regular text, so it makes parsing-sense to start a special markup code with []" is going to help me with that, though. That particular mnemonic only works for coders, though :)
Also, doesn't MediaWiki syntax sometimes also have a pipe | between the url and the text in a link? Or is that only for internal links?
It's 2012 and we have HTML 5 compliant WYSIWYG text editors. We no longer have to write plain text littered with special codes, for the purpose of running through a parser, to produce HTML which looks nice on a web page. Maybe it made sense a decade ago when web forms had terrible editors, but not anymore. I think Joe Internet writing blog posts and forum replies would agree with me.
For developers, a README file in plain text looks great everywhere, and avoids any Github vs Bitbucket vs Assembla display issues. If you need to write structured documentation for a system, there's probably already a designated markup language you're supposed to use, so Markdown doesn't help there.
Markdown just works and always produces clean, easy to style code.
My blog is just a directory of Markdown files that get compiled into HTML files. No cumbersome WordPress or other blog software installation I need to maintain, just copy the HTML to the server. Simple.
Same thing with documentation. Write once, from anywhere, output clean code to multiple formats: HTML, LaTeX (and PDF, DVI, etc. from there). No need to worry about making sure complex documentation toolchains are installed, just a Markdown compiler. A huge advantage of Markdown is that the source is as readable as the output. JavaDoc, for example, sucks in this regard if you try to do any formatting (lists, tables, etc.). If I am reading the source code--and I prefer to read the source if I can--then the javadoc is often useless for complex functions where it is most needed, because the HTML formatting is unreadable. I am actually building my own documentation system because of this (see https://github.com/jdbernard/jlp if you are interested).
I understand the question and wonder why it attracted down-votes. The executive summary of my answer: light markup is used in places other than text areas on Web sites.
Many of us use Markdown to keep formatted text in, and, ultimately, to generate valid html markup from. Markdown is not just for textareas in Web pages. Whole books have been marked up in markdown.
Markdown files are just text files with 'explicit' markup within them and so will always be accessible/editable/processable in the future. Markdown is 'flexible' in the sense that it allows alternative markup for the same output, and is 'light' in the sense that a limited range of styling and block formats are available. Compare that with LaTeX which shifts with each tex-live release, and which means that markup written some years ago may not format without hand editing on a modern LaTeX release.
There is the reference implementation available from Gruber's site, and, as others have mentioned, pandoc implements markdown in a way that can be extended. I'm dubious of the need for an 'official' specification; I rather like the scrappy streetwise nature of Markdown.
What? Certainly details of individual packages may change, including addition or removal of functionality, over time, but I don't even know what it means to say that "LaTeX shifts". The underlying markup language for LaTeX, which is the same as that for TeX, is very flexible, but has had the same flexibility since at least 1982, if not 1978.
I understand the idea that built-in extensibility leaves a platform, to some extent, to blame for its plug-ins; but many people in this discussion have proposed building some sort of plug-in architecture for Markdown, too.
Why don't we start with something like a visual HTML or DocBook editor, and then apply parsers to generate plain text, PDFs, etc?
Is it because we as programmers live in a plain text environment and are happy with the status quo, or is it because the tools aren't good enough?
For techie types it is familiarity: as a programmer I've used mark-down style syntax in code comments (and emails back when plain text was the norm) for years so it feels natural. I can type and edit text with mark-down faster then with full WYSIWYG control (even if the editor has good keyboard shortcuts). Also I can use the same text+markdown in code, emails and documentation that is desired to look a little fancier - I don't need to reformat for a second audience. Markdown isn't perfect, but for me it is a better solution for some things than WYSIWYG.
Simplicity and resulting cleanliness of mark-up can be an issue sometimes. I've seen WYSIWYG editors get tied in knots over bad HTML that they generated themselves.
Simplicity is another bonus for sites taking in user content, particularly sites like the StackExchange family. By effectively having a while-list of formatting options you can keep a consistent look on your site more easily. If you aren't sure what problem this solves, think Geocities.
It makes security a little easier to: you don't have to worry so much about potential injection attacks that are very possible if trying to generate/accept full HTML and filter it for the bad stuff afterwards (you still have to be careful to think about these issues of course, but your available attack surface is much smaller and therefore easier to manage).
> Github vs Bitbucket vs Assembla display issues
This is the very problem that people are actually discussing - competing interpretations make mark-down less useful then it could otherwise be, and a global standard might help that issue if it gets widely adopted. Of course many incompatible interpretations already exist and any new standard will face adoption friction, so we may end up in a situation like parodied in http://xkcd.com/927/ (it worked well enough for Javascript in the long run, but I think there were fewer common and slightly incompatible implementations of that at the time than there are of markdown now).
When users copy/paste from MS Word into HTML editors the paste often keeps the font from the Word document which is not the proper font for the html page. That results in a messy page layout.
In my experience it's easier for users to understand Markdown syntax than to understand the "hidden codes" in HTML editors when they want to fix the layout.
This is extremely important for writing and also for fall back purposes. Any time there is any kind of a problem in writing or reading or interpreting, just fall back to source.
People can say html is human readable but the difference is huge. I didn't even know that markdown is parsed on hacker news, I just used stars as underscores as natural formatting.
There's also potentially much much less overhead. Ultimately, browsers and your cell phone apps etc should just format markdown natively, meaning huge performance boosts, as html, css, js etc has just become so complex for many tasks as a consequence of do-everything.
My phone's email client struggles browsing back and forth of what should be just a few small text files and file attachments. It uses way too many cycles since it's also a jpeg and css and js renderer with scaling and everything. As someone who's used text email and news groups, lightweight formatting is very natural and informative. Those services would not have been possible in the past if they had been bogged down with large amounts of excess complexity.
I guess reducing overhead is not sexy because it doesn't involve new features. In the future everybody will just create their typed static oneliner email as 10000x10000x10000 pixel 100 fps 3D movies of 2 minute length (you see, that's the time it takes to read it), and if some supposedly slightly technically literate person asks that perhaps they should use a word 2020 .doc instead for reduced file size, the answer is "but you can't do a space station flythrough in it".
Personally, I don't trust or use (if I can avoid it) any on-the-web WYSIWYG text editor, because a) they tend to be slow and clunky and b) it's difficult to predict what HTML they will turn out a lot of the time, and I will often end up needing to edit that HTML.
Also, for common blogging / commenting tasks, Markdown is as fast or faster to use than WYSIWYG editors. I've been faffing around with a forum that won't let me use Markdown recently, and forces me to use its clunky WYSIWYG editor to add lists into my posts. It didn't take long for me to start yearning for the simplicity of linebreak - asterisk - item rather than a multiple-click WYSIWYG process.
YMMV, but I've never been fond of WYSIWYG editors. I'd rather say what I mean, instead of say what this other guy thinks I mean.
Markdown and similar formats can also easily be used to produce other output than HTML.
And Markdown is plain text, and that's another appealing factor: Even if you read it somewhere that won't render the Markdown, it is still readable. The same can't be said about HTML.
I love the ability to scratch something down in Markdown really quickly, and being able to convert it to LaTeX if I want to, without any hassle.
I'd love to write blog posts in Markdown. Google+ uses the italic and _bold_ syntax in their own posts, which is very useful.
I don't exactly know why it's better, to be honest. Maybe WYSIWYG editors are inherently dependant on their own implementations and rely on people learning the interface.
Text-As-An-Interface is beautiful in it's simplicity. Markdown is the perfect mix of simple markup without the insanity of BBCode.
2. It sounds like you are only concerned in producing HTML. I like using markdown for everything. Using the excellent Pandoc I produce both HTML and pretty PDFs from markdown text, with nice looking formulas too. (PDFs are produce through LaTeX, for HTML Pandoc offers several options, I use MathJax).
Pandoc looks great. So why not write in HTML or Docbook and then convert to plain text / RTF / PDF etc?
I often find myself writing short plain text files, usually quick personal notes. Occasionally they grow larger than what I'd call a "short" plain text file, and at this point two extra requirements pop up:
1) they are going to need some structure.
2) odds are, that I'm going to want to show this document to someone else at some later point in time[1].
Markdown offers (to me) by far the easiest transition from "basic plain text note" to "slightly longer formatted document", because most times, all I have to do is continue writing the way I was doing already.
[1] corollary: as programmers know, when reading their own code, "someone else" also includes you in three months time.
1. You can pry my {Emacs,vim} from my cold, dead hands.
2. Version controlled markdown is much easier to deal with.
I know git can be configured to use an arbritary programme for diff - just not sure where to find one...
Meanwhile, I write the documentation for my software in Markdown, because it is the fastest way for me to get structured text from my mind to my machine. And with the help of pandoc, it's convertible into all other text formats, from HTML to PDF and LaTeX etc. (Although the html-with-images to PDF support of wkhtmltopdf is even better, so it's Markdown => pandoc[for HTML] + wkthmltopdf [for HTML to PDF]).
The main appeal is Markdown lets us write formatted text without ever having to use the mouse or figure out what the current WYSIWYG editor's key commands are. Often times -- like making nested bulleted lists -- there just plain aren't any key commands. I'd much rather type a few square brackets and create a nice hyperlink on the fly than invoke some silly menu, paste in my link in the window that pops up, create the display text, etc etc.
If you exclude HTML tags, Markdown is safe. People can't inject code.
I know this can be done for HTML as well, but it's harder.
Because of the chickenshit way he attempted to escape things by replacing them with their MD5 hashes and then switching them back, you can encode anything you want to be output by the markdown processor. Reddit was owned by a viral XSS comment because of this in 2009, it's been exploited a few other times since.
http://www.f-secure.com/weblog/archives/00001777.html
http://blog.reddit.com/2009/09/we-had-some-bugs-and-it-hurt-...
Neither works very well with version control or scenarios where you need compatibility between different system.
Plain text ALWAYS works.
I haven't looked at the implementations of these, but they are most certainly grammar-based, not regex-based.
Specs like this should be written as executable reference implementations in a well defined programming language. This can very well be human readable, and should be done without regard for efficient execution. It's less ambiguous, amenable to automated conformance testing, and is easier to evolve than a natural language document.
Everything outside that (e.g. timing) would be fair game to optimize. Moreover, as I hinted at, a spec should not be optimized for speed or efficiency, but rather readability and unambiguity.
So you agree it wouldn't work for specs to be "written as executable reference implementations in a well defined programming language"?
> Everything outside that (e.g. timing) would be fair game to optimize.
But sometimes you want to specify performance characteristics.
The simple fact is that without just embedding a traditional spec within it, an implementation cannot tell you what aspects of its behavior are "specified" and which are just implementation details. This makes reference implementations inadequate specifications. That's all I was getting at.
No, then I would be disagreeing with myself. :)
> But sometimes you want to specify performance characteristics.
For a Markdown processor, hardly. But...
> ... cannot tell you what aspects of its behavior are "specified" and which are just implementation details.
That is a fair point indeed. I was merely disagreeing with the linked article in that a spec should be in a natural language, which I still hold to. Such specs generally have problems with under specifying, leaving too much up to implementation, which often ends with a few main implementations sort of becoming the actual practical spec. Look at browsers over the history for many examples. Your point seems to be that reference implementations might overspecify instead. This is true, but is easier to compensate for IMO. For example with clear types, natural language parts or even by conformance test suites being part of the specification.
One implementation can never be enough.
What people really want is for all Markdown implementations to be basically the same so they don't have to learn any implementation-specific ideosyncrasies to switch from one to another.
The problem though, as I see it, tends to go away under certain circumstances. And the circumstances, I think, are these:
* If you've got a strong implementation, robust, actively-maintained, and runs fast,
* is easy to obtain, install, and use,
* is well tested and well documented, and
* has just the right blend of sensible additions to the syntax (for example, tables, def lists, LaTeX math, etc.) --- done tastefully,
then folks will just use that, model their own implementations after that, and just overall start considering that to be the standard.
I think this has been slowly and steadily happening with Pandoc.
And, aside from all that, two additional "killer features" that Pandoc seems to have over other implementations:
1. it can convert to/from other doc markup formats, thus making it easy to just convert your existing docs to pandoc-markdown and then use that as your master source format to generate other formats you might need; and
2. with its carefully-chosen set of additional features, it has been slowly proving itself capable of being a replacement for raw LaTeX for certain types of longer technical documents.
My understanding is that there's even some features in the works (for the next release) for converting between markdown dialects --- which would make it even easier to convert markdown files of various flavors into plain standard pandoc-markdown.
So, if you're looking for a standard, I'd suggest that it's for the most part already here. :)
Have a look at Markdent - An event-based Markdown parser toolkit.
https://github.com/hypernumbers/erlmarkdown
No regexps.... except for url's and emails :(
_markdown_version_ Rockdown_1.0
This would allow processor implementers to support more than one markdown version (for a transition period or in general).This is almost what you want:
http://news.ycombinator.com/item?id=555153
EDIT: wrong link :(
The point here is that Markdown doesn't have a spec, nor do any of its variants to my knowledge, so I was proposing to come up with some Markdown-like language that does have a spec. Under discussion here is the more ambitious (but also appealing) plan of writing an official spec for Markdown, the same way JavaScript got a spec in the form of ECMAScript that we now identify with JavaScript itself.
A spec is a long, tedious, human-readable document that explains the behavior of a system in unambiguous terms. Specs are important because they allow us to reason about a language like Markdown without reference to any particular implementation, and they allow people to write implementations (Markdown processors) independently that behave identically. The Markdown Syntax Documentation is not a spec (it's highly ambiguous), nor is any implementation (not human-readable; some behaviors are probably accidental or incidental and difficult to port perfectly). The hard part of writing a spec is codifying the details in English, and secondarily making decisions about what should happen in otherwise ambiguous or undefined cases.
My motivation for working on a Markdown spec is first and foremost avoiding "bit rot" of content, which happens when we write content against one Markdown implementation and then later process it with another. We don't have this concern with HTML, JSON, or JavaScript, or at least we know what bounds to stay within to write code that will work on any implementation. This is achieved through specs, even if only implementers ever read them.
I would love pointers to Markdown processors that are implemented in a more principled way than the original code, for example using standard-looking lexing and parsing passes, but that still handle nested blockquotes and bullet lists together with hard-wrapped paragraphs.