News for hackers. Stuff that matters.
Re:Experience porting 4k lines of C code to go by kkowalczyk on Tuesday June 21, 2011 @04:37PM (#2680298)
(I'm the author of the article and also a heavy Python programmer).

That's not an easy question to answer.

If you were to do a faithful port i.e. using using the same techniques as C/Go code, it would be very close. upskirt uses a traditional lexing/top-down-parsing approach. There's nothing in Python syntax that makes writing such code more compact than in Go. It's a bit tedious to write but the benefit is that (in C/Go) it gives the best speed because it minimizes the number of times each source character is looked at.

If you were to use a different approach e.g. brute-forcing your way through the text several times with regexpes, which is the most popular way of doing that in dynamic languages, the code clocks at ~2 thousand lines (like the implementation I use for my home-grown blog system https://github.com/kjk/web-blog/blob/master/markdown2.py)

If you were to use this technique in Go, the code would probably end up smaller than the manual lexing/parsing approach used in upskirt, but it would be significantly slower.

Interestingly, upskirt approach would probably be slower in Python than the regexp approach, because regexpes are heavily optimized C code and looking at individual characters of the string in Python code isn't particularly fast due to python interpretation overhead.

Re:Experience porting 4k lines of C code to go by knome on Tuesday June 21, 2011 @05:31PM (#2680513)
> because regexpes are heavily optimized C code

AFAIK, the python re module does not have a C language component. It is a pure python regular expression implementation.

Re:Experience porting 4k lines of C code to go by kkowalczyk on Tuesday June 21, 2011 @05:55PM (#2680601)
Re:Experience porting 4k lines of C code to go by enneff on Tuesday June 21, 2011 @06:50PM (#2680745)
Most dynamic languages use a C Regular Expression library. That's why they can do so well on Regular Expression benchmarks.

V8 is a notable exception. They use their code generation pipeline to JIT compile regexps to machine code, making it the fastest regexp engine around bar none (I think).

Re:Experience porting 4k lines of C code to go by azakai on Tuesday June 21, 2011 @07:51PM (#2680911)
AFAIK all modern JS engines JIT regexps: V8, SpiderMonkey and Nitro.
Re:Experience porting 4k lines of C code to go by pavpanchekha on Wednesday June 22, 2011 @03:59AM (#2682195)
I'd like to put in a word for CL-PPCRE, the Common Lisp regex library. It does the same except using the CL optimizing compiler, so also achieves amazing speed.

Libraries don't appear in the Comp.Lang.Benchmark Shootout, so CL doesn't do so well.

Re:Experience porting 4k lines of C code to go by silentbicycle on Tuesday June 21, 2011 @11:37PM (#2681597)
What about LPEG? It has a superset of the functionality of most regexp engines, yet is able to handle many things linearly that they would handle in quadratic time.
Re:Experience porting 4k lines of C code to go by silentbicycle on Tuesday June 21, 2011 @11:34PM (#2681585)
Performance-wise, any regex-based implementation may spend a lot of time backtracking and re-parsing data that a DFA hand-coded parser would handle in linear time.
Re:Experience porting 4k lines of C code to go by MostAwesomeDude on Wednesday June 22, 2011 @12:19AM (#2681703)
I'd like to point out that there are libraries (PyPEG, PyParsing, and PyMeta) which take EBNF-style descriptions of grammars and produce decently fast parsers, PEG-style. PyMeta's my personal favorite, since it both takes a straight text description, and also uses OMeta instead of ENBF, turning parse and lex into a single operation.

Of course, the downside is that formal grammars are often a bad fit for custom languages of the web, which tend to have incomplete/undecideable/context-dependent grammars. This would work for Markdown, though, since it does have a parseable grammar.

Re:Experience porting 4k lines of C code to go by keenerd on Wednesday June 22, 2011 @11:08AM (#2683432)
I agree. (Though my favorite is FuncParserLib, very haskellish.)

Using a real parser could probably cut half the code. Been meaning to make my own Markdown parser, everyone else's kind of sucks :-)