News for hackers. Stuff that matters.
Re:Clojure Implemented in Pure Python by frig8 on Wednesday February 29, 2012 @11:19PM (#3650895)
This is an exciting idea, but I think the home page focuses far too much on these hypothesized advantages of a dynamic VM. Not only are the advantages unproven, it's hard to see how this project represents anything novel in that aspect given that ClojureScript already exists.

The things that come to my mind are things like Numpy, Scipy, PySide, boost.python and all other sorts of Python bindings to C, C++ and Fortran code that aren't readily available on the JVM. Also, the startup time that the JVM can't match, the Python standard library and the other random bits people have developed that make Python such a wonderfully flexible scripting environment.

If you have excellent interop, there will be plenty of interest in this project regardless of how fast or slow it is. If it's fast, all the better.

Re:Clojure Implemented in Pure Python by gajomi on Wednesday February 29, 2012 @11:30PM (#3650914)
I agree. Native access to legacy linear algebra functionality exposed by Numpy is the #1 reason this project seems exciting to me. I don't really care if the bulk of the code is O(10) slower so long as the numerical bits are O(1000) faster, which is a typical comparison of LAPACK for the JVM equivalents.
Re:Clojure Implemented in Pure Python by mreid on Thursday March 01, 2012 @04:58AM (#3651561)
(Pedant warning)

I think you may be abusing Big-O notation a little here since, technically O(10) is equivalent to O(1000)β€”and both are O(1).

Re:Clojure Implemented in Pure Python by gajomi on Thursday March 01, 2012 @11:05AM (#3652625)
Haha, thanks for the warning. I am reminded of another time I ended up abusing the Landau notation in a numerical analysis class ( was it probability?) some years ago. The homework was to get and N^2log(N) algorithm of some kind, but I became concerned about certain large constant factors which I could determine exactly. However, these constant factor could be written as a series in another parameter, which I couldn't approximate in a certain limit and... well, you see where I'm going. There where lots of O's with slightly different semantics on the same paper.
Re:Clojure Implemented in Pure Python by Anderkent on Thursday March 01, 2012 @08:49AM (#3652018)
It's fairly clear in the context that he's using O(x) in the informal 'order of' meaning: 'order of 10', 'order of 1000' etc.
Re:Clojure Implemented in Pure Python by mreid on Thursday March 01, 2012 @09:53PM (#3655126)
I agree, and had no problem understanding his intended meaning but I was irked by the unnecessary misuse of technical notation to attempt to get across an idea that would have been perfectly easy to write without it (e.g., "10 times" and "1000 times").
Re:Clojure Implemented in Pure Python by regularfry on Thursday March 01, 2012 @02:10AM (#3651270)
The interesting thing about python is that while it's fast to start, it can be slow to stop. I've certainly seen that in CPython, I don't know if it's something that affects PyPy as well.
Re:Clojure Implemented in Pure Python by ScottBurson on Thursday March 01, 2012 @01:16AM (#3651147)
Huh. I think Common Lisp would be a much better choice than Python. It's much faster, for starters. And it lets you do some fairly low-level stuff if you want.

In general I think Common Lisp's virtues as an implementation substrate for other languages are much greater than most people appreciate. It is flexible, expressive, and fast. Its dynamicity comes in very handy. And some of its vices -- its sheer size, its lack of orthogonality, and its occasionally archaic naming conventions -- are much less problematic for a language implementation task than they are for general programming.

There are exceptions, of course. You wouldn't want to implement C++ in Common Lisp. But for dynamically typed languages it ought to be a leading candidate.

Re:Clojure Implemented in Pure Python by dbecker on Thursday March 01, 2012 @11:48AM (#3652849)
Interoperability between clojure and python is promising, because it lets you do lisp programming with python libraries and data structures.

If you want to do lisp programming with CL libraries, you could just use CL.

I'm not sure what the advantage of clojure in CL is over just using CL.

Re:Clojure Implemented in Pure Python by nickik on Thursday March 01, 2012 @03:31AM (#3651398)
I dougth CL-Clojure is faster then pypy-Clojure.
Re:Clojure Implemented in Pure Python by xyzzyz on Thursday March 01, 2012 @06:57AM (#3651769)
Why? CL compilers (SBCL at least) generate really fast code.
Re:Clojure Implemented in Pure Python by nickik on Thursday March 01, 2012 @07:43AM (#3651857)
It does not matter with how good a compiler you compile an interpreter it stays an interpreter. With Pypy you get a JIT Compiler. Sure you could maybe write a JIT with CL but thats a lot more work.
Re:Clojure Implemented in Pure Python by mbenkard on Thursday March 01, 2012 @08:01AM (#3651898)
I don't understand. Do you mean that you can't compile Clojure into Common Lisp the way you can compile it into Python/PyPy?
Re:Clojure Implemented in Pure Python by halgari on Thursday March 01, 2012 @08:31AM (#3651976)
You can, but I think you'd run into the same issues mentioned here: http://clojure-py.blogspot.com/2012/02/ive-been-asked-many-t...

Basically it reflection (aka dynamic dispatch) is a major performance killer if you don't implement a tracing jit.

Re:Clojure Implemented in Pure Python by mbenkard on Thursday March 01, 2012 @01:53PM (#3653448)
CLOS implementations have had to deal with the problem of extreme dynamism for a long time, so there has been quite a bit of optimization work associated with it. Polymorphic inline caches help a lot, for example.

That said, a good tracing JIT is probably better; but on the other hand, SBCL has profited from many years of optimization work on CMU CL. It would be interesting to see some real-world performance comparisons between SBCL and PyPy.

Re:Clojure Implemented in Pure Python by pavelludiq on Thursday March 01, 2012 @11:25AM (#3652739)
Wouldn't SBCLs type inference(+ type hints) help with this? I'm no compiler expert, but i think clojure(with the possible exception of multimethods, which are slow even on the JVM) would map well to CL(maybe even better than the JVM).
Re:Clojure Implemented in Pure Python by nickik on Thursday March 01, 2012 @11:28AM (#3652749)
PyPy is not just an Python-Interpreter. The PyPy Team build a Toolchain that can creat a JIT-Compiler out of a Interpreter (read some about the pypy project and you will be enlightened. Such a Toolchain does not exist for Common Lisp.
Re:Clojure Implemented in Pure Python by mbenkard on Thursday March 01, 2012 @02:02PM (#3653484)
Interesting! Thanks for the pointer.
Re:Clojure Implemented in Pure Python by ericmoritz on Wednesday February 29, 2012 @08:40PM (#3650586)
Here's a quick benchmark on my machine (a EeePC 1001HE). I used reduce1 with clojure-py because there doesn't seem to be a reduce BIF. I don't know if that effected this benchmark any:

Python:

    (time (reduce1 + (range 100000)))
    Elapsed time: 3882.57193565 msecs
    4999950000
PyPy:

    user=> (time (reduce1 + (range 100000)))
    Elapsed time: 259.984970093 msecs
    4999950000
Clojure via Java Hotspot:

    user=> (time (reduce + (range 100000)))
    "Elapsed time: 75.35225 msecs"
    4999950000
Re:Clojure Implemented in Pure Python by halgari on Wednesday February 29, 2012 @09:01PM (#3650634)
Those numbers a re bit small to get a full view of the way the pypy jit works:

  user=> (time (reduce1 + (range 100000)))
  Elapsed time: 903.011083603 msecs
  4999950000
  user=> (time (reduce1 + (range 1000000)))
  Elapsed time: 777.748823166 msecs
  499999500000
  user=> (time (reduce1 + (range 10000000)))
  Elapsed time: 8764.57095146 msecs
  49999995000000
  user=>

  Clojure via hotspot:
  user=> (time (reduce + (range 100000)))
  "Elapsed time: 174.768593 msecs"
  4999950000
  user=> (time (reduce + (range 1000000)))
  "Elapsed time: 267.60421 msecs"
  499999500000
  user=> (time (reduce + (range 10000000)))
  "Elapsed time: 1131.77367 msecs"
  49999995000000
  user=>
But yes, we still have some room for improvement.
Re:Clojure Implemented in Pure Python by halgari on Wednesday February 29, 2012 @10:37PM (#3650807)
Here's a blog post explaining the issues involved in a bit more detail: http://clojure-py.blogspot.com/
Re:Clojure Implemented in Pure Python by mark_l_watson on Wednesday February 29, 2012 @05:25PM (#3650022)
That looks nice. I have to ask: Clojure on the JVM is fast: can a PyPy runtime really compete after Hotspot has a chance to optimize?
Re:Clojure Implemented in Pure Python by Rayne on Wednesday February 29, 2012 @05:34PM (#3650051)
I'm less concerned with speed on this platform. I think it'd be excellent even if only suitable for small scripts that need to start up fast.
Re:Clojure Implemented in Pure Python by rbranson on Wednesday February 29, 2012 @06:09PM (#3650197)
All VM JITs compile to statically-typed machine code, so it's doubtful that building a VM from the ground up for dynamic code is going to yield major performance improvements. There are some enhancements that give VM-level visibility into dynamic invokation (like INVOKEDYNAMIC in JVM 7), but this has a relatively small performance impact on real-world codebases. In addition, the modern JVM is a sophisticated, highly-tuned machine with millions of man-hours on the books. It's unlikely to be eclipsed any time soon.
Re:Clojure Implemented in Pure Python by halgari on Wednesday February 29, 2012 @08:11PM (#3650522)
This is not true at all. PyPy implements a tracing jit that compiles specific loops for each set of types run through the interpreter. This means it is actually possible to have Python code that runs faster than C code in some rare cases: http://morepypy.blogspot.com/2011/07/realtime-image-processi...
Re:Clojure Implemented in Pure Python by ibdknox on Wednesday February 29, 2012 @08:26PM (#3650558)
Proving that there are cases where it's faster than C doesn't prove that it's faster than the JVM ;) There are plenty of cases where JVM code will be faster than C too...
Re:Clojure Implemented in Pure Python by mattdeboard on Wednesday February 29, 2012 @06:09PM (#3650199)
Er, what about Java interop? The notions of atoms, agents and refs? How are you going to translate these into "pure Python"?
Re:Clojure Implemented in Pure Python by dkersten on Wednesday February 29, 2012 @07:19PM (#3650389)
CLR Clojure doesn't have Java interop either. Neither does clojurescript. I don't think its a requirement for a Clojure implementation.

Atoms, refs and agents seem like an integral part of the Clojure language however.

Re:Clojure Implemented in Pure Python by halgari on Wednesday February 29, 2012 @08:06PM (#3650510)
There's absolutely nothing stopping us from implementing atoms, refs, and agents on clojure-py. Sure, the GIL won't help at all, but PyPy is working hard to fix that.
Re:Clojure Implemented in Pure Python by mattdeboard on Wednesday February 29, 2012 @08:36PM (#3650576)
Is this a hobby project or are you aiming toward getting adoption? I will confess that I don't understand the point. If we accept the Church-Turing thesis then there's no reason that by the time you catch up to Clojure's performance on the JVM it will have been optimized beyond the measurement you made. The only real guarantee you can make is that you will have added an abstraction layer to Python.

While I definitely think this is a cool project, when I think about using it in production it strikes me as more Sisyphus than Prometheus.

Re:Clojure Implemented in Pure Python by halgari on Wednesday February 29, 2012 @08:56PM (#3650622)
First of all, it should be mentioned, that this is not an abstraction layer. Clojure-py functions are actual Python functions (not classes as they are on the JVM). This means that the speed of clojure-py is almost exactly the same as python code.

Secondly, there is some benifit to not having to worry about static typing in a dynamic language. Anyone want to explain how to read a binary file in Clojure? Here's a hint, it takes the use of FileInputStream, DataInputStream. In clojure-py it's as simple as (py/open "foo.bin" "r").

And thirdly, why are we writing a dynamic language on a static VM? Fast Clojure code on the JVM these days takes little hints. You have to tag parameters with ^Integer and ^Double to kick the compiler type inference into gear. None of this is required on a dynamic VM. In fact, it's completely pointless.

Re:Clojure Implemented in Pure Python by mattdeboard on Wednesday February 29, 2012 @09:12PM (#3650655)
Ok, what's the use case? What is the benefit to this versus regular ol' Python? In what situations might I write clj-py assumine identical performance to Cpython or pypy, instead of just Python?

I agree that this is a really cool project and I plan on reading through the source when I have time strictly as a learning exercise, but I cannot think of a use case.

Re:Clojure Implemented in Pure Python by omaranto on Wednesday February 29, 2012 @11:20PM (#3650898)
Consider the case of a person that prefers programming in Clojure to programming in Python... :)
Re:Clojure Implemented in Pure Python by mattdeboard on Thursday March 01, 2012 @07:08AM (#3651787)
I am such a person. That's why I program in Clojure -- yes, including a couple of Clojure repos at work (after gettin sign off from my boss). I also love Python. Best tool for the job and all that. Rewriting Clojure in Python is as absurd to me as rewriting JavaScript or Ruby in "pure Python". You get nothing meaningful except different syntax.
Re:Clojure Implemented in Pure Python by halgari on Wednesday February 29, 2012 @09:36PM (#3650683)
I agree, if you're into Python, use python. However, I find things like macros, protocols, and multimethods drastically reduce the amount of coding that has to be done to accomplish a certain task. But I'm biased there ;-)
Re:Clojure Implemented in Pure Python by mattdeboard on Wednesday February 29, 2012 @10:08PM (#3650746)
Well, don't get me wrong, I love Clojure and have been writing more Clj than Py the last few weeks.
Re:Clojure Implemented in Pure Python by misfo on Thursday March 01, 2012 @10:55AM (#3652575)
If nothing else lispreader.py will be useful for reading Clojure data in Python. Think communicating with nREPL and reading the forms that come back
Re:Clojure Implemented in Pure Python by mattdeboard on Wednesday February 29, 2012 @07:58PM (#3650494)
I have to admit I'm not hip to all the newfangled doings with alternative Clojure targets. To me though java interop is one of the core differentiators for Clojure from other langs aside from its lispness. But, you're right, it's not required.
Re:Clojure Implemented in Pure Python by apatry on Wednesday February 29, 2012 @05:01PM (#3649911)
Really nice. I hope it will make clojure scripts faster to start than with the jvm and thus a real alternative as a scripting language.
Re:Clojure Implemented in Pure Python by moomin on Wednesday February 29, 2012 @06:01PM (#3650163)
ClojureScript scripts can already do this. :)
Re:Clojure Implemented in Pure Python by chadzawistowski on Wednesday February 29, 2012 @07:07PM (#3650363)
Are there many options for running ClojureScript scripts on a system outside of Node.js?
Re:Clojure Implemented in Pure Python by moomin on Friday March 02, 2012 @02:31PM (#3657970)
You want more command line Javascript runtimes? There's Rhino, but it's slower than Sunday with your in-laws...

Truth is, you only need one, as long as it's a good one.

Re:Clojure Implemented in Pure Python by politician on Thursday March 01, 2012 @03:19AM (#3651383)
Does every ES3 browser count?
Re:Clojure Implemented in Pure Python by DanielRibeiro on Wednesday February 29, 2012 @11:05PM (#3650862)
Re:Clojure Implemented in Pure Python by halgari on Wednesday February 29, 2012 @11:12PM (#3650874)
PyClojure (in the link) didn't actually take it to the extent we are. clojure-py goes so far as to implement all the standard collections in pure python. So [1 2 3] in clojure-py is a PersistentVector not a python list.
Re:Clojure Implemented in Pure Python by nickik on Thursday March 01, 2012 @03:33AM (#3651403)
Why? If your going to rewritte the collections, why not in Clojure? Speed?
Re:Clojure Implemented in Pure Python by halgari on Thursday March 01, 2012 @09:12AM (#3652102)
Mostly because we have to have these collections inside the compiler. So it's a bit hard to write collections in a language that has no compiler. Chicken/egg issue..
Re:Clojure Implemented in Pure Python by zephjc on Thursday March 01, 2012 @10:07PM (#3655163)
Well now that you have a compiler you can rewrite it in clojure-py to make clojure-py-in-clojure-py :)
Re:Clojure Implemented in Pure Python by dhconnelly on Wednesday February 29, 2012 @05:14PM (#3649974)
This is awesome.
Re:Clojure Implemented in Pure Python by stcredzero on Thursday March 01, 2012 @02:16AM (#3651284)
Please remember kids, language X implemented in language Y, probably means language X > language Y. (And if you question this, first implement a small, elegant language, and a large "pragmatic" one and then talk to me about it.)
Re:Clojure Implemented in Pure Python by ajuc on Thursday March 01, 2012 @03:35AM (#3651404)
Or language X > language Y in some domains, but language X is not practical in some other domains - see Lua/C[++]
Re:Clojure Implemented in Pure Python by perfunctory on Thursday March 01, 2012 @02:29AM (#3651304)
What do you mean by "language X > language Y"?
Re:Clojure Implemented in Pure Python by islon on Thursday March 01, 2012 @09:29AM (#3652158)
It means that after cast language X and language Y to an int value language X is greater than language Y (that's my understanding).