News for hackers. Stuff that matters.
Re:Better performance in App Engine with new Lisp language C... by mcav on Tuesday May 11, 2010 @01:02PM (#1338505)
A question I've been meaning to ask but didn't want to make a completely new post about it:

If you could write a server (not a plain HTTP webapp necessarily) in Clojure, Python, or Node.JS, provided you know all three, which would you choose and why?

Re:Better performance in App Engine with new Lisp language C... by francoisdevlin on Tuesday May 11, 2010 @01:08PM (#1338528)
I'd go with Clojure if you need any of the following:

1. Concurrency 2. The most (largest qty of) available FOSS libs anywhere 3. Speed 4. The ability to extend the language 5. WORA 6. Text/string processing

Python will win in scripting/*NIX integration, though.

Don't really know enough about Node.JS

Re:Better performance in App Engine with new Lisp language C... by strlen on Tuesday May 11, 2010 @03:35PM (#1338975)
> 2. The most (largest qty of) available FOSS libs anywhere

Quick heads up. The ease of FFI (in Clojure's case with Java) is not unique to Clojure and it isn't necessarily a great thing. It's also an issue with Common Lisp: as FFI with C is easy, there's less incentive to develop idiomatic libraries for the language. E.g., Common Lisp lacks a standard modern way to network I/O, but it's largely been ignored as larger users of the language e.g., ITA could always build and standardize upon an internal C-based extension.

That being said, Clojure is great for reasons beyond running on the JVM: a unique approach to concurrency, incorporation of ML-family features, first order data structures beyond lists (maps, vectors) and the fact it's a Lisp-1 with a great macro facility. I'd love to see Clojure also go beyond the JVM and onto CLR, Parrot and LLVM.

Interestingly enough the other prominent "next generation Lisp", arc, is also a Lisp-1 with true macros (as opposed to Scheme's hygienic macros).

Re:Better performance in App Engine with new Lisp language C... by prospero on Tuesday May 11, 2010 @04:04PM (#1339047)
It's very common to write an idiomatic wrapper for a useful Java library. Your complaint seems to be that those wrappers won't survive the transition onto another platform. I don't really see why that's a problem.

Why target another platform if not for its libraries? If Clojure is exactly the same on the JVM as it is on LLVM, why should it ever leave the JVM?

Re:Better performance in App Engine with new Lisp language C... by mahmud on Tuesday May 11, 2010 @07:12PM (#1339554)
Common Lisp lacks a standard modern way to network I/O

http://common-lisp.net/project/usocket/

Re:Better performance in App Engine with new Lisp language C... by jgg on Tuesday May 11, 2010 @11:14PM (#1340047)
Common Lisp has no standard way of doing Network I/O. usocket is one of several available socket libraries which don't exist in the ANSI spec.
Re:Better performance in App Engine with new Lisp language C... by mahmud on Wednesday May 12, 2010 @10:14AM (#1340975)
It is a de facto standard. Everyone uses it; just like the Metaobject protocol.
Re:Better performance in App Engine with new Lisp language C... by jgg on Wednesday May 12, 2010 @11:24AM (#1341163)
If the usocket library suddenly disappeared, along with all the current Lisp implementations, there would be no guarantee that someone making a new CL implementation had to implement the library in a way similar to usocket, or even implement a socket library at all. It's the same as saying "everyone uses Emacs to edit Common Lisp, so it's the standard Common Lisp editor".
Re:Better performance in App Engine with new Lisp language C... by mahmud on Wednesday May 12, 2010 @09:35PM (#1342932)
What is the standard way for network I/O in C, C++, Python, PHP and Ruby?

I am not in the business of conjuring up hypothetical situations about the demise of Lisp implementations and libraries; that doomsday scenario of having to recreate civilization from specs is simply not in my contingency plan :-)

Never underestimate the power of convention, de facto is the rule, rather than the exception.

Re:Better performance in App Engine with new Lisp language C... by rglullis on Tuesday May 11, 2010 @01:16PM (#1338561)
I really have a hard time seeing Python losing to Clojure on (2), (5) and (6). What kind of software/library would you find on Clojure but not Python? How is Python not WORA? Where does Python lack in text processing?
Re:Better performance in App Engine with new Lisp language C... by felixmar on Tuesday May 11, 2010 @02:39PM (#1338815)
Not Python specific but languages that regularly interface with C code are often troublesome on Windows because the library writer assumes Posix functionality is available or the library depends on Unix libraries that are at best badly ported to Windows.

Haskell is one language where i encountered that too often, whereas Clojure's Windows support does not feel second rate. The portability of the JVM and Java's libraries combined with a joyful functional language like Clojure is very attractive.

Re:Better performance in App Engine with new Lisp language C... by francoisdevlin on Tuesday May 11, 2010 @01:20PM (#1338573)
Java & the JVM give me 2 & 5. While Pythons text processing is great, I think functional composition & juxtaposition give Clojure and edge. Plus, strings are simply sequences, so the core language works well with them too.
Re:Better performance in App Engine with new Lisp language C... by jwr on Tuesday May 11, 2010 @05:06PM (#1339198)
Python has no parallelism story. Once you start using the Clojure parallel primitives, there is no going back: you will never want to deal with managing threads and mutexes again.

It's kind of similar to the feeling you get once you've programmed in a language with GC. You really don't want to go back, unless you have to.

Re:Better performance in App Engine with new Lisp language C... by KirinDave on Tuesday May 11, 2010 @01:42PM (#1338643)
> What kind of software/library would you find on Clojure but not Python?

Java has a lot of great libraries for which there are not terrific examples in Python. Clojure gets all of them.

> Where does Python lack in text processing?

Clojure handles large strings better than Python (unless you start using substrings, in which case an underlying java bug hits you).

(oh, and Clojure is not a deliberately and comically crippled language).

Re:Better performance in App Engine with new Lisp language C... by rufugee on Tuesday May 11, 2010 @02:52PM (#1338850)
How is Python deliberately and comically crippled? I'm honestly curious.
Re:Better performance in App Engine with new Lisp language C... by hugoc on Tuesday May 11, 2010 @03:44PM (#1338999)
I think he may be refering to the GIL.
Re:Better performance in App Engine with new Lisp language C... by KirinDave on Tuesday May 11, 2010 @03:51PM (#1339012)
That and certain deficiencies spawned by a desire for “simplicity.”
Re:Better performance in App Engine with new Lisp language C... by technomancy on Tuesday May 11, 2010 @05:01PM (#1339186)
Don't forget one-line lambdas.
Re:Better performance in App Engine with new Lisp language C... by KirinDave on Tuesday May 11, 2010 @05:02PM (#1339188)
It is true. I really, really, really do not like those. I dislike them to the point of irrationality.
Re:Better performance in App Engine with new Lisp language C... by sreque on Tuesday May 11, 2010 @03:08PM (#1338900)
All your repliers seem to forget that Jython exists and can be run on the app engine as well, putting Jython on par with Clojure in terms of available Java libraries.
Re:Better performance in App Engine with new Lisp language C... by melling on Tuesday May 11, 2010 @03:28PM (#1338951)
Let's not forget JRuby. Java might be stalled but the JVM is really starting to show a lot more value.
Re:Better performance in App Engine with new Lisp language C... by jacquesm on Tuesday May 11, 2010 @03:52PM (#1339016)
Interestingly in the beginnings of java the JVM was just another abstraction layer.

Now that the JVM is really running just about anywhere it has become a viable platform to target, and java is just one more language that targets it.

Clojure, groovy and scala are built on top of the JVM, but there are also plenty of languages that are available on mutliple platforms.

Whoever came up with just-in-time compilation deserves a gold medal, without that none of this would have ever happened.

Re:Better performance in App Engine with new Lisp language C... by btilly on Tuesday May 11, 2010 @04:16PM (#1339072)
Whoever came up with just-in-time compilation deserves a gold medal, without that none of this would have ever happened.

My understanding is that JIT in Java is a variation on what the Smalltalk world called dynamic optimization. That was introduced into Smalltalk by Peter Deutsch and Allan Schiffman in the 1980s, but I don't know whether or not they invented it independently.

Re:Better performance in App Engine with new Lisp language C... by jacquesm on Tuesday May 11, 2010 @06:00PM (#1339357)
Yes, that's true. I'm not claiming that the just-in-time compilation technique was invented specifically for java, merely that it was a very useful addition to it and that whoever did invent it probably helped saving the platform.

That invention pre-dates java itself by quite a few years, but then again, java has a lot of elements in it that were considerably older, iirc there was UCSD pascal P-code, which was another virtual machine like environment with an abstracted machine, and the 'forth' language which has quite a bit in common with how the JVM operates on the lowest level.

Re:Better performance in App Engine with new Lisp language C... by btilly on Tuesday May 11, 2010 @07:47PM (#1339659)
I was giving you the information I had that would help you if you you wanted to track down who invented it.
Re:Better performance in App Engine with new Lisp language C... by jacquesm on Tuesday May 11, 2010 @07:58PM (#1339680)
Right, sorry, the endless downmods made it seem as though that comment was interpreted as though I made it seem as though the jit was invented after or because of java. Of course it wasn't. (see the other comment above).

I was just reading that 'early history of smalltalk' thing the other day, and it struck me as though really, since the 60's there hasn't been that much progress at all.

The 'mother of all demos' combined with the 'dynabook' (iPad?) really pretty much covered everything with the exception of the mobile phones.

Sure, we all have the equivalent of several Cray-1's in our houses now (or even in our pockets), but conceptually we are still were we were back then.

Everything looks great, we're burning billions of cycles on spiffy user interfaces and showing movies.

But under the hood it's old hat.

That's why I keep circling around and around trying to find my new 'home', the environment that I think will last me for the next 10 or 20 years.

I haven't found it yet. All I see is endless repetition, configuration files, minor tweaks. Rarely a bold move (fleet comes to mind, but I don't think it will ever be a commercial success). Programming seems to be mired in endless detail, fiddling the bits and tweaking things to get a link here or a widget there and to get them to talk to each other. User interfaces have become the main focal point, when they weren't (or at least when they weren't as pretty) we were happy if stuff just worked.

I feel like a homeless guy looking for a place to stay.

Clojure might be it, I don't know...

but I'll try it for sure. No point criticizing the soup before you've eaten it.

Thanks for digging up that jit reference.

Re:Better performance in App Engine with new Lisp language C... by agazso on Tuesday May 11, 2010 @05:37PM (#1339286)
Just-in-time compilation is not a new idea. According to the book "The Practice of Programming" (by Brian W. Kernighan and Rob Pike, Ch9, pg. 242):

"Generalizing the idea for the full set of operations, we can write an on-the-fly compiler that translates the current regular expression into special code optimized for that expression.

Ken Thompson did exactly this for an implementation of regular expressions on the IBM 7094 in 1967. His version generated little blocks of binary 7094 instructions for the various operations in the expressions, threaded them together, and then ran the resulting program by calling it, just like a regular function."

Re:Better performance in App Engine with new Lisp language C... by rglullis on Tuesday May 11, 2010 @04:57PM (#1339175)
It seems you are confusing JIT and bytecode interpreters.

Why wouldn't Scala/Clojure/Groovy et al be possible without JIT? It's possible to run on the JVM anyway, whether said VM supports JIT or not.

Re:Better performance in App Engine with new Lisp language C... by jacquesm on Tuesday May 11, 2010 @05:57PM (#1339349)
They'd be possible, but they wouldn't be nearly as fast without it.

Java before JIT was added to the JVM was dead slow.

Re:Better performance in App Engine with new Lisp language C... by lg on Tuesday May 11, 2010 @07:46PM (#1339657)
speaking of which. I'm testing a big multithreaded jruby app I wrote as I type this, and it's running very solidly. anytime I need parallel I'm going straight to jruby.
Re:Better performance in App Engine with new Lisp language C... by dagw on Tuesday May 11, 2010 @03:04PM (#1338891)
Many python libraries have a C component which makes them a no go when it comes to app engine. Two big examples are image manipulation and numeric and scientific computation. The pure java libraries in these categories are much faster and more complete than the pure python libraries.
Re:Better performance in App Engine with new Lisp language C... by icey on Tuesday May 11, 2010 @01:14PM (#1338554)
Node is pretty interesting and seems to have pretty good performance, but I wouldn't build anything critical on it until at least version 0.2.

As far as I know, there are semi-frequent breaking changes. Ryan Dahl has said that API stability should come with version 0.2.

Re:Better performance in App Engine with new Lisp language C... by caffeine on Tuesday May 11, 2010 @02:13PM (#1338745)
"I wouldn't build anything critical on it until at least version 0.2"

Ha! It's a brave new world. It used to be you needed a whole version number under your belt before critical stuff could be built..

Re:Better performance in App Engine with new Lisp language C... by kixxauth on Tuesday May 11, 2010 @03:14PM (#1338910)
Been talking to people working with node.js and the general vibe is 'it rocks'. Although, there is a lot of breakage it's for the best.
Re:Better performance in App Engine with new Lisp language C... by rikthevik on Tuesday May 11, 2010 @03:03PM (#1338884)
Maybe they're taking the emacs approach to version numbers.
Re:Better performance in App Engine with new Lisp language C... by andrewvc on Wednesday May 12, 2010 @12:01AM (#1340134)
Agreed, node development is going at a breakneck pace, but you should count on your code not working 6 months down the line without at least as few tweaks.

BTW, the new Buffer objects are awesome, and provide massive speed increases with binary data.

Re:Better performance in App Engine with new Lisp language C... by draegtun on Wednesday May 12, 2010 @04:32AM (#1340502)
The most (largest qty of) available FOSS libs anywhere

Do you literally mean anywhere or compared to other GAE languages?

If so do you have a rough estimate on how many Java/Clojure FOSS libraries/modules/packages there are?

Re:Better performance in App Engine with new Lisp language C... by mattculbreth on Tuesday May 11, 2010 @02:09PM (#1338733)
I actually did this purely for fun. Could have done Python but I've been meaning to learn Clojure anyway. Check out http://github.com/mattc58/animate for the beginning of my web server in Clojure.
Re:Better performance in App Engine with new Lisp language C... by jacquesm on Tuesday May 11, 2010 @03:42PM (#1338992)
Super, thanks! I've asked if there was some kind of a reference web application in clojure that I could study before to show me 'how it's done', but got no takers, so I'll use your code as a study object if you're ok with that.
Re:Better performance in App Engine with new Lisp language C... by weavejester on Tuesday May 11, 2010 @05:37PM (#1339287)
Mark McGranaghan's Ring library might be a good place to start. The Ring spec is the de facto base library for web applications in Clojure, much as Servlets are in Java, and Rack is in Ruby.
Re:Better performance in App Engine with new Lisp language C... by jacquesm on Tuesday May 11, 2010 @06:31PM (#1339456)
That's another helpful pointer, thanks again!

(found it here: http://github.com/mmcgrana/ring )

Re:Better performance in App Engine with new Lisp language C... by mattculbreth on Tuesday May 11, 2010 @05:00PM (#1339183)
Sure absolutely. It's been a good learning experience. As soon as I finish the blogging features of it I'll start blogging about it. :)
Re:Better performance in App Engine with new Lisp language C... by jacquesm on Tuesday May 11, 2010 @06:34PM (#1339465)
This is something I've been noticing as long as I've been prorgramming. If you are programming a tool then the tool will be a requirement for building the tool.

For instance, you can't really build an assembler for a new architecture from scratch without some form of assembler and so on.

The bootstrapping sequence seems to be endless, you could be blogging about writing your blogging tool if you had it already. We always seem to need some scaffolding underneath it, just one more turtle on the stack.

No more hand calculating branches these days, /me is happy ;)

Re:Better performance in App Engine with new Lisp language C... by mattculbreth on Tuesday May 11, 2010 @07:57PM (#1339678)
This is all for fun and for great justice, so I'm cool reinventing the wheel.
Re:Better performance in App Engine with new Lisp language C... by kixxauth on Tuesday May 11, 2010 @03:44PM (#1338997)
Keep in mind the loading request scenario for running GAE apps that are implemented on the JVM.

http://code.google.com/appengine/kb/java.html#What_Is_A_Load...