> What language systems in existence are using a true incremental or concurrent garbage collector? I know of three: Java, Objective C 2.0 (which just shipped with OS X Leopard), and the .net runtime. Not Haskell. Not Erlang. Not Objective Caml. [...], not Smalltalk not Ruby.

Oh, why could that possibly be. Hmm...

- Java: Sun

- Objective C: Apple

- .Net: Microsoft

Big companies with the manpower and determination to make improvements to their platforms. All the other languages have either grown out of one-man prototypes or academic projects. Building a good garbage collector is hard. It's difficult because you can't keep track of much data while you're allocating, you can't walk the graph of allocated objects because different threads are modifying the graph while you're working on it. When you then add a constraint that you can never stop the world for more than 5ms at the time it becomes a Difficult Problem.

Languages like Ruby, Python, Lisp and Scheme don't have good generational garbage collectors because they don't have the manpower to tackle Difficult Problems. (See for instance the global interpreter lock in Python. It's been 10 years now?)

To tackle a Difficult Problem you need (a) a few smart people and (b) get them to work on the problem for many consecutive hours for many consecutive days. Volunteers just can't afford to spend 50+ hours a week for half a year on a problem. A hundred people can chip in weekends and evenings for a year and make no substantial progress at all.

This is completely different from the hypothetical "sufficiently smart compiler" that is supposed to sense the intent of the programmer and optimize accordingly. A sufficiently smart compiler cannot compensate for bad algorithm design, so the performance differences will always be in the margins. The state-of-the-art in terms of garbage collection is decades ahead of what we see in Python, Ruby, Haskell, Scheme and so forth. The GC problem is simple: spread the computational overhead of garbage collection evenly so that the application doesn't stutter. Simple engineering problem with a concrete and clear goal. Just difficult to implement.