News for hackers. Stuff that matters.
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by mikekersehy on Monday September 16, 2013 @07:05AM (#6392610)
There's only so many GC engineers in the world, and the majority work in academia, industrial research labs and places like the Oracle VM team. So some web-app company isn't likely to hire one. Also they aren't interested in making some toy GC for a language they don't use.
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by justincormack on Monday September 16, 2013 @08:15AM (#6392849)
And this is why Silicon Valley has a hiring problem.

No, you hire someone who can code C and read the GC literature. The problem space is well understood. You are not breaking new ground, just applying known algorithms.

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by dasil003 on Monday September 16, 2013 @08:24AM (#6392875)
This is an absolutely ridiculous comment. Writing a GC for a dynamic language like Ruby is not just a question of knowing C and reading some literature. It's a massive problem to solve. This is not something that you double your burn rate in the hopes that 2-3 years down the line you can double performance of your application cluster. It's simply not worth it. Startups have immediate problems to solve or they die.

And BTW, the reason we have a hiring problem is not because we are too picky about specific experience, it's because when it comes to software engineering some people have it and some people don't, and the ones who don't are the ones whose resumes are consistently out there filling up inboxes, and the ones who do are rarely looking for a job, and when they are the Facebooks, Twitters and Googles of the world are throwing around $200k + benefits like pocket change.

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by coldtea on Monday September 16, 2013 @09:33AM (#6393125)
>This is an absolutely ridiculous comment. Writing a GC for a dynamic language like Ruby is not just a question of knowing C and reading some literature. It's a massive problem to solve.

No it's not. There are tons of good GC's around. You can even improve Ruby's GC with 1990 technology.

(E.g did the LuaJit guy solved a "massive problem"? Mostly by himself?)

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by jamesaguilar on Monday September 16, 2013 @12:00PM (#6393900)
Some people routinely underestimate what a team of two or three smart guys and gals can do if they get in a room together and really work on a problem.
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by Tuna-Fish on Monday September 16, 2013 @01:23PM (#6394416)
GC in Lua is a fundamentally much easier problem because it doesn't do threads.

Also, "Mike Pall can do it so everyone can" is not a very good argument -- Mike Pall is fucking brilliant, and quite a lot of his code is stuff that your typical great C coder can't ever do.

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by justincormack on Monday September 16, 2013 @09:26AM (#6393095)
The OP was suggesting that only someone who had worked at Oracle could solve the problem. At least you are saying that someone who "has it" might have a chance.

The issue of whether startups would fund it is a different question. Quite clearly the performance of Ruby is not in fact a huge problem for most people, and we haven't had a huge organisation that could fund stuff in the way, say, Facebook has with PHP.

I haven't looked at Ruby's GC, but I am guessing it was written by one person originally, and that could be done again. The poster at the top said it was simple. Its a big problem but as I said, its not a perfect solution needed just a better one.

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by sanderjd on Monday September 16, 2013 @11:30AM (#6393726)
There was a line from a talk about Rust[0] that really stuck with me, commenting on the design choice to only garbage-collect task-local heaps using reference counting with cycle detection: "we wanted to avoid spending the next 20 years writing another world-class garbage collector." It is not just a single person project to write a really good GC.

0: I think it was this one - http://www.infoq.com/presentations/Rust

Edit: wording and precision

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by steveklabnik on Monday September 16, 2013 @12:20PM (#6394013)
Note that in Rust it's not as big a deal as in Ruby because of two factors: GC usage is not idiomatic, and since it's task-local it won't ever stop the world.
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by sanderjd on Monday September 16, 2013 @08:34PM (#6396864)
Absolutely. My point being they seem to have purposefully worked the attributes you mention into the design of the language largely because of how hard it is to write a really good GC. The Rust team has lots of people who know C and are very capable of reading literature, and still didn't really want to tackle it, because it's hard.
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by AaronFriel on Monday September 16, 2013 @08:24AM (#6392879)
Implementing GC rarely seems as simple as "reading the literature" and implementing the algorithms they tell you. One problem is that you're working with an existing code base, and you can't afford to make mistakes in your GC. Sure, there are plenty of (good) GC algorithms out there that are public knowledge, but implementing it alone may be difficult, implementing it efficiently is another order of magnitude more-so.
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by astrodust on Monday September 16, 2013 @11:01AM (#6393581)
So this leads to the question: Why there isn't an open-source garbage collector that can be used across different projects?

Surely Python, V8, and Ruby have enough in common, allocation-wise, that the code can be shared? They're all C in the back-end. It's just pointers and more pointers at the lowest level.

It's not about making a toy GC, it's about making a top-rate GC that works and packaging it up as a tool.

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by silentbicycle on Monday September 16, 2013 @12:13PM (#6393961)
Garbage collection is a particularly cross-cutting implementation issue, and tends to interfere with all kinds of abstraction. You're essentially making a system that knows which specific ways abstraction can be broken safely, so that you can defer resource management to it in the future. Retrofitting a GC is even harder. You can bolt on a conservative collector (like boehm-gc), sure, but that can lead to many unexpected performance issues (see e.g. http://timetobleed.com/6-line-eventmachine-bugfix-2x-faster-...).

Writing a virtual machine for a GC'd language and then trying to give it a portable, reasonably convenient C API convinced me that it's a significantly harder issue than most people realize. (Particularly after scrapping my third prototype!) The C API also impacts many things, and tends to bring up nasty edge cases in the GC.

Lua seems to have settled on a design that works unusually well, and is worth studying. Still, Lua can safely cut corners that Python and Ruby cannot - they have threads, which also break a lot of assumptions.

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by __alexs on Monday September 16, 2013 @11:56AM (#6393876)
The Boehm GC [1] has been integrated with various other languages in the past. I'm aware of various attempts to get both Python and Ruby code running with it.

1. http://www.hpl.hp.com/personal/Hans_Boehm/gc/

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by Derander on Monday September 16, 2013 @11:26AM (#6393705)
To a certain extent that's what the JVM is -- you just have to target its bytecode :-).
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by astrodust on Monday September 16, 2013 @11:53AM (#6393859)
That's not exactly a portable back-end toolkit you can just link in.
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by Tuna-Fish on Monday September 16, 2013 @01:19PM (#6394392)
It's not really possible to do that with GC. It requires analyzing and dealing with basically every assumption about object lifetimes that is done within the system, and that typically requires you to read and understand every single line of code.

I have worked on GC projects, and imho bolting on a good GC system on an existing language runtime is considerably harder than reimplementing the entire runtime on top of a GC platform.

Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by ioquatix on Monday September 16, 2013 @07:09AM (#6392623)
Really? Seriously?
Re:Ruby 2.1 Garbage Collection to Address Criticism on Large... by FooBarWidget on Monday September 16, 2013 @07:19AM (#6392663)
I think the guy has a good point. GC is a relatively obscure area, with few people having expertise in it. This, plus the fact that the number of Rubyists who also know C (and know C well enough to work on the GC) is very small, doesn't help the current situation. The Ruby GC is also extremely tied to the VM implementation, making it generally very hard to hack on.

I made the Ruby garbage collector copy-on-write friendly several years ago.