News for hackers. Stuff that matters.
Re:Once Again, Twitter Drops Ruby for Java by theoj on Monday April 11, 2011 @08:59PM (#2435281)
Once you increase the scale sufficiently, it's more cost effective to hire a team that makes the code run faster rather than add large expensive servers. This is what's happening now at Twitter.

It's not that Java is now absolutely 100% better than Ruby on Rails, it's just that Java is better for a site of that scale.

Re:Once Again, Twitter Drops Ruby for Java by acangiano on Monday April 11, 2011 @09:02PM (#2435288)
Yep. Ruby on Rails: suitable for apps with less than 1 billion queries per day.
Re:Once Again, Twitter Drops Ruby for Java by theoj on Monday April 11, 2011 @09:15PM (#2435332)
It's more than just about queries. Ruby on Rails is an absolute pig in terms of memory used. That's something you find out pretty quickly if you build your own VPS setup.
Re:Once Again, Twitter Drops Ruby for Java by aphexairlines on Monday April 11, 2011 @10:00PM (#2435484)
Going to the JVM because of memory usage issues is like going to KFC to lose weight.
Re:Once Again, Twitter Drops Ruby for Java by bballant on Tuesday April 12, 2011 @06:00AM (#2436554)
On my team such remarks cost a dollar. The JVM generally does very well in terms of memory use compared to many languages. It's easy to test: write an a small program in both languages that creates boatloads of objects and then watch the memory.

The JVM will use up all the memory it can (the amt is configurable with some runtime flags) before the GC kicks in and frees up space. It uses a couple of buckets for different types of objects, and can easily clean small short-lived objects with a bit of a CPU hit, but without affecting the performance of the application. The upshot is you'll see Java's mem use slowly creep up, you'll see a small spike in CPU, then the mem use will drop. If you don't see this, it means the code is poorly written and its keeping around object references.

Ruby's GC is much less efficient. It has no concept of memory buckets like the JVM and will effectively stop execution and traverse every object in the heap twice, marking and then freeing up space.

Early versions of Java (pre 5) had some substantial problems with memory management, but it's been very performant for quite some time and easily out-paces ruby.

Re:Once Again, Twitter Drops Ruby for Java by aphexairlines on Wednesday April 13, 2011 @01:06AM (#2440851)
There are no benchmarks showing JVM-based programs using less memory than most any other roughly equivalent programs on different runtimes. The JVM's reputation as a pig also comes from people's personal experiences -- even a simple email proxy (davmail) is easily at 100mb resident, over 1gb virtual.
Re:Once Again, Twitter Drops Ruby for Java by bballant on Wednesday April 13, 2011 @08:50AM (#2441922)
My main point is that the JVM manages memory better than ruby. Its dead easy to benchmark it yourself by writing a pair of tiny programs. As I said, the JVM will use up all the memory it can. You can probably tweak davmail's runtime flags to use less. There is typically a CPU tradeoff if you cut back on the memory a program can use, and usually a happy middle-ground.

In my personal experience, I've re-written a few ruby utils in Java for an order of magnitude speed improvement. I've seen a ruby site crater under load because of memory issues that its java replacement easily manages (which is sorta what the OP is about). The JVM is especially well suited for handling "stateless" http requests because it's ability to quickly create and destroy small objects.

Re:Once Again, Twitter Drops Ruby for Java by wwkeyboard on Monday April 11, 2011 @10:17PM (#2435539)
It's not just the quantity of queries, it's what the application does. Twitter needs to do specific things very quickly(in this case answer search queries). You could look at a more general high-traffic site(like Facebook) to find a different approach(highly optimized php compiler). I don't think looking to such a large team on such a large project for a role model is beneficial to small teams, they have different considerations and abilities.