News for hackers. Stuff that matters.
Re:Mongo Vs Redis, The Increment Battle by antirez on Friday January 14, 2011 @10:31AM (#2103814)
I've no clue about MongoDB performances, and which of Redis and MongoDB is faster incrementing integers, but for sure this benchmark is completely useless, as it is actually measuring a mix of client library latency and round trip time, being it single threaded, making no assumptions about the durability requirements, and so forth.

For sure it seems like that MongoDB client library you are using has a smaller latency, perhaps it is a C library? As Redis-rb is instead written in Ruby itself.

Another huge problem of this benchmark is that it should try to increment a random key per every iteration, out of a dataset with a few millions of keys, as this is realistically what you need. You'll see how Redis with 50 clients will increment this counters 100k times per second or more without any performance degradation as the number of keys grows.

Re:Mongo Vs Redis, The Increment Battle by devdazed on Friday January 14, 2011 @10:42AM (#2103851)
I wouldn't say it's completely useless as it shows that Mongo is faster in a single threaded process, using the most popular ruby client of each system.

If I were writing a high-concurrency multi-threaded system to handle millions of keys at a time, Ruby would not be my first choice by far.

Re:Mongo Vs Redis, The Increment Battle by stevelosh on Friday January 14, 2011 @10:52AM (#2103902)
> it shows that Mongo is faster in a single threaded process, using the most popular ruby client of each system.

No, it doesn't. You ran 100k increments on one key and divided by time ONCE and printed the result.

Plus this was on your Macbook Pro, which is probably running a mail client, Twitter client, web browser, and hundreds of other things that could have affected the times.

Graphing the response times for each increment would be a good way to start looking at what the data really shows, instead of going by a single opaque, fairly useless number.

Re:Mongo Vs Redis, The Increment Battle by agib on Friday January 14, 2011 @10:50AM (#2103894)
I have to agree with antirez here and call shenanigans. "WOW, MongoDB is almost a full 40% faster at incrementing than Redis" is simply a false statement. The correct conclusion for your post is "WOW, incrementing the same key 100k times with the mongo ruby gem in a very specific, single-threaded ruby application is 40% faster than incrementing one key 100k times with the redis ruby gem the in that same ruby application." If anyone makes a decision about a real world application based on your post, they will be making a huge mistake.
Re:Mongo Vs Redis, The Increment Battle by antirez on Friday January 14, 2011 @11:04AM (#2103961)
Check this comment: http://news.ycombinator.com/item?id=2103945

it could even be a dummy server just replying +OK and the result will not change.

You should try this against MySQL too, and you'll see similar numbers, for sure not an order of magnitude different than that even if SQL involves much more query processing and so forth.

Re:Mongo Vs Redis, The Increment Battle by devdazed on Friday January 14, 2011 @10:55AM (#2103915)
Ok, I agree with antriz's statement. My results are very specific. I will make note and be sure to be very thorough next time around.
Re:Mongo Vs Redis, The Increment Battle by antirez on Friday January 14, 2011 @01:28PM (#2104751)
Thanks for agreeing, however I must point out that since your blog post does not accept comments, nor you rectified the content, this is misleading information that will stay on the internet for virtually many years to come.
Re:Mongo Vs Redis, The Increment Battle by stevelosh on Friday January 14, 2011 @10:44AM (#2103861)
More accurate title:

(Mongo + Mongo Ruby Gem) Vs (Redis + Redis Ruby Gem), The Increment Battle with No Useful Statistical Analysis Included

Re:Mongo Vs Redis, The Increment Battle by xal on Friday January 14, 2011 @10:41AM (#2103845)
You could make electricity from the amount of eyerolling this article is going to generate here on HN.
Re:Mongo Vs Redis, The Increment Battle by viraptor on Friday January 14, 2011 @11:54AM (#2104206)
I think I heard a distant scream of Zed... http://zedshaw.com/essays/programmer_stats.html He was so right for cases like this...
Re:Mongo Vs Redis, The Increment Battle by true_religion on Friday January 14, 2011 @01:59PM (#2104923)
I'm a programmer, but not very good at statistics (as in I passed the class with flying colors, but I can't really comment on new situations).

When it comes to things like benchmarking, I find it easiest to avoid making absolute conclusions like "X is faster than Y", but to merely present the data, all the data, and nothing but the data.

In this case, I'd have wanted to see the response times per request, and perhaps the memory trend for the client. At the very least, I'd like to know how Mongo and Redis were setup, and if either had begun to swap.

Re:Mongo Vs Redis, The Increment Battle by godi on Friday January 14, 2011 @11:08AM (#2103984)
Running your benchmark I get 11216 increments per second

Running the redis-benchmark from (http://redis.googlecode.com/files/redis-2.0.4.tar.gz) with 50 parallel clients 37800 requests per second

I would like to see the numbers for 50 clients to mongo doing the same.

Re:Mongo Vs Redis, The Increment Battle by antirez on Friday January 14, 2011 @11:18AM (#2104020)
interesting but 37800 is low too, either not Linux or is a Virtual Machine :) That's what I get in my desktop system with Linux (Xeon E5520 2.27 Ghz):

INCR: 126743.98 requests per second

Re:Mongo Vs Redis, The Increment Battle by thinkbohemian on Friday January 14, 2011 @10:55AM (#2103914)
Can you give the mongo and reddis version numbers?

Just curious if you were using reddis 2.0 or possibly 2.2 ? Also what version of the reddis client were you using? There were some performance improvements recently.

Edit: thanks for sharing your experience, even if your test wasn't done with the help of a nobel winning computer team, it can still be useful data.

Re:Mongo Vs Redis, The Increment Battle by antirez on Friday January 14, 2011 @11:01AM (#2103945)
To know the version may be interesting but not really useful, I bet that if you change Redis with a dummy TCP server that is only able to reply +OK, whatever it receives, you'll get the same numbers here.

Basically every kind of networked server with a simple request-response protocol, in a busy loop over loopback, will show performances between 10k and 20k requests per second, mostly up to the speed of the implementation of the client.

Re:Mongo Vs Redis, The Increment Battle by devdazed on Friday January 14, 2011 @11:01AM (#2103944)
I was using redis 2.0 and mongo 1.6.5 with the latest drivers for each.
Re:Mongo Vs Redis, The Increment Battle by jonahharris on Friday January 14, 2011 @01:09PM (#2104645)
In his test, it appears the Mongo driver is just putting the message on the wire, not actually waiting on the server reply or verifying that the command was ever actually processed. Using the Redis in pipelined execution mode would be a much closer test.
Re:Mongo Vs Redis, The Increment Battle by 198d on Friday January 14, 2011 @11:30AM (#2104072)
My gut feeling tells me Redis should be faster at this, and as everyone else has pointed out, the methods used to arrive at these conclusions are most certainly questionable.
Re:Mongo Vs Redis, The Increment Battle by thomasknowles on Friday January 14, 2011 @11:41AM (#2104129)
Redis is asynchronous, try doing it with a few more connections.
Re:Mongo Vs Redis, The Increment Battle by mrinterweb on Friday January 14, 2011 @12:08PM (#2104276)
MongoDB is web scale.
Re:Mongo Vs Redis, The Increment Battle by ghotli on Friday January 14, 2011 @12:41PM (#2104472)
I think that meme is funny too, but lets keep meme only posts over on reddit.
Re:Mongo Vs Redis, The Increment Battle by mrinterweb on Friday January 14, 2011 @02:02PM (#2104931)
Sorry. I'll remember in the future that Hacker News is no place for humor.
Re:Mongo Vs Redis, The Increment Battle by nkurz on Friday January 14, 2011 @02:46PM (#2105112)
Thanks! That's a fine executive summary. The longer version is that original humor is appreciated, particularly if insightful and non-obvious, but regurgitated memes are not welcome here. If you are rushing to make the obvious joke before everyone else does, please don't.
Re:Mongo Vs Redis, The Increment Battle by mrinterweb on Friday January 14, 2011 @08:17PM (#2106136)
Seriously. I did not know that the use of a meme is violates site ethics guidelines. I checked the comment guidelines (http://ycombinator.com/newsguidelines.html) I didn't see any guideline referring to the use of memes. If using a meme is such an egregious offense as this comment thread suggests, maybe this violation should be mentioned in the guidelines.

Generally the popularity of a comment is a good enough indicator of how well a comment is received. If I got 100 upvotes on that comment I may get the impression that people like that. Considering I have < 0 on the comment, I can take the hint. I don't need the proper patrol reprimanding me about a non-stipulated rule.

Re:Mongo Vs Redis, The Increment Battle by soveran on Saturday January 15, 2011 @11:19AM (#2107188)
Can you post the Redis configuration you are using?
Re:Mongo Vs Redis, The Increment Battle by smoody on Friday January 14, 2011 @12:23PM (#2104360)
of course none of it matters if the machine/vm crashes before the increments are committed to disk. :-)