Re:Reaching 200K events/sec
by spullara on Thursday January 31, 2013 @09:44PM
(#5148885)
You are likely doing something wrong. My in-memory redis server clone that also uses netty reaches 1.5M requests/s on an arguably heavier workload with a more complicated protocol. I'd love to help you optimize it if it matters.
[ Parent
| on HN ]
Thanks for the offer! I, uh, certainly didn't mean to claim that Riemann is extraordinarily fast. Up until this point I've been working as hard as I can just to make the darn thing turn on and accomplish something useful. This is quite early in the optimization process, haha. :)
The principle bottleneck in this particular test is actually the client--Riemann itself spends ~94% of its time waiting on epoll in this test. The client is a total hack using Java OIO, calling flush() on every message, Nagle's algorithm disabled for low per-msg latencies... it's a wreck, haha. Part of next week's optimization push is replacing it with a Netty client and tuning TCP options for various workloads.
Things will probably look better once you switch your client implementation to Netty as a similar test I did with Netty 4, Protobufs and batching of messages produced low seven figure numbers too.
Ah ok, my test was run using the C redis-benchmark client.
Re:Reaching 200K events/sec
by thebluesky on Friday February 01, 2013 @06:08AM
(#5150191)
Is the source available anywhere?
1.5M locally or over a network? What is the size of each message?
http://github.com/spullara/redis-protocol — server module
The test was run with "SET key value" which in the redis protocol is something like 15-20 bytes / message.