News for hackers. Stuff that matters.
Re:Introducing Tank, a very high performance log by deadmutex on Friday August 12, 2016 @05:55PM (#12279066)
Wish the post talked real numbers, as "very high throughput and very low latency" means very different things to different people. For example, a 1 millisecond latency is "very low latency" for web related things, but might be a lifetime when comparing it to a top of the line trading application.
Re:Introducing Tank, a very high performance log by markpapadakis on Saturday August 13, 2016 @02:06AM (#12280478)
You are right, I should have elaborated on those claims, and I plan to add a relevant Wiki page to address this shortcoming. However, you can use the tank-cli bm command to benchmark client to broker to client (end to end) latency.

We get sub-ms (a few hundred microseconds) latency here, and even on a very busy Tank broker (it takes effort to get there), I don't foresee situations where you can get over 1ms latency for any operation.

For throughput, Tank relies on sendfile() and readahead() for very high throughput - and thanks to the very tight packing of the messages, you will likely saturate your disk/network easily.

Re:Introducing Tank, a very high performance log by markpapadakis on Saturday August 13, 2016 @02:17AM (#12280499)
You may also find this interesting: https://github.com/phaistos-networks/TANK/issues/14
Re:Introducing Tank, a very high performance log by dozzie on Friday August 12, 2016 @04:30PM (#12278576)
It would be much less confusing if you didn't compare a storage mechanism (append-only log) to message transporters (Kafka).
Re:Introducing Tank, a very high performance log by _jezell_ on Friday August 12, 2016 @05:21PM (#12278891)
Would love to see a non JVM kafka alternative. But, I wish people would stop building infrastructure with C/C++ when there are safer alternatives like rust and go.
Re:Introducing Tank, a very high performance log by LaurenceW1 on Friday August 12, 2016 @06:01PM (#12279099)
Really? Safer? How about writing things in languages that are more mature instead of hoping on the newest bandwagon.
Re:Introducing Tank, a very high performance log by _jezell_ on Friday August 12, 2016 @10:51PM (#12280016)
Said the assembly programmer to the C programmer.
Re:Introducing Tank, a very high performance log by dozzie on Friday August 12, 2016 @07:01PM (#12279382)
Given that Kafka is a message router, what don't you like about Fluentd (Ruby) or Heka (Go)?
Re:Introducing Tank, a very high performance log by _jezell_ on Friday August 12, 2016 @10:55PM (#12280026)
Besides the fact that ruby performance is terrible and heka was deprecated and abandoned by Mozilla due to design flaws (https://mail.mozilla.org/pipermail/heka/2016-May/001059.html)?
Re:Introducing Tank, a very high performance log by dozzie on Saturday August 13, 2016 @06:10AM (#12280955)
I'm not to defend Heka, as I don't use it, but those "design flaws" only concerned development, not operations. If you read the message you quote, you would see that Mozilla is still using it.

And I take it you have measured Fluentd's performance, so you say so from experience, not flail around the typical opinion about language that can safely be attached to Java as well?

Re:Introducing Tank, a very high performance log by continuations on Friday August 12, 2016 @05:25PM (#12278918)
How does this compare to ZeroMQ?
Re:Introducing Tank, a very high performance log by markpapadakis on Saturday August 13, 2016 @02:08AM (#12280482)
This is a message broker. You can publish messages to it, and it will retain them for as long as you want(it supports retention and compaction semantics), and you can consume the messages, starting from any message(messages are identified by sequence numbers). This is similar to Apache Kafka and - to a lesser extent - Azure Events Hub )