Re:Why we wrote our Kafka Client in Pony
by dmix on Tuesday January 30, 2018 @08:52AM
(#16265235)
That was for what I'd imagine is v0.1 quality code, they said there were plenty of optimizations available to make it at least parity with the C client. Secondly, the issue was the system performance between Wallaro and the Kafka client, not necessarily the raw performance of the library.
[ Parent
| on HN ]
Re:Why we wrote our Kafka Client in Pony
by hackmanytrades on Tuesday January 30, 2018 @09:09AM
(#16265359)
Yes, that's correct on both counts.
Pony Kafka is almost completely untuned as of right now. We expect there is a lot of low hanging fruit on that front that will give us significant gains.
And yes, we're concerned about the potential thread pool contention between Pony and librdkafka.
Re:Why we wrote our Kafka Client in Pony
by manigandham on Tuesday January 30, 2018 @02:49PM
(#16268359)
Copying data in memory between libraries is slower than actually getting the data across the network from the Kafka cluster itself? Seems like a strange bottleneck if so.