News for hackers. Stuff that matters.

A comment on The Basics of Apache Kafka.

Re:The Basics of Apache Kafka by moedersmooiste on Tuesday September 07, 2021 @10:48AM (#28445543)
We use Kafka a lot within the company I work for. I think it's great. The only thing I miss is fast lookups based on some key and/or the ability for subscribers to only receive messages for certain keys.
[ Parent | on HN ]
Re:The Basics of Apache Kafka by rad_gruchalski on Tuesday September 07, 2021 @01:00PM (#28447098)
That’s most likely the trade off between the queue and a log. The property of the log is having to look at every message.

The prefixes can be somewhat done with a custom partitioner, the lookup requires another technology. The only way to know the final value under the key is to look at the complete log.

[ Parent | on HN ]
Re:The Basics of Apache Kafka by dmead on Tuesday September 07, 2021 @11:48AM (#28446207)
oddly enough, zeromq has the prefix/key style subscription.

you might look at ksql for filtering purposes or just tried to work such things into your topics/partitions?

[ Parent | on HN ]