My company currently uses Druid and has for a few years now, but I have been evaluating ClickHouse on the side as a possible replacement, and as a testament to its simplicity, I was able to stand it up and get in going as a PoC pretty quick/easily.
So far I have only found good things about ClickHouse, maybe the only ClickHouse downside has been the management of the cluster and data, but I haven't gotten too far into operationalizing ClickHouse to know how much those kind of items will cost.
Perhaps the other thing is the documentation, while reasonably good imo, still doesn't explain everything as well as I'd like. It was good enough, but I definitely had to experiment on a couple items to get them working as needed.
What do you plan to do if you need to add new nodes and/or you need to rebalance nodes due to concentrated data access patterns? How do you handle cross-node queries like joins?
Correct. Some ETL inserts to specific nodes when sharding is necessary, in other cases Kafka engine tables on a group of nodes subscribe to common topics, and we simply let the whole cluster participate in queries. This works just fine when table scans are acceptable.
Rebalancing is a missing option here, short of moving partitions manually. But in my specific use cases, I have not yet needed to rebalance across nodes.
Note using native Clickhouse replication is still an option if we need it. One cost to it is the extra work needed in the database cluster, so addressing it in an eariler layer works for us.
> How do you handle cross-node queries like joins?
If I understand your question, since we are using the Distributed view type across the cluster definition, a query on any node will receive data from the others as part of a join-less SELECT, and federate on the node with the client connection.
We are not doing any database-side JOINs currently. Plans are to augment data in ETL, or join data post-query (potentially Spark). Clickhouse dictionaries handle simple cases.
In our case, the simple deployment and management of Clickhouse is a key feature. It is masterless [1] with no namenodes or coordinators, so each machine looks the same, and there is only one process to manage.
If you rely on its replication mechanism for sharding, Zookeeper becomes necessary, but writing directly to nodes in an orderly fashion is also an option (as we are).
[1] This means nodes are not aware of what each other contain, so queries hit all nodes with some maybe having no work to do. Depending on your workload this may or may not be a concern.