Re:How Netflix uses Druid for realtime insights
by jpgvm on Monday March 16, 2020 @09:30AM
(#22592471)
Yes this is definitely also true.
Druid complexity is coming down a bit compared to where it started. These days you need brokers, middlemanagers and historicals - for queries, ingestion and storage respectively.
In the past to do batch ingestion it also required Hadoop but there is now a native parallel batch ingestion system that runs on the middlemanagers as worker tasks that can read from S3/GCS/existing Druid segments.
Druid is by far the more complex but you get a lot for it and with k8s it's not as hard to run/manage as it was in the past.
Druid complexity is coming down a bit compared to where it started. These days you need brokers, middlemanagers and historicals - for queries, ingestion and storage respectively.
In the past to do batch ingestion it also required Hadoop but there is now a native parallel batch ingestion system that runs on the middlemanagers as worker tasks that can read from S3/GCS/existing Druid segments.
Druid is by far the more complex but you get a lot for it and with k8s it's not as hard to run/manage as it was in the past.