Nope, no Kafka at Fivetran, nearly all the sources we pull from are intrinsically batch, support retry, and come out of the source already partitioned by user. Our internal architecture is basically just a farm of batch processors.

We have been hearing more and more about people using Kafka to support streaming analytics. I haven't spent a ton of time studying it, I was under the impression that it was just a giant queue, but ordering + retention is basically a database in the way I think about it.

If consumers need to keep track of their own offsets, do you run into complications when trying to run multiple instances of the same consumer? Or do you typically run single processes when consuming a specific topic?

This is what partitions are for. They are used to scale out both publishing and consumption by assigning each consumer a single partition. This logic used to be complicated client-side code but now is included into Kafka itself, along with automatic offset saving on some interval (saved as entries in another topic).

Also consumers can be in a "consumer group" so you can have multiple clusters of consumers each reading the entire log separately, but shared within each cluster.

Also I'd recommend looking at Apache Pulsar for a next-generation architecture that combines Kafka's log semantics with the low-latency routing and individual message acknowledgements of message queues: https://pulsar.incubator.apache.org/