Deep dive into the performance of Queues for Kafka

When Apache Kafka introduced share groups, the first question from every architect was: "What's the overhead?" This talk answers that question with data. The talk presents a comprehensive performance analysis of share groups across throughput, latency, and resource utilization under diverse workload profiles. We benchmark share groups against consumer groups in head-to-head comparisons — from high-throughput streaming workloads to fan-out patterns, bursty task queues, and heterogeneous worker pools with variable processing times.

Share groups add record-level state tracking, acquisition locks, and per-message acknowledgment — but how much does that actually cost, and when does it pay for itself? We show how performance shifts as you scale consumers beyond partition count, introduce out-of-order acknowledgements, vary message sizes, and mix fast and slow consumers on the same topic. The results reveal where share groups match consumer group performance, where they pull ahead, and the effect of application design and configuration.

You'll leave with a mental model of share group performance characteristics and practical information for designing your applications and tuning your cluster for best performance.