Wireshark for Kafka: Catch Your Client's Anti-Patterns on the Wire
Stephane Derosiaux, Conduktor
Thu, Nov 5, 9:30–10:15 AMRoom 2006
You call producer.send() or consumer.poll(). On the wire, that one line is never a simple request: a handshake, a metadata fetch, version negotiation, then a Produce or a Fetch, plus the group coordination, heartbeats, and offset commits behind a consumer. All negotiated across protocol versions (Kafka is strict about backward and forward compatibility). Your logs don't show this. Kafka UIs show only the records. How do you see the protocol exchange itself? You don't. That's where the worst bugs hide: a full handshake on every record, an OffsetCommit after every message, a consumer-group rebalance storm, a producer still talking to a stale leader.
To see it, we put a protocol tap between client and brokers, no SDK instrumentation, no code change, and decode the frames live. It works the same whatever client speaks them (Java, Go, Python, librdkafka, Kafka Streams, Flink, etc.), because the protocol is the contract. The demo uses Kapture UI (open-source) to filter the protocol stream, flag the anti-patterns as they appear, and fix your clients.
By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.
Speaker
-
Stephane Derosiaux
CTO, Conduktor
Stéphane Derosiaux is co-founder and CTO of Conduktor. He's spent 15+ years building large-scale streaming systems and now spends his time deep in the Kafka protocol, multi-agent architectures, and what happens when you push Kafka past what it was designed for. He writes a lot about everything Kafka.