Agenda

Wireshark for Kafka: Catch Your Client's Anti-Patterns on the Wire

Stephane Derosiaux, Conduktor

Thu, Nov 5, 9:30⁠–⁠10:15 AMRoom 2006

Data StreamingIntermediateBreakout session

You call producer.send() or consumer.poll(). On the wire, that one line is never a simple request: a handshake, a metadata fetch, version negotiation, then a Produce or a Fetch, plus the group coordination, heartbeats, and offset commits behind a consumer. All negotiated across protocol versions (Kafka is strict about backward and forward compatibility). Your logs don't show this. Kafka UIs show only the records. How do you see the protocol exchange itself? You don't. That's where the worst bugs hide: a full handshake on every record, an OffsetCommit after every message, a consumer-group rebalance storm, a producer still talking to a stale leader.

To see it, we put a protocol tap between client and brokers, no SDK instrumentation, no code change, and decode the frames live. It works the same whatever client speaks them (Java, Go, Python, librdkafka, Kafka Streams, Flink, etc.), because the protocol is the contract. The demo uses Kapture UI (open-source) to filter the protocol stream, flag the anti-patterns as they appear, and fix your clients.

By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.

Speaker

Also on at this time

Add to calendar

Google Calendar Outlook.com Outlook (work or school) Apple Calendar (.ics)