Your Apache KafkaⓇ Topics Are APIs - Treat Them Like It
Sandon Jacobs, Confluent
Wed, Nov 4, 1:00–1:45 PMRoom 2006
Would you build a REST API and make users guess the request and response formats?
- What's the domain model?
- Name the operation… POST? GET? PUT? PATCH?
- What does this error code even mean?
- Is the documentation - if it even exists - accurate and relevant?
Yet that's exactly what many data streaming deployments do.
Apache Kafka's flexible, unopinionated serialization is a huge strength, but it's also a potential risk. Even if your event streams carry structured data, enforcing that structure, managing its evolution, and validating its contents aren't hard requirements. This leaves consumer applications preprocessing every event - even ones with no business value.
Designing structured events with a schema (Apache Avro and Google Protobuf) models your business domain and promotes safe evolution practices. Schema-aware serializers enforce event structure before it's written to the stream, while consumers leverage those shared schemas to deserialize on the other end. Build-time validation catches schema problems before code ever reaches review - one last line of defense against poison data. Through all of it, the goal is the same: treat your Kafka topics with the same rigor you'd apply to any API contract - because that's exactly what they are.
Join us to learn how to move validation upstream - transforming your event streams into data consumers can actually trust, with a practical blueprint for schema integrity and automated evolution.
Speaker
-
Sandon Jacobs
Sr Developer Advocate, Confluent
Sandon Jacobs is a Developer Advocate at Confluent, based in Raleigh, NC. Sandon has two decades of experience designing and building applications, primarily on the JVM. His data streaming journey began while building data pipelines for real-time bidding on mobile advertising exchanges—and Apache Kafka was the platform to meet that need. Later work in television media and the energy sector included Kafka Streams, Kafka Connect, and provisioning Kafka infrastructure with various infrastructure as code frameworks.
Outside of work, Sandon is actively involved in his Indigenous tribal community. He serves on the NC American Indian Heritage Commission, and also as a powwow singer and emcee at many celebrations around North America. Follow Sandon on Twitter @SandonJacobs or Instagram @_sandonjacobs, where he posts about his powwow travels, family, golf, and more.