BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//sf26-agenda//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VEVENT
UID:SESS-100@sf26-agenda
DTSTAMP:20261020T151709Z
DTSTART:20261105T001500Z
DTEND:20261105T010000Z
SUMMARY:Cluster Mirroring: Rethinking Cross-Cluster Replication from Inside
  the Broker
LOCATION:Room 2006
DESCRIPTION:Luke Chen\, Federico Valeri\n\nCross-cluster replication is ess
 ential for disaster recovery\, geographic distribution\, and data migratio
 n. Today\, MirrorMaker 2 is the standard tool for this job\, but it comes 
 with well-known trade-offs: a separate Connect cluster to deploy and monit
 or\, lossy offset translation that causes duplicate processing during fail
 over\, etc. KIP-1279 takes a different approach. Cluster Mirroring embeds 
 cross-cluster replication directly into the Kafka broker runtime\, elimina
 ting external infrastructure entirely. But getting this right is harder th
 an it looks. The design had to solve several subtle problems: - Unclean le
 ader elections on either cluster can silently diverge replica logs\, yet f
 ailover and failback must remain consistent. - Transactional records carry
  producer state that becomes stale the moment a mirror topic turns writabl
 e\, yet idempotent producers must be able to resume cleanly. - Leader epoc
 hs on source and destination can drift apart\, yet consumers must be able 
 to fetch from their last committed offset without falling into infinite me
 tadata refresh loops. - Source clusters can run versions as old as Kafka 2
 .1\, long before leader epochs or modern fetch features existed\, yet the 
 mirroring protocol must still guarantee consistent replication and clean f
 ailover across this version gap. In this session we walk through the archi
 tecture and design decisions behind Cluster Mirroring\, showing how we sol
 ved each of these technical challenges and what trade-offs we made along t
 he way.
URL:https://sf26.rmoff.net/s/cluster-mirroring-rethinking-cross-cluster-rep
 lication-from-100
END:VEVENT
END:VCALENDAR