BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//sf26-agenda//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VEVENT
UID:SESS-140@sf26-agenda
DTSTAMP:20261020T151709Z
DTSTART:20261105T183000Z
DTEND:20261105T191500Z
SUMMARY:Everything You Always Wanted to Know About Flink Watermarks* (*But 
 Were Afraid to Ask)
LOCATION:Room 2004
DESCRIPTION:Lorenzo Nicora\, Jan Svoboda\n\nWatermarks are arguably the mos
 t misunderstood concept in Apache Flink. While the internet offers excelle
 nt foundational guides\, live production environments and complex Flink SQ
 L statements find creative ways for pipelines to go wrong. You write a str
 aightforward query\, and suddenly windows refuse to fire\, data drops into
  the void\, or processing halts entirely. You try to write a slightly more
  complex query\, only to find yourself wondering: “Hey! Why can’t I pu
 t a window aggregation after a simple join?” Turning to an LLM for quick
  assistance often proves futile; given the sparse documentation and intric
 ate execution paths\, AI tends to hallucinate its way through these subtle
 ties of stream processing. This session skips the introductory definitions
  and dives right into the engine mechanics of watermark generation and pro
 pagation. We will dissect how watermarks travel through execution graphs a
 nd how some SQL operators manipulate time progression behind your back. Th
 e discussion breaks down the tense relationship between partition idleness
  and watermark alignment\, detailing how these two features cooperate or c
 onflict during traffic fluctuations and skewed data distributions. While w
 e explore these internals\, the explanations are tailored specifically for
  Flink users building applications\, not core framework developers rewriti
 ng the engine. This talk focuses on conveying a deep mental model of how F
 link actually tracks time. We will build the first-principles understandin
 g required to diagnose complex event-time anomalies yourself. These core m
 echanics apply universally to open-source (OSS) Flink. To ground this arch
 itectural theory\, we will conclude by analyzing how Confluent Cloud Flink
  default behaviors differ from OSS defaults\, mapping out the out-of-the-b
 ox differences so your cloud migration does not turn into an unexpected de
 bugging marathon.
URL:https://sf26.rmoff.net/s/everything-you-always-wanted-to-know-about-fli
 nk-watermarks-140
END:VEVENT
END:VCALENDAR