BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//sf26-agenda//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VEVENT
UID:SESS-127@sf26-agenda
DTSTAMP:20261020T151709Z
DTSTART:20261105T225000Z
DTEND:20261105T230500Z
SUMMARY:Understanding Puffin File Format for Advanced Query Optimization in
  Apache Iceberg
LOCATION:Innovation Theater
DESCRIPTION:Dipankar Mazumdar\n\nApache Iceberg manifests store essential s
 tatistics such as min/max values and null counts\, but as tables scale\, q
 uery engines increasingly require richer metadata to make efficient optimi
 zation decisions. Puffin was introduced in Iceberg to address this gap by 
 providing a file format for storing advanced statistics\, sketches\, and s
 econdary indexes that do not naturally fit within manifests. In this sessi
 on\, we will explore the Puffin file format under the hood\, including its
  structure\, blob-based design\, and snapshot-aware metadata model. We'll 
 examine the two Puffin blob types supported today - Theta Sketches for app
 roximate distinct counts and Deletion Vectors for row-level deletes introd
 uced in Iceberg v3\, and discuss how they enable smarter query planning\, 
 more efficient delete handling\, and metadata-driven optimizations without
  rewriting data files. We will also look at how Puffin is already being ad
 opted in production\, with systems such as AWS Glue and Impala producing a
 nd consuming Puffin metadata to power smarter optimizers and table operati
 ons. Finally\, we will discuss why Puffin is becoming an increasingly impo
 rtant building block in Iceberg's evolution and how it lays the foundation
  for future metadata-driven innovations. Attendees will leave with: An und
 erstanding of the Puffin file format and where it fits within Iceberg's me
 tadata architecture. Knowledge of how Theta Sketches and Deletion Vectors 
 are stored and utilized by query engines. Real-world examples of how Puffi
 n metadata is being used to improve query planning and table maintenance. 
 Insight into why Puffin is becoming a key component of future Iceberg opti
 mization strategies.
URL:https://sf26.rmoff.net/s/understanding-puffin-file-format-for-advanced-
 query-127
END:VEVENT
END:VCALENDAR