Introducing the Flink State Catalog: A New Era in State Observability

Stateful stream processing is one of Apache Flink’s defining capabilities, yet inspecting and understanding application state remains difficult. Queryable State and the State Processor API addressed parts of this problem, but relied on tightly coupled programmatic APIs, application-specific code, and access to the original job artifacts.

The State Catalog introduces a new approach: representing Flink application state through a clean, read-only Flink SQL abstraction. It automatically discovers checkpoints and savepoints, infers their schemas, and exposes keyed, windowed, and operator state—along with state metadata—as relational tables. Crucially, users can explore and query state without deploying the original application or requiring its JARs on the classpath.

In this session, we will walk through the State Catalog’s architecture, including snapshot discovery, schema inference, state-to-table mappings, and class-independent deserialization. Through practical examples, we will demonstrate how it enables SQL-based inspection and debugging of real-world stateful streaming applications.