Democratizing Access to Kafka at Scale: Running a Multi-Tenant Streaming Platform Under Strict Regulation

On an average day, around 400 engineers across TD interact with our event streaming platform. Almost none of them sit on the platform team, and all of them work inside one of the most heavily regulated industries in North America. So how do we balance broad self-service against strict regulatory obligation, giving engineers free rein while meeting the auditing and access requirements that come with banking? That is the operational problem we set out to solve.

This talk walks through how we built and operate a multi-tenant Kafka platform that treats governance as an enabler of self-service rather than a brake on it. We will cover the tenancy model that isolates teams across a shared cluster estate, role-based access control mapped to topics, consumer groups, and schemas, and the staged approval workflows that let us automate intake without losing the audit trail. We will go into data masking for sensitive payloads, the audit logging that turns regulatory compliance into a byproduct of normal operation rather than a separate effort, and the data catalog to operationalise metadata and give engineers a way to discover and understand the streams available to them.

Throughout, we will be honest about what worked, what we rebuilt, and the operational debt we are still paying down. You will leave with a concrete picture of how to scale Kafka access beyond a single team, where the governance controls actually need to live, and how to make compliance a property of the platform instead of a tax on every team using it.

This is a real-world account of operating event streaming at scale under regulation, from the people who run it.

By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.