Serverless Kafka consumer applications and AI agents with AWS Lambda
Anton Aleksandrov, AWSNihar Sheth, AWS Lambda
Thu, Nov 5, 10:30–11:15 AMRoom 2012
Building Kafka consumers today means running, scaling, and patching compute infrastructure yourself, and writing your own retry, poison-pill, and observability plumbing. AWS Lambda's event source mapping for Apache Kafka removes that operational burden, letting you process streams from AWS MSK, self-managed Kafka, and Confluent Cloud with nothing more than function code. In this session, see how to build low-latency, high-throughput consumers that scale automatically with partition lag, decode and validate records against Confluent Schema Registry, and cut cost by filtering JSON, Avro, and Protobuf events before invocation. The same event-driven pattern can trigger AI agents directly from your streams, so events can kick off agentic workflows without extra glue code. We cover tuning for throughput and latency, extensive CloudWatch metrics and logs for consumer visibility, and error handling patterns including retries, bisect-on-error, and on-failure destinations for dead-letter queues.
By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.
Speakers
-
Anton Aleksandrov
Principal Solutions Architect, Agentic AI, AWS
-
Nihar Sheth
Principal Product Manager, AWS Lambda