BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//sf26-agenda//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VEVENT
UID:SESS-57@sf26-agenda
DTSTAMP:20261020T151709Z
DTSTART:20261105T185000Z
DTEND:20261105T190500Z
SUMMARY:Flink with an AI Agent That Debugs Itself: A Harness Engineering Wa
 lkthrough
LOCATION:Innovation Theater
DESCRIPTION:Purshotam Shah\n\nHarness engineering is the production substra
 te that sits between an LLM agent and your live cluster: structured audit 
 on every tool call\, per-tool quotas\, shadow mode\, dry-run\, human-in-th
 e-loop approval for risky writes\, trace propagation\, retries\, and a clo
 sed feedback loop from human PR review back into the agent's retrieval. It
  is what separates “we demoed the agent once” from “we left it on fo
 r six months and it kept the on-call asleep.” As streaming operations ad
 opt autonomous agents\, the harness — not the model — is the load-bear
 ing engineering. We built and run an AI debugging agent for Apache Flink. 
 It ships as a JobManager sidecar in every Flink application-mode deploymen
 t: loopback to Flink REST\, job-scoped Kubernetes permissions through a na
 mespaced ServiceAccount\, autoscaler state read from PostgreSQL\, and inhe
 rited service identity for outbound calls. Per-deployment blast radius. Pe
 r-deployment audit trail. No central agent service\, no fleet-wide credent
 ials. The diagnostic agent is a LangChain ReAct loop over eighteen Flink-s
 pecific tools: backpressure\, checkpoint health\, Kubernetes events\, auto
 scaler state\, thread dumps\, and log search. It picks two to five tools b
 ased on the symptom. The harness wraps every tool call in eight composable
  LangChain middleware layers: audit\, quota\, shadow mode\, dry-run\, huma
 n-in-the-loop\, model-call limit\, retry\, and trace propagation\, using c
 reate_agent plus middleware composition. Tools are tagged by band — READ
  / WRITE / LEARN / OBSERVE — and risk tier — SAFE / MEDIUM / MANUAL 
 — and the harness enforces tier-appropriate gates. The agent is running 
 in production at Yahoo today. When Grafana fires an alert\, the Flink job 
 debugs itself: the sidecar diagnoses the failure\, the RCA lands in Slack\
 , and the fix is still a PR a human reviews. Merged PRs train RAG positive
 ly; closed PRs flag patterns as “do not propose again.” The reviewer i
 s the training signal — no model retraining required. Attendees will lea
 ve with the eight-layer harness pattern\, the JobManager-sidecar deploymen
 t model\, the band-and-tier risk taxonomy\, the closed-loop RAG design tha
 t turns PR review into agent training signal\, and the failure modes to av
 oid\, including how to protect the feedback loop from PR poisoning.
URL:https://sf26.rmoff.net/s/flink-with-an-ai-agent-that-debugs-itself-a-ha
 rness-57
END:VEVENT
END:VCALENDAR