DevConf.US 2026

Operating AI Agents in Production: End-to-End Tracing with OpenTelemetry and MLflow
2026-09-24 , 107 (Capacity 20)

When an AI agent goes wrong, it doesn’t fail in one log line. It fails across a chain: model decisions, retrieval, retries, and tool calls that jump between services. The painful part isn’t the bug, it’s the visibility. What happened for this specific user request? Where did the time go? Which step actually broke?

This session is a practical guide to end-to-end tracing for AI agents using OpenTelemetry and MLflow Tracing. You’ll see how to carry one trace context from the user request through the agent, orchestrator, and tool servers, so every model step and tool invocation appears in a single, navigable timeline. We’ll use that trace to pinpoint latency spikes, identify the failing tool or prompt path, and explain agent behavior with agent-native signals: time to first action, step latency, retries, success rate, and token and tool cost.


What level of experience should the audience have to best understand your session?: Beginner - no experience needed

Roberto is a Principal AI Architect working in the AI Business Unit specializing in Container Orchestration Platforms (OpenShift & Kubernetes), AI/ML, DevSecOps, and CI/CD. With over 15 years of experience in system administration, cloud infrastructure, and AI/ML, he holds two MSc degrees in Telco Engineering and AI/ML.