Hema Veeradhi
Hema Veeradhi is a Principal Data Scientist working in the Emerging Technologies team part of the office of the CTO at Red Hat. Her work primarily focuses on implementing innovative open AI and machine learning solutions to help solve business and engineering problems. Hema is a staunch supporter of open source, firmly believing in its ability to propel AI advancements to new heights. She has been a speaker at KubeCon NA, Open Source Summit NA, Open Source Summit EU and DevConf CZ.
Red Hat
Principal Data Scientist
Session
Why do AI agents remain fragile and tightly coupled? Most production implementations lack a standardized way to interact with the OS and external tools. This session explores building a modular, cloud-native agent stack using Model Context Protocol (MCP)—an open standard acting as a "USB-C port" for AI interoperability.
In this talk, we’ll share how we integrated MCP into an open AI agent architecture using:
1) vLLM for high-performance model inference
2) LlamaStack as the agentic orchestration framework
3) MCP for standardized tool invocation and data flow
4) Kubernetes for managing distributed agent workloads
We’ll demo the system and share lessons on scaling these components in a cloud-native environment like Kubernetes. You’ll learn how MCP addresses the interoperability gap and helps avoid vendor lock-in. Attendees will walk away with a roadmap for building modular, vendor-neutral AI applications that are firmly in the hands of the community.