2025-02-28 –, Raigad Room | Chanakya Building / School of Business (capacity 200)
Deploying and managing large language models (LLMs) in production often presents significant challenges.By self-hosting LLMs on Kubernetes, organizations gain enhanced data privacy, flexibility in model training, and potential cost savings. This talk aims to enabling beginners by demystifying the process of self-hosting LLMs within the robust Kubernetes ecosystem.We will place a special emphasis on harnessing the capabilities of the Podman Desktop AI Lab extension to accelerate and simplify the development, deployment, and management of LLM workloads on Kubernetes.
Key topics will include:
- Strategically selecting and containerizing suitable open-source LLM models for optimal performance
- Crafting Kubernetes deployment manifests tailored for LLM workloads
- Provisioning and managing Kubernetes resources to meet the computational demands of LLMs
- Deep dive into leveraging the Podman Desktop AI Lab extension for streamlined LLM workflows on Kubernetes
Beginner - no experience needed
Praveen Kumar is a Principal Software Engineer at Red Hat. In his current role, Praveen is focusing on to provide a tool for developers to play with OpenShift locally. In his free time he tries to contribute different container ecosystem projects. He has been a speaker at Eclipse Summit, FUDCon, GNUnify, Flock, FOSSAsia, Jenkins User Conference, Docker-Meet and delivered talks on Docker, OpenShift, Ansible, RPM packaging, Git, Jenkins and various other topics.
Shardul is an AI Engineer at Finfactor, developing AI-driven solutions for fintech. Previously, he worked at Red Hat, driving innovation within the internal ecosystem. His interests include LLMs, Java, DevOps, and Flutter. A passionate educator, he has taught over 500 students across various colleges. Beyond technology, Shardul is an advocate of mindfulness and meditation, emphasizing balance and clarity in both work and life.