Deploying and managing large language models (LLMs) in production often presents significant challenges.By self-hosting LLMs on Kubernetes, organizations gain enhanced data privacy, flexibility in model training, and potential cost savings. This talk aims to enabling beginners by demystifying the process of self-hosting LLMs within the robust Kubernetes ecosystem.We will place a special emphasis on harnessing the capabilities of the Podman Desktop AI Lab extension to accelerate and simplify the development, deployment, and management of LLM workloads on Kubernetes.
Key topics will include:
- Strategically selecting and containerizing suitable open-source LLM models for optimal performance
- Crafting Kubernetes deployment manifests tailored for LLM workloads
- Provisioning and managing Kubernetes resources to meet the computational demands of LLMs
- Deep dive into leveraging the Podman Desktop AI Lab extension for streamlined LLM workflows on Kubernetes

The rise of large language models (LLMs) has opened up exciting possibilities for developers looking to build intelligent applications. However, the process of adapting these models to specific use cases can be difficult, requiring deep expertise and substantial resources. In this talk, we'll introduce you to InstructLab, an open-source project that aims to make LLM tuning accessible to developers and engineers of all skill levels, on consumer-grade hardware.

We'll explore how InstructLab's innovative approach combines collaborative knowledge curation, efficient data generation, and instruction training to enable developers to refine foundation models for specific use cases. In this workshop, you’ll be provided a RHEL VM and learn how to enhance an LLM with new knowledge and capabilities for targeted applications, without needing data science expertise. Join us to explore how LLM tuning can be more accessible and democratized, empowering developers to build on the power of AI in their projects.

Pre-requisites for Workshop.

1. Have a laptop running Fedora, Mac or any linux variant.
   If running windows, you can run a Linux virtual machine inside Hypervisor
   [Please have any one hypervisor installed and linux system installed as Virtual Machine]

2. Download the 3 models locally from GoogleDrive below :
   https://drive.google.com/drive/folders/1NZOk46RU4l4XrYkXI3KkXXk07PHaYtPm?usp=drive_link

We will not have wifi facilities at venue, hence please download these models beforehand.