In this talk, we will introduce vLLM and explore why monitoring matters for LLM serving. We'll start from why serving LLMs is hard in the first place and build up to what that complexity looks like as real, observable metrics.

vLLM is a high-performance library designed for LLM inference and serving. Its core optimizations, including PagedAttention, continuous batching, and memory management, directly shape the behavior you'll see in production. Understanding what those techniques are doing under the hood makes it easier to understand what your metrics are actually telling you.

The session will include a demo showing how to instrument vLLM with observability tooling and interpret the signals that matter: KV cache usage, request queue depth, and end-to-end latency. No GPU or production cluster required. Just a small model running locally and some practical monitoring tools.

By the end of this session, you'll have a clearer mental model of LLM serving, a working understanding of vLLM's built-in observability, and the monitoring foundation to understand what's happening inside your server before something goes wrong.