Robert Shaw
Robert is a director of engineering at Red Hat. Before joining Red Hat, Robert was senior director of engineering at Neural Magic. He is a core committer to vLLM and a maintainer of lllm-d.
Director
Company or affiliation –Red Hat
Session
09-19
09:15
35min
llm-d: Kubernetes Native Distributed Inferencing
Robert Shaw
llm-d is a well-lit path for anyone to serve LLMs at scale, for any model across a diverse and comprehensive set of hardware accelerators. Come learn more about how llm-d enables distributed inference at scale!
General
Ladd Room (Capacity 96)