The rise of large language models (LLMs) has opened up exciting possibilities for developers looking to build intelligent applications. However, the process of adapting these models to specific use cases can be difficult, requiring deep expertise and substantial resources. In this talk, we'll introduce you to InstructLab, an open-source project that aims to make LLM tuning accessible to developers and engineers of all skill levels, on consumer-grade hardware.

We'll explore how InstructLab's innovative approach combines collaborative knowledge curation, efficient data generation, and instruction training to enable developers to refine foundation models for specific use cases. In this workshop, you’ll be provided a RHEL VM and learn how to enhance an LLM with new knowledge and capabilities for targeted applications, without needing data science expertise. Join us to explore how LLM tuning can be more accessible and democratized, empowering developers to build on the power of AI in their projects.

LLMs have been very useful and we have high potential for LLMs in Enterprises. However, evaluating these models remains a complex challenge and one of the reasons for LLMs not being adopted directly.
The responsible and ethical AI is going to be the key for Enterprises to adopt the LLMs for their business needs.
Traditional metrics like perplexity or BLEU score often fail to capture the nuanced capabilities of LLMs in real-world applications.
This talk is about current best practices in benchmarking LLMs, limitations of existing approaches and emerging evaluation techniques.

We’ll explore a range of qualitative and quantitative metrics,
from task-specific benchmarks (e.g., code generation, summarization)
to user-centric evaluations (e.g., coherence, creativity, bias detection).
importance of specialized benchmarks that test LLMs on ethical and explainability grounds

Outcome : The audience will be able to understand how to choose LLMs for the right balance of accuracy, efficiency, and fairness. Additionally understand what has improved in granite 3.0 which makes it better LLM.