DevConf.US 2025

Characterizing AI inferencing stacks
2025-09-20 , 106 (Capacity 45)

Choosing the right technology stack is critical as the prevalence of AI-based products grows. Countless options exist for models, runtimes, operating systems, hardware, and more. Identifying which pieces work together best for a particular use case is crucial to ensure optimal performance.

The project lets users specify and simulate their target AI stack. They can then compare different configurations side-by-side–for example, two different models–to determine which variation is more suitable. These results are then visualized and aggregated into an interpretable format, enabling the users to make informed development decisions.

In this talk, you’ll learn about how models can be deployed on the edge, what metrics are relevant, and how AI models differ in size and performance. You’ll also learn how to run models that you can interact with right on your local machine.


What level of experience should the audience have to best understand your session?

Beginner - no experience needed

I'm a Senior at Boston University studying Computer Engineering with a concentration in Technology and Innovation. I'm also in my second semester as Vice President of the Business and Technology Club here. While I’ve had the privilege of taking a wide range of software-based classes at BU, a lot of my learning has been outside of the classroom, teaching myself Python and working on personal projects in my free time like small video games and even a scheduling application for a summer camp. Outside of tech-related topics, some of my hobbies include hockey, cooking/eating, films, and running. I love staying physically active, as I'm actively training for a full marathon in November!