There is a massive gap between a flashy prototype and an AI service you can trust. While implementing an agentic service to automate RPM packaging, we learned that while AI handles complex tasks, your code must provide reliability. The real challenge was building a "safety harness" to keep a non-deterministic model from breaking deterministic systems.
This talk shares the practical engineering required to make agents production-ready:
- Sandboxing: Why you should never let an agent run free and how to use isolated environments to ensure AI commands can't damage your system.
- Validation loops: Automated checks that treat every AI suggestion as an untrusted draft that must be verified before execution.
- Observability: Move beyond the "black box" by tracking an agent’s "train of thought" so you can debug AI failures like any other software bug.

If you want to build AI tools as stable, observable, and secure as traditional code, this session provides the blueprint we used to get there.

In this interactive meetup, we'll run an un-conference style discussion on the topic of LLMs/GenAI and FOSS. Bring your opinions and thoughts. If you're completely against LLMs, we want to hear from you! If you haven't written any code by hand for the last 6 months and want talk about your tools and techniques, we want to hear from you too!

The goal of this meetup is especially to focus in on those who need/want to LLMs safely and responsibly for "important" software. What are the shared best practices, tools and procedures? What can we do to use these tools to address prior problems around things like supply chain security?

The submitter of this workshop has a lot of of experience and opinions, but is looking to have a realistic in-person discussion where different tools and experiences are presented and attendees can learn from each other.

Proposed topics:

• Scale of change
• Contribution policies
• Sandboxing & Security
• Effectiveness (Ralph loops, Planning vs exec models, skills, ...)
• Orchestrators and workflow (Gastown, Fullsend, Paperclip)
• Discussion of the future