DevConf.US 2026

James Gilmore

James Gilmore received a B.Sc. degree in computer science and a B.A. degree in economics from the University of Calgary, Canada. He is a Staff Software Engineer at Intuit, Inc., where he designs and IT Service Management software solutions targeting change and incident management. Previously, he was a Software Engineer at Amazon Web Services, contributing to virtual machine instance scheduling and capacity management. He holds two U.S. patents on cloud infrastructure resource management (U.S. 10,067,785 and U.S. 10,042,676). His research interests include incident management, change management, highly available systems, developer productivity, and large language models.


Job title:

Staff Software Engineer

Company or affiliation:

Intuit


Session

09-24
16:00
35min
A Hybrid LLM and Rule-Based Risk Assessment Framework for Enterprise Software Change Management
James Gilmore

In many enterprise change management processes, risk assessment is still performed manually, while recurring low-risk change patterns may be pre-authorized or assigned default classifications. This paper presents an automated, quantitative risk assessment framework for software change requests that combines large language model (LLM) evaluations of change artifacts with rule-based operational risk scoring. With this system, risk is divided into two orthogonal dimensions: change risk, which evaluates plaintext change plans, test plans, rollback plans, based on a combination of LLM inference and timing and blast radius heuristics; and asset risk, which uses operational metrics such as code coverage, availability, incident history, and open defects to assess the operational health of the target software asset. These scores are adjusted by configurable multipliers for environment type, asset business criticality, and peak event periods. Deployed within a large financial technology organization and integrated with an ITSM platform, a messaging platform, and operational metrics databases, the framework has evaluated tens of thousands of change requests in production. This paper describes the system architecture, the scoring algorithms, the LLM integration strategy, and discusses design decisions that allow the framework to operate reliably at enterprise scale while remaining interpretable and auditable.

Artificial Intelligence and Data Science
Ladd Room (Capacity 170)