Continuous Performance Testing (CPT) is critical to understanding system behavior beyond functional validation. In this talk, we share how the OpenStack Perf & Scale team built an automated OpenStack CPT framework using Prow CI to continuously test Red Hat OpenStack Services on OpenShift (RHOSO). We automated end-to-end cluster deployment and workload execution across core services such as Nova, Cinder, Neutron, and Barbican. Beyond workload automation, we engineered a metrics pipeline with Elasticsearch for persistent storage and Grafana dashboards for real-time visualization and trend analysis. We implemented change-point detection to proactively identify performance shifts.Along this journey, we developed internal tools like JetBrew, streamlined automation workflows, and reduced multi-day manual efforts into hours. This session highlights our technical architecture, tooling evolution, lessons learned, and how CPT became part of our day-to-day engineering practice.