The Change Failure Rate indicates the ratio of changes made (i.e. deployments in production or staging) resulting in failure, thus leading to a degradation of service or an outage.
As part of the DORA metrics (link to the official page), this metric and the Time to restore service allows for measuring the stability of a development team and the quality of the delivered code.
You can display the change time for the last month, the last three months, or the last 6 months. You can also specify the environment to display, either production or staging.
Note that you must configure one (or more) webhook, incident source, as well as your deployment environments in order to synchronize the data needed to measure this data.
Reading the graph
For the following graph, the time period displayed is the last three months and overall we can see that the team has an average change failure rate of 15% for this period. According to the 2022 State of DevOps report, this rate is in line with the evaluation criteria for a medium-performance team.
The light blue curves visible in the foreground represent the deployments made in this environment, while the more saturated blue curves visible in the foreground represent the failures detected (i.e. bugs created). This indicates a failure rate of 40% for the week of February 14, 2023, as 5 deployments were performed in total and 2 of them led to failures.
A high change failure rate demonstrates low code quality and instability. When correlated with other DORA metrics, it can indicate inefficient processes. For example, a team trying to deploy too quickly, working on too many items at once, or a process requiring manual intervention can lead to a degradation of the delivered quality, resulting in incidents. This results in an investment of time and effort to restore service, limiting the resources allocated to value creation.
Psst! Want to learn more about potential solutions to reduce your change failure rate? Check out our comprehensive Guide to Understanding DORA Metrics!
Calculating the metric
Axify identifies and measures the change failure rate as follows:
- Any bug-type item created following a deployment
- Any item of a specific type created following a deployment, when a matching incident source has been added to Axify (only available to certain users as of March 30, 2023)