Return on CI/CD Is Larger than the Business Outcome
Armory has been collecting input for a larger, more in-depth study on the return on investment (ROI) of continuous delivery. This is the second post in this series. The first post was “What Do We Harvest by Adopting CI/CD?” The larger paper will be published within the next month.
There was a time when adopting CI/CD gave an organization an advantage. In current times, if a software organization is not yet adopting CI/CD, its leadership and management should realize that they are already behind and at a huge disadvantage.
As a quality professional, I also like to expand CI/CD abbreviation into CI/CT/CD, meaning continuous integration / continuous testing / continuous delivery. This makes testing and meeting quality standards more obvious in this process.
In very simple terms, when you adopt CI/CT/CD, every dev work — new feature, bug fix, improvement — is continuously tested and integrated into your “ready to ship” branch and is, well, ready to be released to your customers based on your criteria for delivery.
Since new dev work is continuously tested for quality and regressions, you have high confidence to release more frequently. I used to work at a company where, when a critical patch was needed, we just triggered our pipeline, which performed extensive validations involving just a handful of people and, after a short time, we were ready to cut a release.
However, for a software organization looking into adopting effective CI/CD, the return on investment (ROI) should not be purely focused on measuring its business outcomes. The DORA metrics can give you a measure of the positive business outcomes from adopting an effective CI/CD process — that is, more frequent releases, faster delivery of changes to customers, fewer bugs and incidents, faster recovery from incidents. On the other hand, and equally important, adopting effective CI/CD has positive outcomes to the development teams as well — that is, it leads to higher innovation, higher throughput, quality and automation mindset, and higher team morale. These people/team outcomes then feed back into your organization in terms of high-quality innovative products and employee satisfaction. Combining these outcomes, you then have a perpetual continuous innovation system benefiting your customers, your business and your people and teams.
To understand the people/team ROI angle, let’s start with the business outcomes:
|Metric||What does this metric mean?||In CI/CD|
|Deployment Frequency||How frequently are we releasing to our users?||Faster|
|Lead Time for Changes||How quickly are developer outputs — feature work, bug fixes, improvements — reaching our users?||Quicker|
|Mean Time to Recovery||How quickly are we able to recover when interruptions due to deployment or system failure happen?||Quicker|
|Change Failure Rate||How good and robust is the quality of the changes we are deploying to production and our users?||Higher & more robust quality|
Adopting CI/CD effectively means end-to-end automation of integration, build, testing and validation, packaging and deployment. This means you have a streamlined automated system and faster turnarounds. When teams have such a system that they can rely on, it naturally leads to the following:
- Greater innovation. CI/CD means more time for people to focus on innovation and try new things. Since the system guards against regressions, teams have more confidence in pushing new capabilities to the customers. On the other hand, if anything fails or if the customer feedback is not great, teams have a system to easily roll back the changes or make quick enhancements.
- Higher throughput. Having an automated process means they can focus on building more and better capabilities, and fixing more defects.
- Quality and automation mindset. At the heart of effective CI/CD are automated functional and non-functional testing and quality checks. To keep the validation pipeline relevant and reliable, teams are motivated to think quality first and create test automations that cover new work. In addition, teams have more time to focus on better design, architecture and implementation.
- Higher morale. With more opportunities for innovation and fewer impediments, team morale is expected to be higher. High team morale leads to better products and employee satisfaction and retention.
Looking at the business and people outcomes together, adopting CI/CD effectively leads to a perpetual and continuous innovation system benefiting your customers, your business, your people and teams.