Measuring CI/CD Adoption Rates Is a Problem
Two years ago we noted that not all developers use CI/CD as “only” 42% of a Digital Ocean survey were using continuous integration/continuous deployment (CI/CD). Despite the passage of time, many companies still do not use CI/CD, although companies with mature adoption of microservices and/or containers are almost always using CI/CD. The extent of CI/CD usage is anybody’s guess for two reasons.
First, there is no agreed-upon definition of what CI/CD is. When mabl surveyed 500 software testers, 53% said their team uses continuous integration, while only 38% have continuous delivery and 29% continuous deployment. Just like with DevOps, marketers are keen to define any tool that addresses an area of CI/CD as part of this market or a larger DevOps tooling segment.
Source: Mabl’s “DevTestOps Landscape Survey 2019”
It is also hard to define the market size for CI/CD since most surveys do not measure the depth of adoption. Just because a CI/CD tool is used within a company does not mean it is widely used, nor that its use cases have gone beyond the most basic. A better metric is what percentage of processes are automated Git commit to code to production. A DevOps focused survey from Codefresh reported that a third of companies had automated more than half of their workloads, but only 1% were all the way there. Another way to think about the issue is in terms of the percentage of developers at a company that use a particular product or service.
The relevance of measuring CI/CD adoption came up in a recent twitter conversation, in which GitLab CEO Sid Sijbrandijg said about half of the Global 2000 companies have use CI/CD best practices like feature flags and tracing, but that only about 1% of workloads are being handled through this way. As part of that conversation, Red Monk’s James Governor wrote, “We’re very early in the journey. so many [organizations] are still doing manual testing and builds. Modern development is happening in small pockets.”
We noted last year that most developers have some sort of automated testing. More recent research by Tricentis shows that while DevOps practitioners are very likely to be counting how many automated tests are being performed, they realize that quantity is different than quality. In fact, when measuring their build processes, DevOps practitioners say that the top metric is how many automated tests are addressing issues that are generating the most risk.
As GitLab, GitHub and others compete head-to-head, will they be operating in separate niche markets, or will they both be vendors of bundled DevOps and CI/CD services?
GitLab and Tricentis are sponsors of The New Stack.
Feature image via Pixabay.
