There is a lot of information out there regarding Continuous Integration (CI) and Continuous Delivery (CD). Multiple blog posts attempt to explain in technical terms what these methodologies do and how they can help your organization. Unfortunately, in several cases, both methodologies are usually associated with specific tools or even vendors. A very common conversation in the company cafeteria may be:
- Do you use Continuous Integration in your team?
- Yes, of course, we use X tool
Let me tell you a little secret. Continuous Integration and Delivery are both development approaches. They are not linked to a specific tool or vendor. Even though there are tools and solutions out there that DO help you with both (like Codefresh), in reality, a company could practice CI/CD using just bash scripts and Perl one-liners (not really practical, but certainly possible).
Therefore, rather than falling into the common trap of explaining CI/CD using tools and technical terms, we will explain CI/CD using what matters most: people!
A Story about People — the Dark Ages of Software Integration
Meet Alice, Bob, Charlie, David, and Elizabeth. They all work for SoftwareCo Inc. building the SuperBigProject application. Alice, Bob, and Charlie are developers. David is a test engineer. Elizabeth is the project manager of the team.
The traditional way of developing an application is the following:
Alice, Bob, and Charlie are each working on three different features on their workstation. Each developer writes and tests code in an individual manner. They use long-running feature branches that exist for several weeks or even months before being merged into production.
At some point in time, Elizabeth (the PM) gathers the whole team and makes an announcement: “People, we need to create a release. Please make it happen!”
At this point, Alice, Bob, and Charlie are scrambling to integrate all three features in the same branch. This is a very stressful time because these features were never tested together before. Many bugs and problems appear out of the blue, because of wrong assumptions or environment issues (remember that so far, all features were just tested on each individual workstations, isolated from one another).
Once this highly stressful period is over, the merged result is passed to David who will perform extra manual and automated testing. This period is also time-consuming as he is the one who can approve or block the release depending on how many critical bugs are found. All eyes fall on David while he is doing his part, as his testing can reveal serious issues that would delay the release.
Finally, testing is finished, and Elizabeth happily announces that the release is ready to be packaged and shipped to customers.
So how do people feel in this imaginary (but highly realistic) story?
- Alice, Bob, and Charlie (development) are unhappy because they always learn about integration issues right before a release is about to happen. Integration periods feel like firefights where multiple issues appear at the same time.
- David (testing) is unhappy because his work is truly unbalanced. There are peaceful periods when he just waits for the developers to finish work on features. And then there is the testing phase when he is swamped with work and must deal with unexpected testing scenarios and everybody looking over his shoulder.
- Elizabeth (management) is also unhappy. The integration stage is the critical path of the project. It is a stressful period, as any unexpected issue pushes the delivery of the product further back. Elizabeth keeps dreaming of a software release without any surprises, but this never happens in reality. Estimating the integration phase in the project timeline is always a guessing game.
Everybody on the team is unhappy. (By the way, if your company still develops software like this, please try to understand that this development workflow is damaging to the morale of your team.)
The main issue here is the single “integration” phase that happens at each product release. This is the pain point of the workflow and it prevents the team from having stress-free releases.
Adding the “Continuous” to Integration
Now that we have seen what “Integration” means, it is very easy to understand what “Continuous Integration” entails. As the adage goes, “if something is painful, do it more often.” Continuous Integration is essentially the repetition of the integration step with a high frequency to alleviate its pain. And the most obvious way to do it frequently is to integrate after EVERY feature merge (instead of waiting before an official release is announced).
When a team practices Continuous Integration…
- All features are merged directly into the main branch (mainline).
- Developers are not working in isolation. All features are developed from the mainline.
- A feature is considered done if the mainline is healthy and not if it works on an individual workstation on its own.
- Testing happens automatically both at the feature level and at the mainline level.
And that is the gist of Continuous Integration! Of course, there are more details (actually there is a whole book on this topic) but the main point is that instead of having a single stressful integration period where everything is merged and tested at the same time, “integration” happens all the time in a continuous manner.
Continuous Integration is a better way of developing software (compared to “plain” integration) because it:
- Reduces the number of surprises that appear when features are merged.
- Solves the “works on my machine” problem.
- Slices the testing period into many periods where each feature is merged gradually into the mainline (instead of everything at once).
The result is that a team that works using CI is not living through a roller coaster (calm periods of development followed by stressful releases) but instead gets better visibility into how close to completion the project is in a gradual way.
Working using CI is one of the pillars of modern software development. The technique is very well documented and known at this point in time. There is no excuse for your organization if you are not practicing CI today in your software projects.
The Dark Ages of Software Delivery
Now that we have seen the history of “integration” and how Continuous Integration works, we can take this to the next level with Continuous Delivery.
If we go back to our original story we can see a similar pattern with the way releases are happening:
Performing a release was essentially a “big-bang” event. After the software was deemed to be tested, somebody was tasked with the packaging and deployment process. Deploying software to production was also a very stressful period and traditionally involved many manual steps (and checklists). Deployments were happening very infrequently (there are companies that deploy once every six months to this day). In extreme cases, deployments were happening ONCE (the waterfall design approach).
Delivering software only when the final deadline arrives presents the same challenges as infrequent integrations:
- The production environment is usually found to be different than the testing environment requiring extra configuration at the last minute.
- Features that worked OK in the test environment are found to be broken in production.
- Features that are not ready at the time of the release either are never given to customers at all or they push back the release date even further.
- Releases create tension between developers (who want to ship new features) and operations (who want stability and don’t want to deploy too many new features at once).
You should be able to see the pattern here. If we alleviate the pain of the “integration” phase by doing it more often, we can also do the same thing for the “delivery” phase.
Adding the “Continuous” to Delivery
Continuous Delivery is the practice of packaging and preparing the software (as if it was sent to production) as frequently as possible. And the most extreme way to deliver is after EVERY feature merge.
CD, therefore, takes CI one step further. After each feature is merged to the mainline branch, the application is not only tested for correctness, but it is also packaged and deployed into a testing environment (that ideally matches production). All this happens in a completely automated manner. Notice the lack of stick figures (that represent manual steps) in the previous illustration.
Notice also that each new feature is a potential candidate for pushing to production. Not all candidates are actually sent to production. Depending on the organization, the decision to deploy to production requires human intervention. The human only decides if a release is going to production or not (but doesn’t prepare the release itself). The release is already packaged, tested and deployed in a test environment.
Continuous Delivery is a bit harder to adopt than Continuous Integration. The reason for this is that since every release candidate can potentially reach production, the full lifecycle needs to be automated:
- Builds should be repeatable and deterministic.
- All release steps should be automated (this is harder than it sounds).
- All configuration and associated files should exist in source control (not just source code).
- Each feature/release should be tested in its own test environment (ideally created and destroyed in a dynamic way).
- All test suites should be automated and relatively fast (also harder than it sounds).
While the cloud can certainly help with all these requirements, there is a certain level of discipline required in the software team (both developers and operations) in order to truly embrace Continuous Delivery.
Once CD is in place, releases become trivial as they can be performed with a push of a button. Everybody (not just the project manager) has visibility on the current release candidate. The current release candidate might not have all the requested features, or it might not address all the requirements yet, but this is not important as far as the release process is concerned. The important point is the fact that the release is fully tested and packaged, ready to be sent to production (if needed). Any project stakeholder should be able to give the green light and move the release to production immediately.
If you are using CD, the software lifecycle can be summarized as below:
Each release candidate is always prepared in advance. A human decides if a release candidate will be also pushed to production. Release candidates that don’t reach production are still stored as an artifact if they need to be recalled in the future.
Like Continuous Integration, there is a whole book around Continuous Delivery if you want to know all the details.
Bonus: Continuous Deployment
The “D” in CD can also mean Deployment. This development approach builds upon Continuous Delivery and essentially completely removes all human intervention. Any release candidate that is found to be ready (and passes all quality and testing gates) is immediately pushed to production.
Admittedly, only a very small number of companies can work like this. Pushing straight to production without a human should not be taken lightly and at the time of this writing, many companies are not even practicing Continuous Delivery, let alone Deployment. It should now be clear that each development approach requires the foundations of the previous ones.
Your organization should make sure that each foundation is really solid before moving up. At Codefresh, we have seen a lot of companies that are trying to move into the cloud era by attempting to shoehorn their existing practices (optimized for datacenters) in their CI/CD pipelines without really understanding that some of these practices are now obsolete. Trying to adopt Continuous Deployment without fully embracing Continuous Delivery first is a losing battle.
An alternative way to see what these methodologies cover and how CD requires CI is the following diagram:
Make sure that you approach each development paradigm in the correct order. Targeting Continuous Delivery is a much more realistic goal and one where tooling options are abundant.
Feature image via Pixabay.