Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
CI/CD / DevOps

The 3 Ds of CD: Defining CD and Why It Matters to the Business

How an organization defines the “D” in CD — delivery, deployment, or development — deeply influences how “continuous” it can be.
Aug 13th, 2021 9:00am by
Featued image for: The 3 Ds of CD: Defining CD and Why It Matters to the Business
Lead image by Illus_man via Shutterstock.

Margaret Francis
Margaret is president and chief operating officer of Armory and a seed investor in minority and women-led businesses. Previously she was senior vice president for product and general manager of Heroku. She holds a bachelor's degree from Yale University and an MFA from SFAI. For fun, she reads a lot of science fiction and plays with her kids.

In a previous article about the ABCs of CD, we talked about the different ways stakeholders — ranging from developers to SREs, operations teams and even business leaders — define continuous delivery. How someone understands the value of the concept typically reflects their individual perspective, echoing the primary concerns and responsibilities that are associated with their role. Thus, developers are invested in being able to deploy frequently; site reliability engineers (SREs) want to maintain service-level agreements (SLAs); operators need to control costs; security teams must maintain compliance; and executives care about innovation, meeting revenue targets and gaining competitive advantage.

CD is a powerful concept that can, in fact, offer all these things to all these people. For CD to realize its full potential, though, it must be part of a cross-organizational cultural imperative. One that incorporates the tools and technologies to automate software delivery, but also embraces collaboration, communication, innovation and agility.

What’s in a C and a D?

The “C” in CD stands for “continuous,” which in practice usually means “as often as possible.” But only the very top-performing development organizations (fewer than 20% of those surveyed in the 2019 DevOps Research & Assessment Report) are able to deploy code to production multiple times per day. The majority release changes to end users between once a week and once a month, which isn’t exactly in accordance with the mainstream definition of continuous.

How an organization defines the “D” in CD — whether it is delivery, deployment or development — has a great deal of bearing on how truly continuous it can be. In fact, what that D stands for within an individual enterprise’s DevOps practice is a direct reflection of what we at Armory call software delivery maturity. An organization’s software delivery maturity indicates how completely it has standardized on best practices for software delivery and how well it’s able to maintain a rapid release cadence while ensuring the quality and reliability of its software. Simply put, the greater an organization’s software delivery maturity, the better it’s able to innovate.

No D At All: Continuous Integration at the Earliest Stages of Software Delivery Maturity

Most organizations begin their journey toward CD by implementing continuous integration (CI). Developers practicing CI merge their code changes into the main branch as often as possible. These changes are validated with automated testing, which ensures the application won’t break when new commits are integrated into it. In essence, CI is the automation of the process of packaging an application for deployment.

CI is a prerequisite for implementing any form of CD. The majority of organizations using centralized code repositories are already doing CD, usually using popular tools like Jenkins, GitLab and CircleCI. These tools weren’t designed to be extensible into the production environment, however. Plus, they weren’t built to support collaboration with teams outside of the development organization. As a result, there can be a wide gulf between being able to maintain an operative CI pipeline and being able to deploy code into production as frequently as you might like.

D Is for Deployment: Advancing Toward a Golden Path to Production

Continuous deployment, the first possible way to define CD, builds upon continuous delivery by automatically deploying all code changes to a specific environment once the build is complete. This type of CD involves taking the artifact that’s created in the CI pipeline and automatically publishing it to a staging or test environment. In some discussions, continuous deployment is also taken to mean pushing the code all the way through to production. For our purposes, it doesn’t matter which environment the changes are deployed to, as long as the process is automated, though in early stages of adoption there may be multiple stops along the way.

Organizations that have adopted continuous deployment are typically able to achieve multiple deployments per month and are often well on their way to breaking down all of their monolithic applications into sets of microservices, embracing immutable infrastructure and adopting Kubernetes.

D Is for Delivery: On the Road to Full Automation

Continuous delivery, our second definition of CD, entails automatically promoting an application between environments just like in continuous deployment, as well as automating some or all decision-making about whether to promote builds. This makes it possible for software to be deployed on an ongoing and truly continuous basis, just like running water. Once delivery has been fully automated, with no manual steps at all, deployments to production take mere minutes, and it becomes possible to deploy at will. In fact, leading organizations at this stage of software development maturity can deploy 100 times per month or more.

To support continuous delivery, organizations will need to have broken down all of their applications into microservices, and have standardized platforms and introduced performance monitoring tools that provide clear visibility across all stages of their pipeline.

D Is for Development: Leading Innovators Eliminate All Manual Steps

Finally, there’s an emerging category called continuous development, sometimes known as progressive delivery or continuous software development (CSD). This CD describes the ability to automate all the infrastructure and application delivery resources into a single unit of work, completely eliminating the need for manual promotions or, at the very least, reducing it to a bare minimum. Organizations that have implemented this version of CD typically have mature DevOps practices where individual engineers take end-to-end responsibility for all applications. These organizations can operate with a customer-first mindset, putting software quality and user experience at the very top of their priority list.

Moving between the stages of software delivery maturity requires more than just a set of automated tools and platforms. In addition, your team must take full advantage of the visibility that a CD platform can give you, harnessing the performance data it provides to automate decision-making throughout your CI/CD pipeline. When every one of these decisions is data-driven, developers can automate with full confidence.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma, Simply.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.