Making Deployments Boring
How times have changed. It wasn’t so long ago most organizations developed software using the waterfall method and released software on solid media, such as compact discs. There was no real business need for an agile, fast-paced release cycle and companies responded slowly to changing requirements.
For many software companies, big releases are still the norm. The fanfare around the release of major versions is still a great marketing tool for relaunching or refreshing a brand. And these releases still contain bugs customers must endure until next year’s major new version — if they don’t take their business elsewhere.
However, many software dev teams have adopted the continuous delivery (CD) or progressive delivery (PD) methodology, instantly shipping their code to production. New features and fixes arrive automatically on the user’s device as frequent updates. That bug users complain about today might even be fixed next week and forgotten not long after.
Companies that embrace CD and PD have deployments that are, well, just more boring. When these software makers fix bugs quickly and add new features frequently, customers stay happy, and the releases remain drama-free.
The Importance of Automation
Keeping your software in a releasable state is essential to CD. An automated CD pipeline gives you the confidence to increase your feature delivery cadence and reap the benefits of swift feedback loops and high codebase quality. Problems tend to rear their heads early, and studies link CD to increased productivity.
Without automated CD, deployments are risky. Inconsistent processes and manual steps become a source of pain. Often, two developers on the same team have different methods of deploying the same software. Developers execute many manual deployments based on scripts individuals have written. When they move on to other jobs, the knowledge of the scripts and the ability to maintain them walk out the door with them. Remaining developers continue to deploy the scripts with little or no understanding of them — a recipe for disaster.
In contrast, standardizing and automating production ensures you maximize your product’s uptime and ship what you intend. Rolling back is also best automated, so, rather than patching in the small hours of the morning after something goes wrong, your deployment team can move back to the stable version at the push of a button. This option gives your team confidence to deploy with higher frequency.
After all, the whole idea here is that automation is synonymous with reliability. So the goal is to automate more and minimize deployment’s challenging manual aspects.
Let’s look at a way to break up your monolith and implement some best practices for CD. Turn your software delivery from a risky, stressful event to a boring, everyday occurrence.
Break up the Monolith
One way to make deployments boring is to avoid moving a monolithic application into the cloud. Instead, plan on splitting it up into microservices. This task is worth the time investment and is pretty straightforward, especially since you don’t have to do it all at once.
Approach the task first by putting the whole app in a Docker container. Then, gradually split pieces into separate microservices, each service running in its own container. This fragmentation on its own doesn’t make deployments better, but it helps ensure the app is made up of smaller pieces that are easy to develop, test, control and deploy.
Your software engineers will thank you as they worry less about making changes to the codebase for fear of breaking something elsewhere. Self-contained microservices are a joy on which to work.
There are many other benefits to microservices, too. Your deployments are smaller and more manageable. Conquering this monolith also empowers teams to deploy at a time that is right for them. No more hanging around waiting for the whole monolith to be production-ready. Even this sense of achievement will have an encouraging effect.
In addition to monolithic architectures, some companies develop monolithic deployment scripts. When selecting a deployment tool, it is important to ensure it has features that help avoid this problem. Some deployment tools, such as Spinnaker, allow composing imperative pipelines made up of other imperative pipelines. This encourages reuse and modularity, which can help avoid monolithic processes developing.
Another emerging pattern is declarative deployment definition, in which you declare a set of requirements your code must follow but do not provide a step-by-step process. These lists of requirements tend to be simpler and able to evolve over time while avoiding brittle monoliths.
Deploying Microservices with Kubernetes
Once you chunk your app into microservices in Docker containers, you need a way to manage them. Kubernetes is one of the go-to tools to orchestrate, scale and manage such processes. It monitors your containers’ health and reacts to node failures and other events.
One tip to help make deployment run smoothly is configuring Kubernetes with a YAML or JSON file. Don’t forget to keep these configuration files in source control for restoration if needed. Configuration files also help detect configuration drift, where one team makes tweaks for reasons perhaps only they know and store them on a scrap of paper.
Making the most of the technology available to set standards makes the process run smoothly. Working with many technologies can be daunting, but by bringing it all together with third-party services, you can harness the power of various packages to build CD pipelines seamlessly. You can build guardrails to ensure your team implements best practices, resiliency, security and compliance.
Introducing checks on deployment is also key to preserving consumer confidence. No one wants to damage brand reputation, but any deployment has the potential to push a substandard product. For this reason, organizations have checklists of tasks and final safeguards before pushing to production. Deployments to production are one place where you should consider automating product adherence to corporate policy and compliance.
For example, some companies may have a “division of responsibility” policy: The person who writes the code can’t be the same person who deploys it. You may rely on monitoring, audit trails and alerting to catch policy violations, but these manual steps are prone to weakness. An automated system can implement these checks and halt deployment if there is a problem, and you can easily add this kind of policy enforcement to your pipeline.
When automating CD, you no longer have to wait for a senior dev to get back from vacation to deploy. Company culture changes from disparate teams working in isolation to empowerment across groups, making everyone — from dev to ops and beyond — feel like they’re an essential part of the process. Additionally, your engineers won’t have to worry about being paged after hours because someone found errors just before deployment.
What about When Something Does Go Wrong?
When you’re ready to deploy, advanced deployment strategies can help you limit your exposure should there be a problem with the code you’re pushing out (typically called limiting the blast radius).
Techniques like canary, where you limit deployment to a specific, targeted number of users, analyze results and then proceed on a broader scale, allow you to avoid catastrophic failure.
Another strategy is blue-green (Netflix calls it red-black). With this approach, you maintain two production environments as similar as possible. You perform your final stages of testing on a new release on one of the servers and as soon as it’s stable, you cut over all traffic to it. If you notice any issues, you can automatically push traffic back to the other server (instantly rollback). Using this approach, you can avoid costly outages.
There are no longer fixed release dates that cause stress and frequently changing deliverables with this prolonged deployment. As a developer, it’s frustrating knowing you have repaired a bug, but it takes two weeks to get it out. While bugs and edge cases do still come up in continuous deployment, your team can respond to them quickly.
Continuous deployment offers many benefits, including to your bottom line. Per McKinsey & Co., companies who have created high developer velocity have substantially outpaced those who have not, including:
- Four to five times faster revenue growth
- 60% higher total shareholder returns
- 20% higher operating margins
- 55% higher innovation score
- Higher customer satisfaction, brand perception and talent management
Deliver your product to users with confidence, velocity, stability and across targets. When you do this, you have achieved the competitive advantage of enterprise-scale deployments. And in the end, this keeps your deployments uneventful and even boring, as it should be. Speed of innovation is what we all need.