One of the key takeaways from cdCON2021 — CD Foundation’s annual conference last month— was that there is progress to be made about communicating what continuous delivery (CD) is and how it is perceived in order to better support the developer’s mission. Misunderstanding also exists about how it can help organizations improve their application deployments, regardless of their level of experience.
A number of talks during the conference both served to set the record straight and addressed specific ways organizations can make changes to improve CD. These include the adoption of certain tools and platforms, such as the open source CD delivery platform Spinnaker, and especially, processes and cultural changes such as improving inclusion.
No One ‘Right Way’
The consensus is that software development teams strive to deploy secure, high-quality software at faster speeds and that CD is highly instrumental in achieving that goal. However, that is not the only mission DevOps teams need to have. And although some organizations that are struggling to update or release new code at a pace that they may think is embarrassingly slow compared to the Netflixes and Googles of the world, they have often themselves adopted CD — and in many cases, without even realizing it.
In other words, a true definition of what CD truly is has yet to be established. This is one of the goals of the CDF, where “we can all collaborate on these definitions,” Christie Wilson, a software engineer at Google, said during the keynote “The State of Continuous Delivery presented with Tracy Miranda, executive director of the CD Foundation.
“Here’s what I’ve found so far: continuous delivery is an umbrella term for the practice of releasing software safely and sustainably,” Wilson said. “The goal is to always be ready to release, and to automate your release processes — that’s the goal and that’s the practice. Does that mean that if your codebase is sometimes broken, you’re not doing continuous delivery? No, it means you’ve got improvements to make.”
Google’s Christie Wilson’s talk on the “State of Continuous Delivery”: For every fix that you work on, “feel really good about it, even if it’s small, because even if you don’t change anything, you are already doing continuous delivery.” @GoogleCloudTech @thenewstack #cdCon pic.twitter.com/bkUNXCBrq9
— BC Gain (@bcamerongain) June 24, 2021
Inclusion also plays a large role in both helping to define how CD can work, not to mention, establishing best practices. Just because an organization cannot execute software releases like it wants to, for example, does not mean that the organization has not adopted CD, and is thus excluded from the club. In other words, “you don’t have to be doing continuous deployment to be doing continuous delivery,” Wilson explained.
“Fortunately, we’re starting to realize that keeping people out is not the way to make the best software, what we really want is to be inviting people in and hearing from more perspectives, we need to be inclusive,” Wilson said. “Now this is why I think definitions are so important.”
Inclusion is also not about dictating developers’ CD processes. Empowering teams to adopt the best practices that work the best for them can ultimately reap the best results. “It’s night and day — I’ve seen where teams go ‘well, I’m just going to mandate it to my other teams,’” Isaac Mosquera, CTO of Armory, said during the keynote “Slicing and Dicing: Building DevOps Word Salads.” “Even if you have the power to mandate onto these other teams — and I can measure this with the data that we have — the adoption of your platform, when it’s mandated, is much lower than the teams that are not mandated and feel a sense of ownership on the project or feel a sense of flexibility that they can do what they need to do.”
Beyond a few established best practices, CD thus represents more of a cultural shift and a mindset, than as a carved-in-stone way to improve the quality of software and the rate at which it is deployed. A step-by-step checklist for best practices for CD, in fact, has yet to exist and probably never will become a reality.
“The questions that I get asked every time I walk into a customer site is, ‘please tell me the best practices,’ while there is “really no such thing,” Mosquera said. “Best practices are really like what’s good for your organization, what you need. Because I can tell you what’s good for Brandon at Autodesk (Brandon Leach is a senior engineering manager at the CAD drawing platform provider Autodesk who joined Mosquera in a keynote panel) is not the same as perhaps a bank or a financial institution that has different security needs and requirements.”
A consensus approach among the developer team thus works better than having to follow a set of predetermined CD practices. Summarizing Autodesk’s approach as Leach had described it, for example, the keynote panel moderator Kelsey Hightower, principal developer advocate at Google Cloud, noted how Autodesk created a working document for collaboration. “Brandon laid out this fact that you go and put this in a doc and say ‘hey, here’s the strategy we want. we’ve talked to AWS, they gave us guidance, we’ve worked with the Spinnaker team over at Armory and they gave us the integration work that we need inside of the tool itself.’”
In this context, Hightower asked Leach to describe how platform teams are dealing with “a lot of responsibility in terms of creating these golden paths” and “learning that there will probably be no golden path.” He also noted that “you’re going to have to build something a little bit more extensible, something that actually captures feedback and allows maybe people beyond the platform team to actually help extend and build the platform.”
To better incorporate feedback beyond the developer team scope, Leach described how Autodesk has recently pivoted in that regard.
“We’re now having multiple organizations come together and they have a say in how the platform is built, what gets in the boat with their resources,” he said. “If they want to come and actually contribute something and add something, they can do that.”
Spinnaker can also play a key role in incorporating feedback for and among developer teams for CD. For example, Leach said Spinnaker provides “a really elegant abstraction” between the system’s deployment logic and the underlying reference runtime.
“Spinnaker allows us to provide a unified developer experience across all of these technologies,” Leach said, noting that all of the major cloud providers are also developing for Spinnaker.
@jkriggins discussed productivity challenges during #cdCon—burnout, cognitive load etc.—that today’s developers face. Panel guests discussed how the industry is dealing. #keynotes @CDeliveryFdn with @ab415 Gerard McMahon, and Ramin Akhbari. @thenewstack pic.twitter.com/5PO7kqCcZ1
— BC Gain (@bcamerongain) June 23, 2021
The Human Stack
The conference also highlighted how it is easy to forget that the act of creating and delivering software is a very human process. Just as dictating how CD must be achieved can be counterproductive, holding developer teams to meet specific performance goals and other arbitrary means to gauge performance has been shown to lower developer productivity.
The typical developer, for example, has been shown to only really spend about four hours per day of actually developing software, outside of taking part in meetings or performing other tasks, such as debugging post-production deployments. Dictating that developers somehow pack in eight hours a day of actually developing software might force the developer to routinely work at night or on the weekend, thus leading to burnout.
“There’s a myth called a full stack developer. There’s just so much going on and I guess the opposite of productivity can be burnout and cognitive load,” Jennifer Riggins, a The New Stack correspondent, noted as the moderator of the panel discussion “Developer Experience and Productivity: Level up Your Engineering Effectiveness.”
What is known as “scope creep” can create additional work that has not been predefined, Abby Kearns, CTO for Puppet, noted during the talk. “Scope creep leads into those hero sprints, and then all of a sudden over time really can kind of accelerate burnout because everybody is doing all this extra work every single sprint,” Kearns said. “It’s not really being leveled out across multiple sprints.”