Thinking in Systems: A Sociotechnical Approach to DevOps
The first wave of DevOps focused on automation and collaborative development between Dev and Ops. As it was practiced, DevOps split into two broad camps. One believed in the power of software tools to realize the benefits of faster feedback loops, permissive delivery and resilience. The other camp focused on organizational change at the cultural level. This camp bemoaned the DevOps engineer as an example of dysfunction in the organization, because DevOps should be a global practice of immaculate birth. Each of these camps was doomed to repeat the mistakes of the past by splitting DevOps between technical solutions and cultural practice.
We need a holistic approach to DevOps. An approach that treats tools, the workers who use them and their broader organizations as contributing parts of an interdependent whole. As Andrew Clay Shafer, founder of Ergonautic who’s often considered the de facto inventor of DevOps, says, DevOps is “optimizing the human experience and performance of operating software … with software … and with humans.” Let’s explore that definition and consider how to lead a second wave of DevOps that addresses humans and the tools they use together.
The Problem with DevOps Today
Today’s developer is drowning in complexity. In organizations that have adopted Kubernetes, the microservice stack may be so large and a developer’s concern in that stack so small that understanding it and how it serves to produce value for its end users is nearly impossible.
One way organizations have dealt with this sprawling complexity is to build increasingly ingenious — and complex — platforms to manage it. These platforms are a liability in themselves. We may be building (or buying) ever more efficient tooling, but as Adam Jacobs, of System Initiative, points out: “The way the whole system is put together? The experience of using it? That’s basically identical to how it was in 2009.”
Teams that profess to do DevOps the right way invest in cross-skilling teams until everyone is “full stack.” Managing complexity by training everyone to cope with its complexity reduces specialization, isn’t sustainable and burns out your people.
The once-humble CI/CD pipeline now must be cared for by a professional class of DevOps engineers. What is continuous about a pipeline that requires so much manual attention, forces developers to wait and imperiously hands down blocking errors? Worse still, most CI/CD systems can’t be run locally, so developers wait until the last minute to validate their code against the pipeline’s test suite.
So the tooling sucks and the developers don’t understand the overall business strategy because that information isn’t being effectively communicated to them. Dev and Ops are cooperating, but their goals may be misaligned with the overall strategy because their collective ownership, their piece of the pie, is isolated. What’s the solution?
A Whole-Body Approach to DevOps
A second wave of DevOps is both a tool-based approach and an organizational approach, addressing the problem from the ground up and top down. It involves systems thinking on two levels: the tool level and the organizational level.
At the tool level, we need tools that address a company’s technical problems systemically, not in parts. At the organizational level, we need to understand how the employee is contributing to the organization as a whole and give them agency and the context to make decisions that align with the organization’s goals.
Developers and team leads have the facts on the ground. Leadership has the facts in the air. Aligning individual tasks with the organization’s mission statement ensures everyone is working toward a common goal. We do this by adopting a sociotechnical approach to DevOps.
Sociotechnical Systems, a Brief History
What is a sociotechnical system and how are the software tools we use a component of it? The sociotechnical theorist Jabe Bloom names two pillars of sociotechnical systems in his talk “Whole Work: Sociotechnicity & DevOps”: how people work together and how their skills fit into that work.
I would argue this definition doesn’t cover an important third pillar, which is the use of tools in that system. Any sociotechnical solution must incorporate powerful tools that change the way we produce. We are seeing a movement toward tools that are not piecemeal fixes for very specific problems, but rather holistic solutions that smooth the jagged edges between tools.
Sociotechnical Developer Tools
The late philosopher Bernard Stiegler argued that technology is constitutive of human cognition — that is, our use of tools and technology fundamentally shapes our minds and our understanding of the world. That means that adopting better tools improves the ways we think and work. Specifically, tools that map the dizzying array of inputs and outputs within the organization help us to reason through value chains and where we fit within it. This new breed of tools uses directed acyclic graphs to capture your software infrastructure, microservices, tests and jobs.
Imagine you are a software developer in a big organization that has thousands of developers. Your team owns an API that controls the flow of widgets through time and space. Your API is used by dozens of other teams, but you lack a sense of greater value, of place in the overall system. How does your API result in business value? What are its upstream and downstream dependencies? If your API were to disappear tomorrow, what consequence would it have on the business?
Tools like Garden capture a map of value for software so teams like yours can, at any time, view their part of the whole. The more we understand our part in the broader sociotechnical system, the more empowered we are to make better decisions and captain our ship in alignment with leadership.
Yet, understanding and interacting with this complex web of systems is not just a technical challenge, it is also deeply human. A graph of your software infrastructure and its services is useless if the people, the socio in sociotechnical, are not absorbed in their work.
Empowering Developers with Focus and Agency
For developers, flow is a state of effortless, continuous concentration on a task. A sociotechnical practice of DevOps requires not only sophisticated tools but also an environment conducive to deep work. How does one foster practices and a culture conducive to deep work? By adopting holistic tools that are conducive to fast flow and remove blocking dependencies. Your monster of a GitHub Actions pipeline is a blocking dependency to fast, iterative development.
When developers can run pipelines locally, they have the agency to develop as fast as they can think, fearless and focused.
To achieve a state of flow, developers need the agency to act, to make decisions and to see the impact of their work within the larger system. This agency is critical; without it, achieving flow is like trying to sail into a headwind.
When teams possess this agency, when they “own their own destiny,” their collective efforts multiply. Bloom suggests workers with agency ask questions like, “How can I apply my particular skills and knowledge that will contribute to the greater good?”
The goal is to intertwine these systems of tools and people, to treat them as a single body of practice. A whole-body DevOps takes a step back to view the entire system of human inputs and the software produced by them as outputs, and to address the needs of the system as a whole.
Enabling Teams to Succeed
Before DevOps engineers became custodians of CI/CD pipelines, their work was to de-silo. Be they carriers of cultural practices, technical know-how or strategy, these enablement teams shepherded skills to other teams and individuals with the habits, methods and tools they need to succeed.
The tool, the worker who uses the tool, their team and the organization are all deeply interdependent. Severing any of these from the whole is like a doctor who treats disease in isolation from the whole body of a patient. A sociotechnical DevOps is a holistic practice that strives for visibility, team and individual agency, and adopts software tools that treat our complex map of service dependencies at the level of the system.