To Survive the Age of Amazon, Stop Managing Projects

What is the biggest organizational difference between a software innovator and a company struggling to maintain relevance as tech giants and startups threaten its market position an viability? Over the past decade, I’ve met with hundreds of IT leaders and executives who are struggling with the answer to this question as their digital transformation initiatives unfold.

Frequent answers to this question include the availability of developer talent, technical debt in the form of legacy systems, and/or the inability to leverage cloud-native and microservice architectures that can facilitate innovation as witnessed with Amazon. While these differences are relevant, in my experience there is one dominant organizational paradigm that determines if an organization is on track to software innovation or whether its struggles will continue. Innovators manage their software offerings as products measured by business outcomes, while laggards manage theirs as projects measured using timeframes and budgets.

Over the last ten years or so, movements such as Agile and DevOps have transformed the team and technical dynamics of how software is built, both in companies that are digital natives and increasingly in larger enterprise IT shops. These practices focus on incremental and continuous delivery of business value to shorten feedback loops. While the practices have matured at a technical level, organizations who manage their technology and IT investments with a project-oriented mindset end up with a structure that is fundamentally misaligned to the flow, feedback and continual learning that forms the core of DevOps and Lean software delivery. Why then is the project-oriented management mindset so prevalent, and why is it such an impediment to effective software delivery?

To better understand this problem, we need to revisit what purpose project management was created for and what it excels at. Some of the world’s most visible and impressive accomplishments were achieved through project management. The discipline was iconified by Henry Gantt in 1917, and used shortly afterwards to construct the Hoover Dam — the largest concrete structure of its time. Project management excels in cases like this when the outcome is well-defined, resources and dependencies can be planned ahead of time, and there are limited degrees of freedom and emergent phenomena that arise during delivery.

However, what the success of Agile methods taught us is that software development processes are better when they can be adaptable to variables like the ever-changing technology landscape, or the need to pivot around product-market fit. The fact that project-management is suited to solving well-defined problems causes some major disconnects. For example, the uncertain nature of a new software product can mean that long projects are padded with such large timeframe and cost buffers that the market and technology landscape can change significantly during the project’s delivery, rendering the assumptions made at the start of the project useless.

By virtue of being closely tied to business inputs, budgeting is one of the first places that we see the breakdown when trying to manage software delivery with a project-oriented approach. By design, project budgets need to bake in all of the uncertainty and risk of a software project. They need to have a well-defined start and end. If something changes fundamentally during the course of the project, a new project is usually created. Because there is a focus on cost and scope, addressing risks or considering redesign opportunities to reduce core issues such as technical debt are resisted. And at project end, there is a switch from a development focus to a maintenance focus, often with outsourcing.

In terms of bringing a successful product to market, and evolving it based on market feedback, all of this spells disaster as it is based on flawed assumptions. Software products do not have a discrete start and end, they have a lifecycle. In a changing marketplace, it is not feasible to bake in all product risk up-front, and as such the project is either going to bake in too little risk or too much padding in terms of budget or timeframe. And given that a software product has no discrete end,  the investment in the product needs to continue as long as the product is generating business results.

It is this disconnect from project management to business results that are at the crux of the problem. When building a dam, or a new building, the business results are all specified up front. But in product delivery, business results need to be tracked incrementally across the lifecycle of the product. For example, when a manufacturer creates a new car, depending on the market demand for that car, they can increase production, add additional options and editions, and respond to market demand. This is because a business value metric is tracked, as is the flow of business value through the product’s value stream.

This is the exact mindset that software innovators adopt with their digital offerings, by releasing software early, often, and tracking the business results generated by the software product. In contrast, the results of a project are generally not known until project end, or even later, at which point the market could have shifted completely. As the project end nears, the pressure for results increases, but the market knowledge that was the input for what was to be delivered may be a year or two old at this point.

Complaints of lack of visibility, lack of results, and lack of fit then start abounding, prompting blame to be targeted at the various project stakeholders. However, the fundamental problem was not with the people doing the work, but with the paradigm used to manage the work. If you want to be a software innovator, it is time to start making the transition from project to product.

This fundamental shift in management paradigm is required to rapidly experiment and innovate on digital offerings. Companies born in the age of software already exemplify how business strategy can be connected to software execution through the discipline of product management.  Those that want to compete in a business landscape increasingly defined by software innovators like Amazon must change their software and IT management practice to be more like Amazon.