Favorite Social Media Timesink
When you take a break from work, where are you going?
Video clips on TikTok/YouTube
X, Bluesky, Mastodon et al...
Web surfing
I do not get distracted by petty amusements
Software Development

Think Like a Developer to Help Dev Teams Ship Faster

To accelerate the development process, the team building an internal developer platform needs to understand dev processes from the inside.
Dec 6th, 2023 5:57am by
Featued image for: Think Like a Developer to Help Dev Teams Ship Faster
Featured image by Hassan OUAJBIR on Unsplash.  

I started programming professionally back in 2011. It’s safe to say that, back then, most companies felt no need to build in-house dev tools.

Most applications were monoliths, and your programming language of choice usually came with a vibrant ecosystem of frameworks, libraries and tools that gave you live reloading, test automation, debuggers and everything else you needed to write and ship software.

Now your production system consists of 30 services deployed to Kubernetes and uses several managed cloud services. It’s no longer straightforward to write, debug and test these systems with community-standard tooling.

Creating dev environments, live reloading them, running API tests, streaming logs — the automation for all of this must be cobbled together with CI pipelines and tools built by various vendors. This leads to a lot of complexity, frustration and slow feedback cycles. The end result is that developers struggle to be productive when developing modern applications.

Because of this, and to maintain compliance and consistency across the organization, more and more companies recognize the need for an internal developer platform (IDP).

While it’s great if your IDP helps you centrally manage compliance and security concerns in the development process, don’t stop there. Your primary goal should be developer productivity and happiness. That’s how your company can start shipping quality software faster.

In short, focus your IDP on providing development workflows that enable developers to iterate, test and deploy faster and with less friction and frustration. Ideally, it brings the same level of cohesion and quality to developing a system of services that developers already have for developing a single service with language-specific tools (this is hard to achieve but is a worthy goal).

Understanding Prod Doesn’t Equal Understanding Dev

Understanding how the system is deployed and operated in production is not the same as understanding how it’s developed.

The hard part about getting all this right is that the engineers responsible for building an IDP are often site reliability engineers (SREs) and other engineers whose expertise lies in how the system is deployed in production rather than how it’s developed.

The tools, workflows, friction points and other design constraints specific to the dev process (e.g., which parts of the codebase are hard or easy to work on) end up strongly influencing how the system is developed, debugged and tested in practice. And these practical constraints constantly shift and realign in relation to each other as the system grows and changes.

More often than not, this knowledge is embedded in individual dev teams and developers as tribal lore and mental models in individual human brains.

To accelerate the dev process, the platform team needs to understand it from the inside. At its most basic, this boils down to two principles.

1. Know Your Developers

Locating the bottlenecks in your company’s dev workflows should start by working closely with your developers. Understanding the dev and testing process from their perspective — not just from the perspective of production operations and CI/CD systems — is the best way to ensure they’ll benefit from and love using your IDP.

This process should start by mapping out different types of developers’ preferred tools and workflows.

For example, frontend web developers may prefer to do local (not containerized) development against the API of a production-like dev environment. Conversely, backend developers may want to use Docker Compose when working on multiple services but switch to noncontainerized local workflows when working on a single service. This can also vary widely by team and the programming languages or tools used for the various services.

As you talk to more developers from different teams, you will see the main clusters of developer types and tooling preferences emerge. The tools and products you choose to include in your IDP should follow from the workflows and processes you discover while engaging with the developers. The term “developer experience” labels many of these tools as “nice to have,” even though they can have a significant impact on a team’s ability to ship quality software fast.

Every developer dreams of working at the speed of thought. The closer you get to that ideal, the happier they’ll be — and so will management!

This isn’t just about grinding out features and meeting quarterly targets — developers also want programming to feel magical and exhilarating.

2. Identify the Worst Bottlenecks. Fix. Repeat.

The throughput of any process is limited by its bottlenecks. This is true of chemical reactions, factory assembly lines — and shipping software. The best way to improve throughput is to go after the most serious bottlenecks across the entire software delivery life cycle.

If the worst bottlenecks happen during development and testing, self-service ephemeral environments that don’t easily support live reloading or debugging won’t help much when figuring out why an API test is failing on your branch.

Certain areas of a software system tend to be hard to debug because of the difference between the dev and production environments or because of a lack of tooling. When a feature touches on one of these areas, that’s when the slowness — and frustration — set in.

Identifying (and fixing) the trigger points in your architecture and your automation that force developers to drop down from fast iteration to repeatedly trigger slow pipelines or wrestle with unfamiliar operational details to reproduce the behavior they’re writing (or debugging) is a key part of the process of creating a great IDP.

Dig into the metrics: Where are your developers spending their time? How long do certain critical steps in common dev and testing workflows take? How are they trending over time?

Once you’ve got a clear-enough understanding of your company’s dev and testing workflows, you’ll be well placed to track the health of your dev automation with data and dashboards.

Regardless of what tools and methodologies you choose, a laser-like focus on fixing the worst bottlenecks in the development process, reevaluating and repeating will help developers ship faster — and more painlessly.

Putting It All Together

Talking with your developers will provide insight into:

  • What kinds of tools and workflows they like to use when things are going well, such as:
    • Local web dev tools for frontend developers, combined with a shared API.
    • Docker Compose for backend developers.
    • Locally run unit tests, debuggers and other high-productivity dev tools.
  • The trigger points that force developers to use slower, more frustrating workflows, for example:
    • Can’t easily deploy all the services they’re working on at the same time without Docker Compose running out of resources.
    • Can’t easily create a dev environment with live reloading that reproduces the bug that’s just come up in CI.
    • Can’t verify that their fix to a broken integration or API test works without repeatedly rerunning a CI pipeline.

Whenever possible, your IDP should try to augment the tools your developers already love using with the automation and capabilities that allow them to stay in the fast lane.

Creating an IDP that gets all of this right may seem like a daunting task. But working with your developers to find the trigger points in your current setup will narrow your range of options and provide a clear path forward.

At Garden, we have built a tool that’s the perfect foundation for building an IDP when using Kubernetes and other cloud native technologies. Garden bridges the gap between your infrastructure and your dev tooling, orchestrating your Dockerfiles, Helm charts and Terraform configs in accelerated, portable pipelines that run everywhere. Check it out!

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Docker.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.