Automation: All Fun and Games Until Something Goes Wrong
The story of technology — and the software industry — is one of increasing speed and efficiency. Less friction. A faster route to done. This is more than just a shared myth; it can be seen in the types of products that emerge on the market, how they’re positioned and how they find their way into workflows and toolchains.
For instance: COBOL made it easier for non-specialists to program. DevOps removed the friction between building and operating software. Platform engineering is building guardrails and golden paths so developers can focus on shipping code. Low code is making it possible to build something despite knowing barely a single line of code. And AI … well, you’ll have to wait and see.
While many innovations, techniques and tools do make parts of software engineering better, the notion that we are progressing to an endpoint — a blissful plateau of full automation where none of us ever have to do anything — is, of course, fanciful (to put it politely). And often, in the drive for efficiency, simplicity and speed we end up introducing unexpected complexity and new problems we hadn’t anticipated.
Rather than treat this reality as little more than an occasional anomaly, we should take note of them. They tell us something important about our experience of technology and also highlight how the solutions we design and bring into the world always have consequences we don’t expect.
Shifting Boundaries
One way of thinking about the challenges of automation is to recognize there is always a boundary between different types of work. Yvonne Lam, a software engineer who has worked in operations and developer tooling (most recently at Kong and Chef) told The New Stack that in the world of developer productivity, she sees a constant process of boundary moving.
“Some things fall more to the developer side, and some things more fall to people like me, who work outside specific feature development,” Lam said.
In other words, whenever we automate something or make something more efficient in one location, that’s always because something is happening elsewhere. Someone else is thinking about the problem.
There isn’t necessarily anything particularly unique about this. Problems and processes have always been broken down into constituent parts with a view to making them easier to solve. Where things become tricky, though, is when we need to interact across boundaries.
“It feels like we’re moving boundaries around without necessarily getting better at changing what happens when people meet at a boundary,” Lam said.
A good demonstration of this question of what happens at a boundary? was offered by Maggie Johnson-Pint, co-founder and head of product at Stanza, a reliability tools company.
“When we do analysis of incidents and why they occurred or why we weren’t able to resolve an incident, what happened is that the engineer who was intended to manage that system was working through so many layers of automation that when something went wrong, they didn’t have a clear way to fix it,” she said.
Here, the boundary is such that it becomes a limit — a roadblock that stops you from finding a solution to your problem.
“People don’t want the control, right up until they do.”
— Maggie Johnson-Pitt, Stanza
According to Johnson-Pint, this isn’t an issue caused by bad technology but, ironically, its elegance.
She highlighted the experience of using the frontend stack Vercel, which abstracts backend operations — “it’s freaking magical” she said. But “when you use it, it’s almost as though you’ve been completely separated from the idea that physical computers exist, that you’re just running above a platform that is infinite.”
This means that, as a user, you forget that the very thing on which you’re relying is something that has to be made and maintained.
It’s only when something goes wrong that you realize you lack control over something. The sense of mastery you may have felt over your system gives way to something else: if not a feeling of complete helplessness at least a distinct sense of frustration.
On this, Johnson-Pint was wry: “People don’t want the control, right up until they do.”
Bad Habits
The logic of efficiency can encourage bad habits.
“We are spending way more money than we need to be across this industry,” Pint-Johnson said. She was referring here to cloud — the simplicity modern cloud systems offer makes it all too easy for people to forget that computation is happening somewhere.
“People aren’t paying close attention to how that spend racks up because they don’t have to sit there and think about, am I going to have to spend more on physical compute and have to add additional racks?
“Of all the problems to solve related to climate change, it’s maybe not the biggest one. But actually, we should not be burning this many computers!”
This is an important point. While there are signs that the industry is beginning to appreciate the environmental impact of software, it’s incredibly difficult to see how we can properly address the issue of sustainability while also aiming for less friction, more convenience and greater simplicity.
“There’s sort of this view that automation is always better than people and I think that’s a position we could stand to interrogate more.”
— Yvonne Lam, software developer
Yes, certain automation technologies could help to identify and then minimize inefficiency and waste, yet it remains highly questionable exactly how significant this kind of change will be. Indeed, there’s a danger that this will be little more than a bandage on an issue that requires serious and sustained attention.
“There’s sort of this view that automation is always better than people and I think that’s a position we could stand to interrogate more,” Lam said. “There are plenty of times where yes, your automation is going to do the same wrong thing over and over again, and you can’t stop doing the wrong thing until a person can go and interact with them in some way.”
Lam was talking about errors that disrupt a system. But it’s not hard to see how it also applies to macro-level challenges: how can we begin to solve them when we’re buried in layers of abstractions?
Frictionless for Whom?
Another important consideration when it comes to efficiency is who, exactly, are improvements going to benefit?
Not unlike sustainability, issues like accessibility and neurodiversity have proven to be issues the tech industry has professed to care about. But again, how much this squares with a model of increasing convenience and simplicity is open to question.
With accessibility still treated as a “nice-to-have” rather than a requirement, it’s all too easy for it to be excluded from the process of building guardrails and golden paths for developers, according to Myles Lewando, a software developer and accessibility advocate.
This, he told The New Stack, can have consequences for who is actually able to thrive in the industry. “For neurodiverse developers, some may be lured into the appeal of a copilot or digital pair-programmer,” Lewando said. “But this emphasis on efficiency, shortened deadlines, increased speed, puts pressure on workers, increases stress and anxiety.”
In other words, if we approach the question of effectiveness and productivity without the necessary self-reflection, we could find the industry becomes more difficult — ultimately completely undermining what we surely want to achieve: better work for everyone.
‘We Made One Stupid Thing Easy’
It’s important to recognize the serious implications of these issues in relation to sustainability and accessibility, but often the issue is best demonstrated through the most mundane and stupid examples.
Pint-Johnson provided a good one: for a period, she contributed to Moment.js, a JavaScript library that helped developers manage dates and times. When you called the base API, it would return the date in your computer’s local timezone. If you wanted UTC, or Coordinated Universal Time, you had to explicitly state “moment.utc” in your code.
For a time that wasn’t an issue — in fact, most Moment users didn’t pay much attention to UTC. But this changed, Pint-Johnson said, with the introduction of Node, when JavaScript began to move onto the server: “You actually want to do everything in UTC because who the hell knows where the user is.”
Subsequently, this led to bugs across all sorts of projects and products. “We had to explain over and over again on Stack Overflow!” she said, laughing.
“It’s interesting because you’d think this wouldn’t be that big a deal and people would just learn. But because this library is sticky and it’s in millions and millions and millions of codebases … every developer in the world has to stub their toe on this same thing over and over again.”
Eventually, the Moment team went to TC39 (part of ECMA, the body that looks after JavaScript standards) and proposed the date default to be removed so developers have to select it. Said Pint-Johnson: “We made one stupid thing easy and it made everything hard.”
Getting Started with No Direction
The problem Pint-Johnson relates is a good reminder that when something is easy — or, indeed, simply disappears from view — we seemingly don’t need to learn about it. Lam sees this sort of thing in the way products are marketed: “Whether it’s intentional… a lot of the language around automation, is really a language of abstraction: ‘Nobody needs to know that.’”
As we’ve seen, this has consequences for our ability to solve problems, potentially making us more reliant on specialists. But that’s just one issue: it also changes how we even think about problems — we simply hit the boundary and then wait for the expert to arrive. We are perhaps less likely to countenance the idea that there might be other ways to solve a problem. Or, indeed, that the problem stems from decisions made months or years ago.
Lam believes that this issue is partly caused by the industry’s focus on what she calls the “getting started experience.” She noted that the focus on onboarding developers to products is something the industry has done particularly well over the decade, but claims that there is now “a giant missing middle.”
“It’s really easy to get started with things but there’s nothing that tells you how to maintain your environment while you’re using them or thing that you might want to do later on.” she said. “Once you get past the getting started, it’s like falling off the edge of a cliff.”
She gave the example of Docker — while the documentation is great for telling you how to begin, it doesn’t explore some of the implications and trade-offs of using it.
“It didn’t tell you, ‘Oh, by the way, this thing that makes your environment look seamless — you are downloading gigabytes of images to your hard drive, and at some point, you might want to clean those up!’”
This can have serious technical implications for us down the line — after learning to avoid the hazards of vendor lock-in, we seem to have bumbled into a case of many micro lock-ins.
Automation has consequences for our ability to solve problems, potentially making us more reliant on specialists. But that’s just one issue: it also changes how we even think about problems — we simply hit the boundary and then wait for the expert to arrive.
“Many tools can end up being harder to maintain or may have unforeseen limitations which need to be worked around at a later date,” Lewando said. “Technical debt gets more complex to resolve the more tools there are, and a compromise needs to be found between the time and financial cost of integration, maintenance and developer training, and a new tool’s potential benefits.”
According to Lam, what’s needed is a shift in mindset away from the impulse to do little more than move quickly with less effort. “For me, the question is how do we talk about what people’s needs are,” she said. “How do we bring somebody up in this infrastructure — how do we explain it to somebody?”
The Costs of Convenience
There’s a lot to worry about at the moment. It makes sense, then, that the software industry is interested in things that make life easier and more comfortable for the people that sustain it.
However, this doesn’t mean that we should lose sight of the costs of comfort and convenience — even if we don’t think we need to pay for them now, we might well have to at some point in the future.
