TNS
VOXPOP
Where are you using WebAssembly?
Wasm promises to let developers build once and run anywhere. Are you using it yet?
At work, for production apps
0%
At work, but not for production apps
0%
I don’t use WebAssembly but expect to when the technology matures
0%
I have no plans to use WebAssembly
0%
No plans and I get mad whenever I see the buzzword
0%
Cloud Native Ecosystem / Infrastructure as Code

Can Automated Infrastructure Eliminate Job Tickets?

Investing in robust workflow automation for infrastructure can reduce drag and help companies build scalable products.
Dec 19th, 2023 7:43am by
Featued image for: Can Automated Infrastructure Eliminate Job Tickets?
Featured image by Isis França on Unsplash.

There was a time when developers wrote code very close to the infrastructure it would run on. Infrastructure was simple enough for engineers to handle the full stack. This meant they could manage UX, business logic and servers all at once. But enterprise software has changed dramatically: Multiple environments, the rise of the cloud, and massive increases in demand around scalability, monitoring and testing make it much harder for product engineers to create, access and manage infrastructure on an as-needed basis.

Indeed, as a product scales, its infrastructure grows both wide and deep (more services and modules, and more complexity per service and module), which makes managing local environments difficult. A product typically consists of one to two services when it’s at the scrappy seed or Series A funding level. A couple of services can easily run locally or be deployed to test instances in the cloud. However, a more mature product generating over $2 million in revenue will likely consist of more than five services on multiple environments, accessed by many developers who have queued up to use them.

Enter infrastructure workflow automation. With it, product engineers without the time or expertise to spin up cloud infrastructure that closely matches their production environment can use automation to do the work. If a developer requires something at runtime, they no longer need to wait for a support ticket or a DevOps professional to intervene — or worse yet, to set it up themselves. They can bypass that step and automate the process, gaining immediate access to a self-service option.

Infrastructure as Code (IaC) plays a role here, allowing developers to define, provision and manage cloud infrastructure declaratively in code files. This requires much less time and is less error-prone than manually navigating cloud user interfaces for configuration. However, IaC still requires significant effort from the development team.

Designing an application with IaC demands proactive infrastructure planning and code alignment. Despite a proactive design, the strong yet loosely unified relationship between application and infrastructure poses replication challenges. Platform engineering shoulders the responsibility to bridge this gap, but prevailing self-service infrastructure concepts tend to weaken those connections. Integrating cloud primitives into the application layer, rather than managing separate entities, facilitates inherently cloud-aware applications, streamlining the deployment of optimized cloud solutions. Instead, infrastructure workflow automation allows actual code to be used for IaC.

Code block showing extracting a pattern into a reusable library

Extracting a pattern into a reusable library using the Nitric framework.

Build Product and Platform at the Same Time

For startups or smaller companies that lack robust platform teams, every minute your product engineers spend trying to spin up infrastructure is time that could have been spent building revenue-driving features. Strong automation can allow your product engineers to grow the product without having to worry about how to provision the infrastructure they need. The developer can — and should be able to — invoke automated tools that provision infrastructure as defined in the code.

Code used for IaC

Use actual code for IaC.

As a case example, when software-as-a-service (SaaS) products are new, their platform, infrastructure and build system are all a little unstable. Developers in this stage need the flexibility to experiment and decide which tech stack best fits their product needs. Navigating complex cloud processes like spinning up virtual private clouds (VPCs) and Amazon Elastic Container Service (ECS) clusters, manually managing privacy and access control rules, and ensuring proper resource scaling hinder developers’ abilities to innovate.

Code-driven infrastructure automation can manage these from the beginning, allowing product engineers to do what they do best: grow revenue by building features that customers want.

With intelligent code-driven platforms like the open source Nitric framework, product engineers can be quick and scrappy and declaratively set up critical infrastructure simply by referencing it in their code. This has shown to be a much quicker and simpler process than struggling with cumbersome AWS or Microsoft Azure interfaces that few have the expertise to manage correctly. With the platform, Nitric customer Drop Bio Health — an Australian health technology company — reported a 60% reduction in its AWS cloud bill, a 12-fold boost in deployment cadences and other efficiency improvements.

Lack of Automation Is a Risk

Infrastructure architecture

Infrastructure automation also improves the security and risk posture for startups with lean, product-focused teams. Improperly setting up infrastructure can create huge security risks. In 2001, Gartner found that most IT security problems were due to manual misconfiguration of software. More than 20 years later, that threat has not changed.

Many leanly staffed startups are forced to accept the risks of bypassing security best practices until they build out a platform team capable of provisioning and managing their infrastructure securely. This leads to a higher risk profile, evidenced by the number of companies that have suffered privacy breaches. But sacrificing security for faster go-to-market is not always an option.

Companies in highly regulated industries like financial technology, insurance and health care have to devote effort to carefully managing their cloud resources. And for lean startups, this can severely reduce the amount of time their engineers can devote to building the product.

Instead, automated provisioning can bridge this gap by applying security rules to infrastructure until the organization can invest in the platform engineering talent needed to manage a larger and deeper infrastructure securely.

Benefits for Companies Large and Small

Larger companies that already have platform teams still see significant benefits from automation because it frees site reliability engineers (SREs) to work on their own priorities rather than constantly tending to product engineers’ needs. So, for those companies that have platform engineering teams, investing in infrastructure automation can allow their platform teams to work on their own priorities, like containerization, cloud migration and IaC, without spending time provisioning new microservices.

IaC adoption in particular yields substantial benefits. But it has a high adoption inertia, especially for companies with large or distributed architectures. Despite IaC adoption being high on the priority list for platform engineering leaders, organizations often struggle with meeting targets. SRE teams are often too busy standing up and tearing down resources for product engineers to change how provisioning works.

The problem only gets worse as the product scales. Instead of standing up a single compute instance for a small product, a larger architecture involves more complex elements like serverless computing, elastic storage and permissions management. This means platform engineers need to spend ever-increasing amounts of time manually creating, securing and cleaning up infrastructure for product engineering teams at the expense of making progress on their own initiatives. The cumulative effect of these constant requirements has engineering leaders increasingly turning to infrastructure automation.

Infrastructure automation can also have non-technical benefits for tech organizations. For platform engineering leaders, reducing the toil makes it easier to attract and retain top talent. Finding skilled SREs is already difficult, and keeping them becomes even harder if they spend most of their time on repetitive housekeeping tasks. They should be working on challenging initiatives like observability or containerization that move the company and their careers ahead.

Pitching Workflow Automation to Tech Leaders

Tech leaders are often averse to introducing new technology if they don’t feel there’s a crisis. But adopting an infrastructure automation suite is as much about prevention as it is about treatment. It does not have to be disruptive to the roadmap. Young companies have an advantage in that they can build their product with automation included.

But even scaleups and midstage startups can automate provisioning workflows without too much trouble as their product scales and legacy systems come due for replacement or refactoring. Introducing code-based infrastructure automation at this stage as part of a gradual overhaul of the SRE tech stack will grease the wheels for future development. Even restricting automation to newly created services is better than nothing — it gives engineers experience with the toolchain and builds leadership’s confidence in the effectiveness of the approach.

Regardless of when it’s introduced, investing in robust workflow automation for infrastructure can reduce drag for tech organizations of all sizes. It makes life easier for both platform and product engineers. Most of all, it allows the company to devote itself to its core mission: building a scalable product.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.