Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
DevOps / Operations / Platform Engineering

The Real Business Value of Platform Engineering

If your focus is solely on the developer experience, you’re missing half the point — and probably half the value — of platform engineering.
Aug 27th, 2023 5:54am by
Featued image for: The Real Business Value of Platform Engineering
Image from Ground Picture from Shutterstock

There’s been a lot of noise made the past few months over platform engineering’s role in displacing (or somehow dispatching?) DevOps.

While platform engineering is unlikely to supplant DevOps as a methodology, it can certainly augment DevOps processes. Because of the many potential benefits of platform engineering, I would expect popularity to continue to grow.

Unfortunately, engineering and technology teams often get a few things wrong about platform engineering. So, what is platform engineering? And what are the tangible values it can offer to businesses?

What Is Platform Engineering?

Platform engineering is the practice of building and maintaining internal toolchains that support software delivery workflows. A dedicated team implements tools and processes that provide developers with self-service access to infrastructure. Platform teams deliver shared infrastructure platforms as products to be consumed by software development and related teams. The methodology aims to reduce complexity for both developers and infrastructure and operations (I&O) teams to accelerate the software development life cycle.

Many platform vendors and other organizations emphasize the developer experience as the value driver for platform engineering. These benefits are derived from self-service access to application infrastructure for developers, typically via an internal developer platform (IDP) like Backstage or something similar.

This can alleviate the hassle associated with service tickets, protracted wait times or the cognitive load of managing their own infrastructure. Simplifying developer access to infrastructure can increase productivity, improve employee retention and speed up application delivery.

But if your focus is solely on the developer experience, you’re missing half the point — and probably half the value — of platform engineering.

What Else Is Platform Engineering?

The real value of platform engineering stems from consistency and visibility as a result of improving control over infrastructure.

When executed correctly, a platform engineering strategy can make it easier to understand how cloud infrastructure is configured and deployed, enforce standards for cloud configurations and operations, optimize productivity, enforce governance and manage costs.

Enforcing Security and Compliance Policies

Assuming control over infrastructure utilization puts platform and I&O teams in a unique position to enforce security and compliance policies.

Because the platform configures and deploys all environments, platform teams can ensure that security protocols and compliance policies are “baked in” to each environment deployed. They can set parameters around what end users have access to, how long those resources can be used, what can be changed and what is locked down. They’re also able to leverage secrets, permissions and keys to infrastructure designs so security mechanisms are automatically embedded into infrastructure deployments.

Using customizable role-based access controls (RBAC), platform teams can ensure that infrastructure aligns to an organization’s team structures and personnel skills/roles to control access to cloud accounts and source repositories.

Optimizing and Reducing Cloud Costs

One of the biggest obstacles to managing cloud costs is understanding the business context behind resource consumption.

Since the platform is the source of all deployments, it can provide end-to-end visibility into environments launched across all phases of the software development life cycle (SDLC).

On its own, cloud billing data lacks transparency. Some platforms, however, can expose how cloud costs are incurred. Integrating application infrastructure into a platform can automate tagging as part of the deployment process. This ties usage back to the specific applications, pipelines, stages and teams that they pertain to.

Tracking real-time configurations with this kind of business context can help engineering and technology teams make informed decisions about cost optimization and resource consumption. For example, they may be able to pinpoint a person or team that often leaves environments running, and incurring costs, over the weekends or holidays when they are not being used. These insights can inform the implementation of cost-management guardrails and consumption policies.

Associating Platform Engineering with Real-World Business Value

Due to a growing desire for businesses to understand and manage their infrastructure, a platform engineering approach makes a lot of sense. At its core, it can give IT and operations teams greater control and visibility over infrastructure configuration and utilization. While the developer experience can certainly be improved using a platform engineering approach, the methodology has wide-ranging benefits that can be realized throughout the business.

Understanding how, when and for what purpose infrastructure instances are used provides the basis for prioritizing cloud spend intelligently and can result in immediate cost reductions. And hardening infrastructure by applying centralized policies and standardized environment blueprints mitigates business risk. All these things taken together greatly improve DevOps processes and overall business outcomes. These benefits will cease to be “nice to haves” and become critical as DevOps continues to evolve.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma, Real.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.