Year in Review: Platform Engineering Still Run By Spreadsheet
We entered 2023 with the recognition that platform engineering could ease the complexities that modern developers face, but we ended the year realizing we still get need to get the tooling correct to support the practice.
The report grew from a survey of 100 full-time IT professionals who worked at companies with 150 employees or more, including platform engineers, developer experience professionals, platform product managers and site reliability engineers.
The survey found a widespread support for the emerging practice of platform engineering, but respondents differed on the tooling they felt was required to bring the benefits of platform engineering to developers.
Some need a full-fledged internal developer platform (IDP), either of the open source or commercial variety. Some just need a portal that will provide the dev with access to the all the resources already available to them.
And, in at least 35 cases of those surveyed, developers have to consult spreadsheets to learn how to access a needed service.
Port scolded the practice of using Excel as some sort of primitive service catalog.
“Using spreadsheets should not considered a true internal developer portal given its highly manual nature and the absence of developer self-service capabilities,” the report stated.
The survey found that smaller companies tended to have internally built catalogs, whereas the larger organizations were kicking the tires on commercial IDPs or portals.
In fact, the study suggests the more developers an organization has, the more likely that it needs a developer portal, noted TNS analyst Lawrence Hecht. Depending on the complexity of the environment and microservices used, portals begin to become a worthy value proposition at companies with 50 or more developers, he reckoned.
How IDPs Could Tame Cloud Native Complexities
Portals and IDPs are both physical manifestations of platform engineering.
Just as Kubernetes simplified workload management for distributed systems, IDPs promise to simplify the way the developer can procure needed resources from this cloud native environment.
“We’re trying to make the developer experience a lot more like it was before the complexity of trade-offs we made to get deployment resiliency and a lot of automation in these container orchestration clusters,” he explained.
Wood stipulated that it is strange to be discussing portals again (after a brief industry fad about a decade ago). But they may soon become a necessity, given the complex environment today’s new developers may be entering into. How can they use the tools like Kubernetes without spending endless time configuring it for, first, their environment and then for its final production setting?
For many, setting up an IDP means building off of Spotify’s Backstage.
Spotify developed Backstage, originally internally, to respond to this developer need for a service catalogue of some sort. When the company open sourced this software in 2020, it tagged the package as an open platform for “building developer portals.”
The engineers described it thus: “Backstage unifies all your infrastructure tooling, services and documentation with a single, consistent UI. All of it! Imagine if all your tools — GCP, Bigtable, CI pipelines, TensorFlow Extended and whatever else is hiding in your stack — all had the same, easy-to-use interface.”
Red Hat’s own entry into this space is Red Hat Developer Hub, which was released in May as a beta. When ready, it will be an enterprise-scoped version of Backstage, one pre-populated with suggested application links.
The Hub runs on OpenShift, mostly by way of the Kubernetes API. But Backstage itself is not specifically geared to OpenShift or even Kubernetes. It can also run on containers or bare metal.
Though the end users will be the developers, the management of the IDP falls to the platform engineer, who uses it to ensure all the best practices, routine configurations and policy encoding is baked into the IDP, relieving the dev as much as possible from the burdens of filling out YAML files.
An IDP is a platform that “your developers use to assemble code,” Wood said.
Backstage runs on plug-ins, which can be used to manage applications and services (as well as provide dashboard widgets, CI/CD integrations, document generators, etc). The apps and services can be bundled into catalogs. Templates provide the company-specific setup instructions for the services and applications.
But, Wait, Do I Need a Portal or Platform (or Both)?
Newcomers eager to learn more about the platform engineering space might have gotten tripped up on the nomenclature, with many wondering if the “P” in IDP stood for “platform” or “portal,” a confusion certainly encouraged by opportunistic vendors and insufficiently-savvy trade publications.
The “P” stands for platform, though you will still need a portal of some sort as an interface.
Humanitec, which offers an enterprise-grade IDP with its Platform Orchestrator, defines an IDP as “the platform layer for the enterprise that is built and shipped as a product by a dedicated platform engineering team, in order to remove complexity (without removing context) and enable developer self-service. ”
A portal can be front-facing part of a platform, but it is not the platform itself, clarified Humanitec’s Luca Galante in the post.
But, in many cases, an organization already has a platform of some sort, whether it’s called an IDP or not, wrote Port CEO Zohar Einy on The New Stack in February. It might be a spreadsheet. It might be some awful internally built service catalog, but a lot of platform engineering is already codified somewhere in the organization, and may, or may not, need an actual overhaul.
“You probably don’t need to build a platform; you just need to make it better. The real question you’re facing is whether you should use an internal developer portal on top of your platform, given it is the most mature element in the platform engineering stack,” Einy wrote.
By the Numbers: Platform Engineering is the New DevOps
The larger picture, however, is clear, in that the discussion shows that platform engineering is taking hold, perhaps in a way that DevOps never did.
Back to the Port “2024 State of Internal Developer Portals” survey. It found that 99 of 100 people surveyed indicated their organizations have already started platform engineering.
“Several years ago, we would often see the same high levels in DevOps surveys, but upon further investigation, the maturity of their DevOps was infantile, ” TNS analyst Hecht said.
In this case, 54% of those with platform engineering started implementing the practice in the last year. European respondents were almost twice as likely to have begun in the past year as compared to the United States (70% vs 37%).
Spreadsheet users aside, 50% of respondents’ organizations are using an internal dev portal and another 35% plan to do so in the next year.
“Although this study is small, the finding is almost identical to results from the 2023 StackOverflow Developer Survey’s data in which 47% of microservices users utilize developer portals or other central places to find tools/services,” Hecht noted.
Why set up a dev portal? The most common answers were improving software quality (48%), followed by 46% citing developer productivity. The survey shows that 70% of respondents said they spend three to four hours a day on non-core work due to a lack of platform engineering. Another 8% saying they spend even more time.
How much time they save, however, remains to be seen.
“Almost all studies that quantify potential savings achieved from adopting platforms focus on before and after comparisons where the company does not have a platform,” Hecht wrote. “In real life, there are few true greenfield situations.”
TNS analyst Lawrence Hecht contributed to this post.
Correction: A previous version of this article incorrectly stated the enterprise version of Backstage that Red Hat offers. It is called the Red Hat Developer Hub.