Which Features Does Your Platform Engineering Portal Need?
Platform engineering has emerged as a sociotechnical solution to fill in many of the gaps that have been missing in DevOps. In particular, platform engineering’s main benefits are how it can improve agility at scale and remove silos — even though this is what DevOps was supposed to do all along.
Improving the developer experience is key, as it’s the main way to improve agility. Practically speaking, this means that developers — with optimized platform engineering in place, providing self-service capabilities with automated infrastructure operations — can do their work without having to assume the more cumbersome tasks associated with managing operations.
This is particularly useful for working in cloud native environments, so that the developer can create applications or complete pull requests without being involved in managing the enormous complexities of Kubernetes and other associated tasks, such as security.
Gartner expects that by 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery. Platform engineering should ultimately solve the central problem of cooperation between software developers and operators, the analyst predicts.
However, a critical element of platform engineering that is often overlooked remains the actual developer experience. This is where a so-called internal developer portal for developers comes into play, as an interface on top of engineering platforms that will ensure that the developer is able to do their work, with a learning curve akin to using a no-code alternative.
With the right portal, gone are the days when the developer must first have to spend over a year learning how to create applications for and deploy applications on Kubernetes. Instead, devs can then begin to apply their talents almost immediately to develop software that achieves direct business goals at cadences that reflect Agile development practices.
Indeed, during the past few years, many organizations were focused on the platform itself in their attempts to adopt platform engineering. They might typically engineer CI/CD pipelines, the cloud infrastructure, implement TerraForm and design a Kubernetes infrastructure.
However, in many cases, attempts to make the platform accessible for developers can get lost in the weeds.
“Developers need to be able to consume the different platform elements with a product-like experience in a way they can understand, since they don’t need to know everything behind the scenes,” said Zohar Einy, co-founder and CEO of Port, which offers an internal developer portal to provide a self-service experience for developers.
“But by putting proper platform engineering into place, you can reduce the cognitive load of the developer and help them get everything they need,” Einy said. “This can be done with an off-the-shelf product that is an internal developer portal.”
Why Open Source May Not Be the Solution
It’s a major engineering feat to successfully design a portal platform that offers what the developer truly needs. While open source offerings exist, they require a significant amount of in-house resources to set up and manage.
The leading open source developer portal is Backstage, according to Eran Stiller, a principal software architect at Luxury Escapes, a travel service provider. Backstage, developed by engineers at Spotify, is highly customizable and scalable, but it requires significant time to adopt by the developer and “you also have to host it yourself, which requires additional time and effort,” Stiller said.
In recent months, Backstage has also revealed some major vulnerabilities, which can introduce security risks.
On the other hand, commercial offerings are often limited in the features and the amount of extensibility they provide, Stiller said.
“If the offering does what you need and works with your current tech stack and platforms, then that’s great. But if it doesn’t, its benefit to you could be limited,” he said. “A commercial platform that provides a high level of customization and extensibility out of the box has an enormous potential to help developer teams out there that don’t want to or can’t invest the time in tweaking and hosting their own Backstage implementation.”
A Checklist for Developer Portals
Features that developer portals should offer, according to Einy:
Self-service that offers devs flexibility but also maintains policy compliance. A portal’s self-service capabilities should extend beyond microservice scaffolding to allow developers to provision and perform Day 2 operations on any asset exposed in the software catalog. At the same time, the portal should ensure that any action adheres to the policies and allow manual approval processes for some operations.
“Port was first to take this ambitious approach with regards to developer self-service actions, going beyond scaffolding, which is also the Backstage approach,” Einy said. “As validation of our vision, we see other players in the market beginning to adopt this approach, too.”
Loose coupling of the portal and the platform. It is possible, in this way, for the platform engineering team to have the freedom to build the underlying platform to their specifications and let developers consume platform data in a way they comprehend via the developer portal.
The platform engineering team has the choice to select processes they want for CI/CD, IaC, software architecture, and everything DevOps.
The decoupling allows making changes to the underlying platform while keeping the developer experience consistent.
“We don’t give you a premade house with everything in it and tell you how to do DevOps or platform engineering,” Einy said. “But we give you the no-code elements to create the house of your dreams. That is the key differentiator between us and other solutions.”
The ability to integrate the organization’s own data model. The data model is brought to the software catalog, which is a visibility layer to the infrastructure and the software developed and managed within the organization.
An ideal software catalog should show the entire ecosystem surrounding the software development life cycle. These include CI/CD flows, developer environments, pipelines, deployments and anything cloud, “across all [research and development] deployments,” Einy said, “from the application development and even data, security, product and research — basically every team that has something to do with software.”
A workflow automation pillar inside the developer portal that allows machines to interact with it. Within the developer portal, machines can access a single API with Port’s real-time software catalog. This provides machines with the information they need, in context, to act as part of their workflows: fail CI jobs, auto-terminate resources or run a CD flow based on software catalog data.
In addition, machines can subscribe to triggers, such as: resource time-to-live reached zero, Cron job timer elapsed, or the creation, modification or deletion of an entity. You can use subscriptions to trigger automated workflows or tasks.
“This is also because the software catalog acts as a general purpose software catalog, and isn’t there just to abstract information for developers,” Einy said.
Such portal features “are essential for modern software development teams,” Stiller said, adding, “This is especially true for those teams who are either large or build software for multiple deployment units such as microservices.”
The Architect Experience
In the post-monolith age, software design requirements have grown more complicated and “development teams grew along with them,” Stiller said. “We started distributing the software we write.”
“This distribution caused a problem where multiple teams kept stepping on each other’s toes, so we moved to service-oriented architecture and later to microservices. The problem is that distributed architectures are much harder to grasp and maintain than monoliths, raising new challenges.”
In microservices-based application development, Stiller said, an architect overseeing the developer process typically is challenged to understand how the system works and asks the following questions:
- Where are the applications deployed?
- Which version is currently running?
- Which service relies on which other service?
- Which service is responsible for most customer errors?
“They need tools to help make sense of the distributed complexity and guide them toward where they should focus,” Stiller said. “A good internal developer portal enables me to do exactly that, by allowing me to build a catalog of all the software that comprises my app, whether it be services, deployment pipelines, infrastructure components, cloud environments, etc.”
A good portal then allows for the architect to overlay additional information on top of their catalog, which is often available in other systems but needs to be organized coherently in one location, he said.
“For example, I could add a layer of information that states the error rate for each service based on data available in my observability platform,” Stiller said. “Or I could add a layer of information that displays the version of a specific cross-cutting dependency based on information in my source control system or CI/CD pipeline.”
Once the information is added, the architect can add rules and create dashboards.
“For example, I can rank the services based on their error rates and alert on those that don’t meet a [service-level agreement], and I can alert on services that haven’t updated a dependency to a minimum required version,” Stiller said. “The ability to do that and show all this information in an accessible manner to all developers in the team is priceless.”
