SaltStack POP: Code More and Forget About Those Pesky YAML Files

SaltStack’s open source Plugin Oriented Programming (POP), introduced at SaltConf2019 in November, promises to be a potential pain reliever for DevOps headaches.POP, among other things, allows developers to work on code in decoupled code bases or repositories.

Jan 28th, 2020 11:12am by B. Cameron Gain

Featued image for: SaltStack POP: Code More and Forget About Those Pesky YAML Files

One of the major pain points in DevOps is when infrastructure maintenance, configuration and other admin-related obstacles bring development work to a screeching halt. When writing and developing applications for the web, for example, many developers must spend more time than they like configuring YAML, SSH tunnels and other infrastructure-maintenance chores — and less time writing code. This is after spending weeks and often months mastering these admin-like skills before being able to do their real work.

SaltStack’s open source Plugin Oriented Programming (POP), introduced at SaltConf2019 in November, promises to be a potential pain reliever for these headaches.

POP, among other things, allows developers to work on code in decoupled codebases or repositories. Changes to an application are then automated to extend across several data center and often multicloud environments in the form of plugins. The standardized plugin configuration is thus applicable to all of the environments, thus eliminating much of the separate operations work in ensuring the new code works across the entire application ecosystem on which the application is deployed.

“The advantage POP offers is it can make more small codebases that can be maintained more efficiently by small teams. Small engineering teams that can focus on small codebases and therefore accelerate development,” — Thomas Hatch, SaltStack

In a nutshell, you write the code with Python through POP and the resulting codebase can be distributed across an organization’s environments minus much of the developer and operations folks’ work who otherwise have to repackage the code — usually manually — for compatibility.

Developers, for example, might have software deployed across a number of cloud environments and platforms, such as on Kubernetes platforms with Amazon Web Services (AWS) and virtual machines on Microsoft Azure. Integration might include using a number of different tools. But instead of having to use several different ways to manage separate interfaces, POP can serve as a single interface to accomplish multicloud and multiplatform software deployments.

In other words, POP is a development platform for pluggable code, explained Thomas Hatch, the technical founder and Chief Technology Officer for SaltStack. The other projects are expressions of pluggable code built on top of POP. POP allows you to build software that consists entirely of plugins.

“The advantage POP offers is can make more small codebases that can be maintained more efficiently by small teams. Small engineering teams that can focus on small codebases and therefore accelerate development,” Hatch said. “The underpinning philosophy is that we can achieve significant acceleration of innovation and development when we can decouple codebases because that allows individuals and small teams to push code forward more quickly.”

POP can be used to develop plugins monolithic applications, such as those that run on servers in traditional data centers, as well as in multicloud environments. “The approach behind POP is agnostic of whether you store public cloud configuration data, say CloudFormation YAML scripts, OS registry information, or Beethoven’s 4th. It is simply a unified data layer where development environments can tacitly exchange the configuration data they need to remain consistent,” Torsten Volk, an analyst for Enterprise Management Associates (EMA), said. Of course, whether there are any wrinkles for specific use cases remains to be seen, but the overall approach seems to be independent of development language and the type of target environment.”

An IT Framework Alternative

POP is also seen as serving as a viable IT framework for DevOps. As a centrally managed hierarchical layer for development languages, like Python, POP allows developers to access, save and update configuration parameters, EMA’s Volk said.

“When a developer starts a new project, their environment can automatically fetch all needed configuration parameters from a central hub, instead of having to use custom scripting and often overlooking one or two things that then come back to bite you in production,” Volk said. “Among the things that are really cool, POP is agnostic, whether you are developing plugins with Python for traditional data center servers, public clouds, or whatever, without having to also learn and master infrastructure constraints, such as having to learn YAML, etc. POP manages infrastructure as code without adding more ‘vocabulary’ — namely more YAML to learn.”

POP’s capability to run on monolithic applications is thus important, as many organizations’ DevOps make use of a network of traditional monolithic applications running in traditional data center, even as much of their infrastructure is ported to the cloud. “The monolithic applications have never completely gone away. Neither have traditional data centers,” Chris Gardner, an analyst for Forrester Research, said. “For DevOps professionals, being a monolith in a specific location matters a little less than automating the app and its infrastructure in a consistent way.”

A Salt-Centric Project

Hatch began the Salt-based open source project over eight years ago. While continuing to juggle maintaining the project while working on product development and innovations, such as the recently released SaltStack Enterprise 6.1, Hatch said his teams began to use POP to “creates modularity for developers while allowing them to efficiently maintain code while adding innovative and creative new capabilities,” Hatch said. “We’re working to liberate software development and open source innovation traditionally bogged down by tedious maintenance demands,” Hatch said.

SaltStack relies on POP for its open source innovation projects. This is “also very important to the future of Salt,” Nicholas Hughes, an automation consultant and chief executive officer at EITR Technologies, said.

“Salt is a very large open source project, with modules that allow for a wide range of functionality. The problem that you encounter at this scale is that it often causes release cadence to slow down as testing and integrating new contributions becomes time-consuming,” Hughes said. “Implementing a lot of these features as POP plugins in separate code repositories which can be tested and iterated upon at their own pace should allow for a more agile product.”

EITR Technologies has also begun work on an undisclosed internal software project using POP, Hughes said. “We’re creating separate database plugin subsystems for that application which can be merged in depending on which backend database is required for a deployed environment,” Hughes said. “Any new Python project where it would be valuable to break the code into pluggable atomic units should consider using POP. It’s definitely been a great choice for our current project.”

Getting Started

Adding configuration data.

Your entry into using POP to write your own plugins is “easy,” claims POP’s online tutorial. After creating a directory using to Python to write a script called run.py in order to create the requite hub, only a few extra commands come between you and your ability to start creating plugins in POP. However, obviously, it will take major investments in time and training before your organization can begin using POP to deploy and update software at scale across a number of different platforms and environments.

“Starting up with POP is not as simple as quickly writing a script in your command shell. With POP you need to establish a set of reusable patterns that define all the parameters needed to deploy and run your app, including specific functions that define how operations data is gathered, or how data is processed and routed,” Volk said. “As so often, once it is all setup nicely you will reap the benefits but enterprises need to find a way of gradually introducing this technology, which needs planning and consensus between numerous teams. This is the same problem you run into when trying to establish any type of standard.”

However, in any case, one can assume that using POP serves as a better alternative to one person creating their own scripts, which leads to a number of potential problems, not to mention if that script creator is needed while on vacation or leaves the organization.

“The difficulty in writing one’s own scripts is that you create snowflake automation — automation that’s not reusable,” Forrester’s Gardner said. “POP would enable a lingua franca for plugin development.”

SaltStack is a sponsor of The New Stack.

Feature image via Pixabay.

BC Gain’s obsession with computers began when he hacked a Space Invaders console to play all day for 25 cents at the local video arcade in the early 1980s. He then started writing code for very elementary games on the...