Puppet Launches Lumogon, a Container Inspection Tool
Puppet has unveiled an open source container inspection tool it calls Lumogon — a tool that the company argues has become necessary, given that developers in everyday practice build their container images using a layered approach that has become less and less predictable.
“Suddenly there’s a tool that anyone can use, and that anyone can build things on top of,” said Puppet senior software engineer Gareth Rushgrove, “that can introspect all of the information in a container, and the contents of that container.”
The case was made early on that nothing makes a Docker container more transparent and open about its contents as its Dockerfile, the short list of commands executed by the Docker engine to build a container image. Read the container’s Dockerfile yourself, and you should know what it will contain.
Docker Inc.’s official documentation advises, as part of best practices, that developers build as simple a Dockerfile as possible, not only to ensure that the container is built efficiently but to let it live only temporarily. “By ‘ephemeral,’” the documentation reads, “we mean that it can be stopped and destroyed and a new one built and put in place with an absolute minimum of set-up and configuration.”
It’s part of the elegance of containers, at least in principle. In a personal blog post just a few months ago, Red Hat senior automation engineer Chris Collins demonstrated the construction of a continuous integration methodology around Dockerfiles, saying, “The simplicity and hands-off approach of this process is effectively Made for Automation.”
But all this assumes that the commands that an organization uses within its Dockerfiles are straightforward, and their outcomes pre-ordained if not largely predictable.
This may not be the case in general practice, Rushgrove argued.
Uh-Oh, It’s AppOps
Lumogon will be released in tandem with — at first, as a limited tech preview — of its Puppet Cloud Discovery online service. Rushgrove described this as a kind of perpetual data warehouse for the metadata produced by container images. He suggested that Lumogon’s reports, coupled with the information about particular packages included with container images stored within Cloud Discovery, could make it feasible for management tools going forward — not necessarily more tools from Puppet — to apply policy to the production of container images. Imagine a security system that prohibits the building of an image with a known-vulnerable package, and you get the idea.
Rushgrove describes Lumogon as feeding an urgent desire by a class of IT personnel he calls “AppOps.” It’s a term we’ve used only a few times before, referring to people who have been given operational responsibility for applications. They’re not developers, so they’re not “DevOps.” But they own the platforms upon which these applications run. And in organizations adopting cloud-native methodologies, he contends, it’s these personnel who marshal the lifecycle of the application through the production phase — more often AppOps instead of developers.
“In organizations, in lots of cases, someone — invariably in the IT operations group — ends up [declaring], ‘We will own that platform,’” explained Rushgrove. “And the responsibility is getting pushed down to development teams, for them to run their applications on top. That doesn’t really remove operations; it removes a lot of the traditional business administration effort.”
AppOps personnel, as Rushgrove perceives them, are delegated operational responsibility for individual platforms — for example, the Google Container Engine guy, or the Amazon EC2 guy — and all the applications running on those platforms. More than one platform? More than one person. These individuals, as he described them, inherit the dependencies upon which container images rely, from developers who are not so subdivided among the platforms they support.
Puppet’s Rushgrove continued: “We have lots of good developer tools, and lots of people are building good tools for building these types of applications. But there are fewer tools focused on running them day-to-day.”
As a result of this decoupling of job functions, he argued, the processes of building container images for production, become disconnected from the original, transparent processes used to build them for testing. An AppOps team may be delegated responsibility (or it’s simply assumed by default), and that team immediately begins making alterations and adjustments. Those adjustments are (let’s not misinterpret what he’s saying) vital to their proper running order.
“But operations in that context is different,” Rushgrove went on, “to traditional business administration.”
The Impracticality of Immutability
All this makes Lumogon a container management tool for someone who is technical and who does have operational authority, but who is not a developer. But there’s a deeper implication as well: This cultural and/or political division in organizations may actually be working against the immutability of containers — the ideal that an image is always the predictable product of the process that builds it, and if the process should change, the image should be entirely replaced.
In the environment whose picture Gareth Rushgrove paints for us, everyday work processes inevitably lead to a situation where the build file cannot speak directly to the structure or contents of its own product.
“It’s not about runtime,” he explained. “The ideal, which is often the case, is that people are using immutable containers. As part of the build, people are running commands that build things that are not immutable… If you take a Dockerfile and you run docker build on it,” he explained, “and if that Dockerfile, for example, has — and lots of them do — npm install package or apt-get install package, it isn’t just installing that single package. They have transitive dependencies. So they’re not just saying, ‘This is the thing you’re installing.’ Even just taking an image and saying, FROM ubuntu, that in itself is a hundred software packages, each with individual gversions.”
The Alpine Linux package is comprised of a mere handful of dependent packages. Even so, said Rushgrove, the addition of something as simple as the Python runtime will add more bytes overall to the final image size, than the Alpine image had by itself.
It may not be possible for a management tool to look forward into the contents of a container image that any container engine will eventually build. What Puppet is advising is a more forensic approach to container image management, that keeps a close and continual watch over its constituent components, and continually monitors Puppet’s new Cloud Discovery service for news about the safety and interoperability of those components.
Puppet is making these additions as part of its latest quarterly Enterprise 2017.2 update, announced Thursday. We asked Omri Gazitt, Puppet’s newly appointed Chief Product Officer — formerly the vice president and general manager of HPE’s cloud-native business unit — whether this latest update marks a shift in the profile of the customers he’s targeting, to more of an AppOps model such as Rushgrove described.
“Operations people today have always been able to get a list of all the packages installed on their systems,” responded Gazitt, “using MCollective. Not exactly the easiest of things to use. Now, it’s just in a GUI. You click it, you get it. You search, you’re done. You’ve always been able to use the orchestrator from the command line. Not always the most engaging thing in the world; not the most collaborative, not the most introspective. The GUI gives a new path for that. As a result, by reducing friction, you can also bring more people into the fold. But that’s not necessarily our target; that’s something that we see is going to happen, as a result of lowering friction, using Puppet as it exists today.”
Puppet Enterprise 2017.2, with Lumogon included, is officially released as of Thursday morning. The company said the tech preview of its Cloud Discovery platform will have a limited release later this month. Version 5 of its open source platform is being geared for release later this month as well.
Feature image: A puppet show taking place in Yangon, Myanmar (formerly Burma) circa 1895 by Philip Adolphe Klier, from the U.S. National Archives.
