Primer: What Is Container Security?

The tech behind software containers has existed since at least the early 2000s. However, software containers have only become a trending topic in software development in the last few years — after the launch of container development platforms like Kubernetes and Docker.

According to the RightScale 2019 State of the Cloud Report, software containers, and containers-as-a-service, are one of the most sought-after new uses of the cloud — and their popularity is growing year after year.

While software containers are nothing new, the sudden rise in demand for them is. As a result, not all developers may know how to build and use containers in a way that will ensure container and platform security.

If you or your team is interested in using software containers, you’ll need to know about the best security practices you should use to keep these containers and the platforms they run on secure.

What Are Software Containers?

Software containers are similar to — but distinct from — virtual machines.

“With containers, complex applications can be deployed on a single host utilizing many different microservices host in containers on that host,” said Brian Bernstein, systems engineer at Lacework, a Mountain View, Calif. based provider of cloud security solutions.

“In the past,” Bernstein said, “with a host-centric application network or host-based security [you] could see the interaction between each service as it moves between machines. With containers, you may only see inbound and outbound connections, and don’t forget those are likely encrypted. All of the internal connections between those containers will be a complete mystery. By shifting security to the container, you regain the view as we did back in the service per host days.”

“Additionally,” he said, “containers are not pure sandboxes like VMs are. If someone can gain access to a container there is the possibility of a container escape. If they get access to the base host, they then can get access to all of the containers on that host. This is yet another reason why controls should always be deployed at the risk vector, in this case, the containers themselves.”

Almost always, containers will be lighter weight than virtual machines. Containers share the host OS’ kernel during runtime and do not need to fully emulate an operating system to work, unlike a VM. As a result, one physical server running four or five containerized pieces of software will only need to run one operating system. In contrast, a virtual machine would need to run four or five copies of that OS at the same time.

“Containerization provides many benefits in flexibility and faster application development,” said Michelle McLean, vice president of product and corporate marketing at StackRox, a Mountain View, Calif.-based company specializing in security for containers and Kubernetes. And those advantages have driven widespread adoption, she predicted, noting that, according to Gartner, by 2020, more than 50% of global organizations will be running containerized applications in production.

“Containers, along with orchestrators such as Kubernetes have ushered in a new era of application development methodology, enabling microservices architectures as well as continuous development and delivery,” she said, adding that Docker is by far the most dominant container runtime engine, with a 91% penetration according to the company’s latest State of the Container and Kubernetes Security Report.

In addition to taking up fewer resources, containers are also useful because containerized software can run reliably even when moved from one environment to another — from public cloud to private data center to a developer’s laptop.

Many variables can differ from environment to environment. Containers avoid this problem by bundling everything a piece of software needs to run — its libraries, dependencies, configuration files and so on — so the software will function even when running on systems with different OSes or network infrastructure.

The bundled components are called images. Layers of these images make up a software container.

Common Security Risks of Containers

However, this same flexibility that makes containers useful to developers also poses a security challenge. In every container, there are naturally going to be many different and individually complex components that can all introduce security risks and vulnerabilities.

“Containers have revolutionized application development — in particular, the way they are built and scaled,” said Thomas Hatch, CTO and co-founder at SaltStack, a Lehi, Utah-based provider of intelligent IT automation software.

“This, in turn, has led to challenges with security, storage and networking, as well as the complexities faced with deploying to the public cloud or on-premise servers,” Hatch said. .The biggest cloud security threat facing container users is the false assumption that containers equal security.”

“The container is only as secure as the code that runs in it and as the infrastructure it runs on,” Hatch said. “If you redeploy container images that are out of date or carry a vulnerability then you are likely to amplify the risk or enable repeat violations. And keep in mind, a high percentage of containers currently found on Docker Hub have known vulnerabilities and yet people use them as their main container or the basis of their containers.”

“Waiting until going into production to bring in a container and Kubernetes security solution can limit the advantages gained from DevOps processes, because deployments get delayed when security issues are identified only at the end of the development pipeline” —Michelle McLean, StackRox.

“There is a false but prevailing sentiment within the Docker community that the OS is not relevant and that the OS is a commoditized layer,” Hatch said. “This issue highlights how amazingly wrong this assumption is. Secure and validated delivery is the foundation of enterprise operating system. The software its running should always be taken from trusted sources, but the flood of insecure Docker images is a serious problem.”

“What organizations really need to look at is the trust of software sources relative to the complexity of said software,” he said. “Simple, new and small software projects are easy to validate and find issues in. They also develop trust over time. Docker bypasses this natural defense built into open source software because trusted software is being distributed from an untrusted source. Docker also represents a significantly large forest in which these threats can hide. When distributing an OS image there are hundreds of binaries that can be manipulated to stealthily hide issues that current security and DevSecOps tools are not going to see.”

Typically, if a piece of software is at risk, developers only have to worry about the security of the software itself. With containerized software, there are more components to worry about, compared to standard software or virtual machines.

According to the company’s recent survey, even though two-thirds of organizations have more than 10% of their applications containerized, 40% of the organizations remain concerned that their container strategy does not adequately invest in security. Another 34% report their strategy lacks sufficient detail.

“These data show that companies are embracing containers and Kubernetes rapidly but don’t have the security plans or other details for running these systems in place yet.”

There are some security benefits to using software containers. Generally speaking, containers are easier to patch than virtual machines, and you won’t have to worry about operating systems as much. Software developers often build containers with the assumption that software may be able to escape — whereas, with most virtual machines, there is the assumption of a specific level of security.

Cybersecurity and threat detection are two essential components of software development — whether your software is containerized or not. If you want to ensure your containers are secure, you’ll need to take certain precautions.

Securing Software Containers

“Too many companies are waiting until they’re ready to go into production to bring in security,” McLean said. “Most of these environments are often complex, with multiple security, monitoring, logging, and event management solutions deployed. Waiting until going into production to bring in a container and Kubernetes security solution can limit the advantages gained from DevOps processes, because deployments get delayed when security issues are identified only at the end of the development pipeline.”

“Security should be brought into the mix as early as possible,” she said. “[I]t must become a core component of the container infrastructure build and deploy cycles as well as production runtime.”

When developing containerized software and determining what resources a container will have access to, follow the same secure coding best practices you’d follow with any piece of software.

Security starts at the OS level. Many modern versions of popular container platforms enable OS security features by default and make it easy to use whatever kernel safety features are available. For example, in previous versions of Docker, the only permissions settings available for containers were root access or none. Now, you have more flexibility when using what permissions your containers can have. The same holds true across most popular container development platforms.

When developing software and working with containers, you or your team will need to answer certain questions like how you expect containers to talk with each other, how you will manage container updates — especially if you need to update multiple containers at a time — and how you will manage the availability of shared resources, like storage and network access.

“Today’s digital transformation initiatives are all about moving fast,” said Andy Feit, vice president at Aqua Security. “Understanding the business need and quickly creating or adapting the online experience to best position the company for success. A key part of that is getting code from development to production in the least amount of time possible.”

“One way that is being done is to remove the traditional obstacles of refactoring code to run on the chosen IT Operations platform, and instead deploy the application as it comes from development in the form of a containerized image,” he said. “Often this is done in a cloud native model, either on-premise or on public cloud infrastructures. However, these new patterns change the security model, as the classic perimeter security solutions struggle to secure the dynamic nature of these deployments, and continuous updating of the applications through automated CI/CD pipelines leaves little to no time for standalone security testing.”

“Container security solutions address these needs by integrating into the development cycle, flagging any vulnerabilities as the code is created,” Feit said.  “As the containers move towards deployment — wherever they will ultimately run — container security ensures that the image as approved has not been modified.  Finally, runtime monitoring of containers looks for any signs of malicious behavior and allows for granular enforcement of policies that only allow authorized activity.”

Ideally, you’ll be using a container platform designed to automatically and regularly scan and patch containers, ensuring these containers are always using the latest versions of their dependencies and libraries. Doing so will reduce the security risk presented by the other code your containers depend on.

Working with a well-known container platform like Docker or Kubernetes will give you access to security protocols that can help you or your team ensure your containers are secure. You don’t, however, need to work with a container platform to use containers securely.

If you or your organization is willing to spend some money on security, enterprise container security solutions exist. These solutions can audit container images and policies, alerting you to outdated resources and potential security risks posed by your containers.

In many ways, however, software container security is still a bit of a Wild West situation — software developers haven’t come to any solid conclusions on how best devs should approach container development to reduce security risks. Some organizations, like the National Institute of Standards and Technology, have published security standards that can help developers build and use software containers securely. Guidelines like these might be helpful if your organization lacks institutional knowledge that can help you guide your approach to container security.

How Devs Can Keep Containers Secure

“As new development and operations technologies gain adoption,” Bernstein said, “the speed and dynamic nature of containers grow as well. It is not enough to treat a container like a host and scan it for vulnerable packages. You must take a holistic approach across the whole lifecycle.”

“Security tools are now looking to solve the problem from build time, shift left, to run time,” he said. “By addressing security across the entire lifecycle, you raise the likelihood of catching issues before they happen. For example, how do you know that a container that is built today doesn’t have a vulnerability a week later? How do you know if a mistake in your web interface doesn’t lead to leaking of data? It is examples like these that show that security tools need to address the whole problem, not just points along the way.”

“The way modern DevOps operates,” he said, “it is impossible to implement successful controls without automation. The scale and dynamic nature are too much to try and keep up by using static or manually updated tools. If your security tool cannot keep up with the way your DevOps teams operate, then it is unlikely it will be adopted or effective.”

Feature image by Volker Glätsch from Pixabay.