TNS Research: How the IT Landscape is Shifting to Accommodate Containers
From a standing start, an ecosystem has quickly grown to support the use of containers. The year 2015 picked up where 2014 left off, with continuing venture capital investment, new company formation and a slew of new products and technology initiatives.
To better understand the dynamics of this ecosystem, The New Stack initiated a survey of companies that provide or plan to provide services associated with containers. We found that vendors are adjusting their portfolio of services to meet the demands of both application developers and IT operations.
Executives at these companies tell us their products have an impact on the IT landscape, with PaaS and provisioning technologies affected most. The survey also gave us the opportunity to learn how these vendors, leading adopters of container technologies themselves, have adapted their own technology stack.
The container ecosystem consists of a wide variety of companies that support the use of containers with their own products. Some of these companies are what might be called “pure-plays,” meaning their offerings were specifically designed to support Docker and containers; others are traditional vendors, quickly adapting their own offerings to remain relevant in a containerized future.
In April and May of 2015, The New Stack invited 106 vendors associated with Docker and containers to participate in an ecosystem survey. About 45 percent of these companies responded. The companies surveyed represent a broad cross-section of the overall container ecosystem, as can be seen by comparing the 48 responding companies listed below with our broader take on the container ecosystem from the Ecosystem Directory.
- Management and orchestration are top priorities for Docker and container ecosystem companies.
- Companies plan to invest in developer tools, networking and security functionality.
- Vendors fear competition from end-to-end solutions offered by IBM, AWS and Heroku.
- The PaaS market is evolving quickly as vendors offer platforms to deploy containerized applications.
- PaaS providers will face new competitors but are likely to benefit from increased market demand.
- VMware and configuration management systems like Chef and Puppet Labs face disruption.
- Ecosystem vendors are targeting enterprise/ISV practitioners of DevOps.
When asked to broadly categorize their current offerings, the top two categories of products reported by surveyed companies indicate that early players are taking broad, platform-oriented approaches to the market, tackling areas such as application development and deployment platforms, schedulers and orchestration. The actual categories that these products are being marketed under varies. Companies in this space market tools targeting similar pain points as PaaS platforms, cluster management systems, data center “operating systems,” CI/CD systems and hosting environments.
Many companies in the container ecosystem sell products and services traditionally viewed as being for developers, but which offer added functionality to manage infrastructure — and vice versa. The perception that application development and orchestration are interconnected is a strong theme within the ecosystem. Half of the companies surveyed currently have offerings that purport to be in both spaces. In fact, among companies that are currently in one of the two main categories, over 80 percent expect to have products in both spaces within two years.
A company’s current product portfolio is the result of its journey over the last few years. Many of the younger companies in the survey seem to be building solutions based on broad visions of how customers should be using container technologies. We expect that as the market matures and clear leaders in the platform space become apparent, activity in the space will shift to offering niche solutions to addressing specific customer pains, such as security.
The larger, older companies we surveyed were significantly more likely to offer configuration management tools, security and networking functionality. This is partially because many of the leading configuration management vendors have started to add Docker support to their existing products. It is also likely that many of the larger companies were describing a security or networking tool they sell that may or may not actually be purpose-built to deal with containers.
We also asked the surveyed companies where they saw their product portfolios going. The answers we received reflect vendors’ focus on solutions that address more narrowly defined pain points. Developer tools, networking, security and configuration management functionality with support for Docker and containers all rank high on vendor roadmaps.
Nearly one-third of companies are planning to invest in developer-focused tools, either via internal development or acquisition. Every company that is planning to develop either new developer tools or configuration management tools with Docker support is already providing application development/deployment platforms. As these platforms mature, it makes sense that their providers will need to provide additional tools to fill gaps in the developer experience.
While many companies surveyed are offering broad product suites, there is also broad recognition that they cannot do everything themselves. We found companies most likely to seek a partner for hosting services. While companies like the Docker-owned Tutum offer end-to-end Docker-centric hosting environments, many vendors are offering software that can be deployed to any cloud or on-premise infrastructure. Container vendors will likely look to partnership programs offered by Amazon Web Services (AWS), Microsoft Azure and Google Compute Engine (GCE) to provide a more complete solution for customers, while remaining leery of those companies’ own container offerings.
Similarly, while companies recognize the importance of databases and big data capabilities, most choose to leave those capabilities to specialists. For networking functionality, almost half of the younger companies in the ecosystem are seeking to partner to address this gap in their offering.
Most vendors see themselves as automating traditional application development or IT processes, but a few are targeting more specific areas. When we asked about what types of end users they are targeting, about three-quarters said either “IT Operations & Admin” or “Application Development.”
However, an unusually large number of participants opted to write-in answers that mentioned DevOps or a combination of IT operations and application development. One respondent wrote: “We consider DevOps — ops people who code or developers who have to think about running an application — as our target customer.”
The New Stack believes that if DevOps were offered as a choice, it would have likely been chosen more often. One company noted:
“Today, the primary use case is to provide application development environments to build new applications that will benefit from Linux container packaging.”
Surveyed companies cited targeting both large enterprises and SaaS/ISV firms in approximately equal numbers. This indicates an attempt at balancing the pursuit of early-adopter technology companies with a desire to tap into the large budgets of traditional “big enterprise.”
As use of containers expands, vendors are learning the types of functionality this target market wants. As a whole, executives believe that customers are looking for security, simplified container management, and new orchestration and deployment features.
Participant quotes about technical issues driving user demand:
- “Complexity in getting from dev to production efficiently. Existing alternatives are complex and constraining, and have a large management overhead.”
- “Want to use containers in production while changing as little else as possible.”
- “Challenges dealing with security and compliance in public, private, hybrid and multi-cloud environments, including gaining visibility into the security state of containers and microservices.”
- “The need to modernize infrastructure in order to capitalize on the business opportunity driven by the emergence of container-dependent technologies in the cloud services marketplace.”
- “Docker/containers in production (at scale) is a lot of work and hassle; pain of handling servers/clusters at scale; technical overhead for server operations; moving to microservices architectures comes with a lot of hurdles.”
- “Managing their own continuous delivery platform is a distraction from working on their product. Furthermore, a scalable CD infrastructure that grows with your team is a lot of work. They much rather have somebody else take care of scalability, security and maintenance.”
IT Ecosystem Impact
Without a doubt, the rise of Docker and container technologies is making waves in IT. The New Stack asked a series of questions about how vendors think their products are affecting customers’ technology stacks. Their answers point to those vendors and technologies most impacted by the industry’s adoption of container technologies.
Only 15 percent of companies say their products are being used for specific, narrowly-defined use cases. As one respondent noted: “Tools and services for this new type of infrastructure [are] full of promise to one day replace or supplement significant portions of data center environments.”
PaaS is the category that surveyed companies expect to be most impacted by the introduction of their new container-related products, with almost three-quarters of survey respondents indicating this choice. This might seem like a high number in light of various studies indicating that actual adoption levels of PaaS are significantly lower than for other cloud services.
The New Stack believes, however, that a primary cause of this disconnect is a gradual redefining of PaaS that has taken place over the past year, significantly broadening the meaning of this term beyond the tightly integrated stacks that traditionally used it. In this context, containers are not eliminating the need for PaaS, but are rather the catalyst for a new generation of IT professionals’ reevaluation of how applications are developed and delivered. The companies targeting application development teams sure believe so — all but one said their products are affecting the PaaS part of the IT stack.
After PaaS, many respondents see their products influencing the IaaS and provisioning/configuration management categories. It is not surprising that AWS was the leading IaaS cited as being affected. The situation that Chef and Puppet Labs find themselves in is more interesting.
Among those that named a provisioning or configuration vendor being affected by their company, eight out of 17 named both Chef and Puppet Labs. These results indicate two things: these companies are synonymous with the space itself and they are being threatened with disruption caused by container technology. As more and more vendors offer “pure-play” container-based alternatives to traditional configuration management, we expect this to present new challenges to Puppet Labs and Chef.
The New Stack found that the surveyed companies are generally aligned with Docker, with 42 out of 43 respondents either currently partnering with Docker (60 percent) or planning to partner with the company in the future (37 percent). Only one company did not expect a Docker partnership in the future. Although Docker has good relations with these companies, there is a split about how important Docker’s technology plans are to their own direction, indicating that the partnerships are tactical rather than strategic.
While we believe Docker must execute very carefully to secure its position in the container market, it faces limited threat from surveyed vendors. Only two of the 31 companies willing to name their primary competitors cited Docker. Instead, companies are more concerned about PaaS providers, like IBM, with its Cloud Foundry-based Bluemix offering, and Heroku.
AWS is named as a competitor, at least in part because of its EC2 Container Service (ECS). It is notable that Tutum, a provider of specialized container hosting services, was mentioned by several startups as their main competitor. These responses suggest that the providers surveyed fear being locked out of participating in end-to-end hosted solutions more than they fear competing with individual point products or projects.
VMware was also named as a competitor by several companies. Although VMware has been developing new offerings in this space, it is more likely that container vendors are positioning themselves as an alternative to existing infrastructure that relies on hypervisors. In fact, among companies seeing an impact on their customers’ hypervisor or container environment, 80 percent say VMware is the vendor impacted most. As use of containers continues to progress, the threat to VMware is that new workloads will not rely on hypervisors, and that VMs will not only be supplemented by containers but made superfluous.
Who’s Using What?
A common refrain in today’s “software is eating the world” era is that traditional enterprises need to act more like software companies in order to be successful. Along these lines, we can learn a lot from looking at what the vendors we surveyed are using to support the development of their own projects.
Being able to operate across multiple cloud platforms is one of the main benefits of using containers. Thus, it is no surprise that companies are supporting the use of more than one infrastructure to deploy their services. While it’s also no surprise that AWS leads the lineup of supported infrastructures, most companies are supporting it in combination with other platforms. In fact, nearly a third of vendors’ products are used by AWS, Microsoft, Google and OpenStack. This indicates a dominant strategy to be platform independent.
Peering Into the Container Ecosystem’s Technology Stack
Participating companies eat their own dog food — that is, they’re following their own advice about container technologies. 92 percent of respondents use containers in their internal operations and 96 percent rely on at least one open source project. Yet it’s surprising that “only half” of the companies say their technology relies on Docker. Even if many companies say they do not rely on Docker per se, they are still heavily invested in open source, with two-thirds having more than five employees contributing to a project. In addition, almost 50 percent of companies say they are using a NoSQL database internally.
In terms of how surveyed companies are managing their own container-based environments, responses were all over the map, with no single response garnering more than four percent of total responses, and some 30 tools mentioned just once. This suggests a very high degree of fragmentation and an impending shakeout.
Perhaps surprising to those not following closely is the degree to which the Go language is used by container companies. While it is no surprise that the relatively mature Ruby and Java are being heavily used, 25 percent of companies said that Go is needed to use their product or service. We expect Go is even more popular among these companies for internal applications where developers don’t need to worry about whether a customer knows the language or not.
Docker, IBM, Red Hat and VMware are sponsors of The New Stack.
Feature Image: The U.S. Office of War, 1945, via the U.S. Library of Congress.