Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Cloud Native Ecosystem / Containers

Containers Aren’t All Public Clouds and Rainbows

The 2020 CNCF survey confirms that container development is important, but that doesn't necessarily mean things are going smoothly.
Feb 18th, 2021 8:12am by
Featued image for: Containers Aren’t All Public Clouds and Rainbows

Nicholas Chase
Nick is Head of Technical and Marketing Content at Mirantis.

Every year CNCF (the Cloud Native Computing Foundation) runs a survey to see what the container and Kubernetes market looks like. The 2020 CNCF survey is out, and while much of the information confirms what we’ve already known (Containers are popular! People are using Kubernetes!), there are a couple of interesting data points to look at.

In a nutshell, container development is important — and in more ways than you might think — but that doesn’t mean things are going smoothly. Here’s what to do about it.

What Are Companies Running in Production?

When the survey first began in 2016, containers and Kubernetes were fairly new and immature. Only 23% of respondents were using containers in production at all, let alone an orchestration platform such as Kubernetes. Over the last 4 years, however, that number has exploded. Today a whopping 92% of respondents are running containers in production, which increases to 95% for each of the testing, development, and proof of concept phases. If you’re part of that last 5%, I’d love to hear what you’re doing instead. Are you still running legacy monolithic applications, or are you building new ones?

So the vast majority of companies are trusting their production workloads to containers, with more than a third (38%) running more than 1,000 containers.

To coordinate those containers, 91% of respondents report that they’re using Kubernetes; with 83% in production. More than 80% of those are running multiple clusters.

Where Are Workloads Running?

As for where those clusters live, conventional wisdom for the last few years has been that workloads would inexorably move to the public cloud — but this year’s survey shows a surprising twist. While use of public cloud is up slightly to 64% this year, the use of on-premise/private cloud is also up, with an increase from 45% in 2019 to 52% last year.

How can that be? Simple: the survey also notes responses of 26% and 36% for multicloud and hybrid cloud respectively. The multicloud option is new this year and the foundation doesn’t believe that it accounts for the slight decrease in hybrid cloud. But considering the semantic confusion between the two, it’s something to consider.

This is perhaps the biggest surprise in the report, showing that for the next few years (at least) private cloud is standing its ground. And that means that companies have to deal with the particular difficulties of on-premise cloud. These difficulties may be as general as ensuring cost structures for hardware, support and other ancillary OPEX are accounted for — as specific as implementing GDPR, or as crucial as finding the means to guarantee cloud security. All of them are, however, surmountable with proper planning and support.

What Problems Are People Facing?

Of course, it’s not all unicorns and rainbows in cloud native land. Two particular concerns stood out among respondents, with 41% of respondents noting both complexity and cultural issues when moving to container-based development.

For example, virtually all companies (97%) are either using, evaluating, or considering CI/CD pipelines for their development. However, while 53% of respondents report checking in code multiple times a day, only 29% actually release software daily. In fact, automated releases are becoming less common, dropping from 40% to 33% as people take more control over their release process.

A number of different factors might be contributing to this drop. For example, how much of this is due to people newly working from home?

Let’s think that through. The survey states clearly that “the portability that Kubernetes and cloud native tools provide makes it much simpler to switch from one public cloud vendor to another,” and by implication, to build functional hybrid- and multiclouds. But that comes from getting a consistent Kubernetes experience across multiple environments, which maximizes workload, tooling and process mobility.

However, since more developers are now working from home, does this imply a less standard development environment than when everyone was in the office? Could the (pretty startling) decrease in fully-automated releases reflect differences between, for example, local development clusters and public-cloud-based production clusters?

Solving the Issues Raised in the CNCF Survey

Let’s look at the potential problems that the survey has exposed:

  • Most companies are working with many nodes making up multiple clusters spanning multiple clouds, which contributes to the fact that…
  • Working with containers is complex, which leads to the tendency to find that…
  • Getting developers to switch to containers is difficult

So what can we do to solve these problems? In general, you’ll want to do some variation on the following:

In the end, you will discover — as most companies already have — that container-driven development is going to be a crucial part of your efforts to ship code faster. The important part is to make sure you have a partner to work with, so that you can concentrate on your applications instead of the infrastructure itself.

Cloud Native Computing Foundation (CNCF) is a sponsor of The New Stack.

Feature image via Pixabay.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma, The New Stack, Mirantis.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.