Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Containers / Kubernetes

Data on Kubernetes Has Its Moment in the Sun

Here's what's possible now in terms of using Kubernetes as a data layer, with the help of the Container Attached Storage pattern as embodied by OpenEBS.
Aug 18th, 2020 3:00am by
Featued image for: Data on Kubernetes Has Its Moment in the Sun

MayaData sponsored this post.

Evan Powell
Evan Powell is CEO of MayaData.

Apps need data. Apps on Kubernetes are no exception, which is why we at MayaData and the OpenEBS community have been working for years to become the most popular solution for running data on Kubernetes (according to last year’s survey from the Cloud Native Computing Foundation). What’s the secret to state on Kubernetes? Container Attached Storage, or CAS.

In this article, I’d like to explain what is possible now in terms of using Kubernetes as a data layer, with the help of the Container Attached Storage (CAS) pattern as embodied by OpenEBS.

Don’t Break Your DevOps. Small Teams Rule!

One of the central tenants of DevOps and the shift to microservices is that small teams should be able to select the tools they need and should also be autonomous and accountable. Shared storage or the use of a centralized “database to rule them all” breaks this pattern. Container attached storage from OpenEBS puts these small teams back in control. They can pick whether they need a data engine that replicates their database or other workloads — or if they trust their Kafka or Cassandra or other workloads to handle resilience itself. Or if they have NVMe systems, they can — for the first time — deploy extremely high-performance PVCs with the help of MayaData’s MayaStor (just reaching beta).

Your Mistakes or Operational Debt Don’t Trend on Hacker News

In the last few weeks, Garmin and Red Hat (via have made the news. Garmin’s data was held hostage and Quay seems to have had a number of cascading outages of its MySQL and other data management, as explained in its root cause analysis blog.

Using CAS solutions such as OpenEBS minimizes your blast radius. Yes, you can and often will run it on top of an old shared storage system from core IT — or even a storage service from your favorite big cloud. However, a) you don’t need to and b) whatever you do, your blast radius will inevitably be much much smaller.

Hey, this “loosely coupled” thing (read about it in the CNCF’s definition of cloud native here) has lots of benefits! Avoid the perils of the distributed monolith pattern that depending on cloud A or shared storage B subtly introduces into your environment.

Lock-in: Don’t Debate It, End It

In a recent article, The New Stack asks whether cloud lock-in is such a big deal. They conclude that it isn’t really, since cloud providers don’t increase their prices much.

Here’s the thing: the baseline for costs in IT were predicted by Gordon Moore 55 years ago. So when clouds do not drop their prices, they are raising prices. If your competitors are able to capture more of the exponential improvement in the cost-effectiveness of IT — perhaps because they are not locked in and can better negotiate with their underlying clouds with the help of MayaData’s Kubera and OpenEBS — then they’ll be able to better write and operate software, and hence they’ll out-innovate and out-compete you.

What’s more, lock-in is a fundamental part of the business model of traditional storage companies. They encourage you to leverage their particular capabilities and once you do so, they use their leverage to reduce your negotiating power. They also make it hard to move data across systems from different vendors.

By running your data on Kubernetes while extending the functionality of Kubernetes, you have a data plane that is as mobile as Kubernetes. If those extensions to Kubernetes, like OpenEBS, are open source, then you have in one move reduced your cloud lock-in and your lock-in to that old storage company as well. The result is that your teams are freed to make more intelligent decisions and freed from the risk of lock-in — plus you save money.

You Are Not Alone

Retaining small team autonomy, reducing your blast radius, and preventing cloud and vendor lock-in are age-old dreams. For the dozens of public reference users of OpenEBS — ranging from Bloomberg to Arista and from Orange to Comcast and others — that dream is coming true.

So head over to to try it out, or grab our free forever for individual use SaaS solution called Kubera, to both deploy OpenEBS and manage Kubernetes as your data layer.

If you are like many in and around the Kubernetes community, running data on Kubernetes is top of mind. If you’d like to compare notes with other practitioners, we strongly recommend an independent vendor and project neutral community called, perhaps not surprisingly, the Data on Kubernetes Community. Head over to the Data On Kubernetes forum where you can find coordinates to upcoming talks, their Slack channel, past discussions saved on YouTube, and more.

The Cloud Native Computing Foundation and Red Hat are sponsors of The New Stack.

Feature image via Pixabay.

At this time, The New Stack does not allow comments directly on this website. We invite all readers who wish to discuss a story to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email:

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.