How to Adapt Data-Centric Applications to a Kubernetes Architecture

27 Jan 2021 3:00pm, by and

Kubernetes is an on-demand orchestration platform that is, by nature, loosely coupled. The term “loosely coupled,” is not used as widely as, say, microservices but has a similar meaning. In both respects, components are lean, have a single purpose and have few dependencies. A microservices architecture provides a way for each service to run independently, as opposed to a tightly coupled approach, characterized by interdependent systems, integrated software and hardware and code that can be more complex to update when changes are made to the application.

In this The New Stack Makers podcast hosted by TNS founder and publisher Alex Williams; guest Nanda Vijaydev, distinguished technologist and lead data scientist, HPE, discusses how the concept of loosely coupled architectures is now playing a part in running data-centric applications on Kubernetes. It’s an evolution that has been taking shape by the use of Kubernetes for microservices development — as opposed to data-centric approaches that have historically been deployed on tightly coupled, monolithic architectures.

Subscribe: SoundCloud | Fireside.fm | Pocket Casts | Stitcher | Apple Podcasts | Overcast | Spotify | TuneIn

As a way of putting Kubernetes and data-centric management in context, Vijaydev relied on the often yet aptly used pets versus cattle analogy to describe their similarities and differences. In this way, the at-scale and highly distributed features that Kubernetes offers represent cattle, in that numerous clusters can be herded and managed en masse, like cattle.

“When you talk about what was Kubernetes is good at or what made Kubernetes popular is a way of stamping out the same type of application or deployment over and over — and that’s where the scale comes into picture where you want to deploy a single web server, then you want to do the same thing, 10 times 10,000 times or 10 million times,” said Vijaydev.  The end result is that repeatability and standardization are completed “in terms of how do you take that to scale,” said Vijaydev so that “when you don’t need it, you shrink it down — so it’s the just-in-time nature, and the stability that this framework offers.”

Scaling up and down microservices at will is a very cattle herding-like exercise while managing data-centric applications on Kubernetes in this loosely coupled context requires more care and attention in many ways.

When integrating data-centric applications into Kubernetes, the challenge thus begins with the fact that “it’s not just a cattle-type of application,” Vijaydev explained.  “You do have to understand that there is a data-access layer and a data-persistence framework that have to work at the same scale, which is really about being able to containerize, being able to provision on-demand and being able to reduce the footprint when not needed, so you can get the resources back,” said Vijaydev.

HPE has bolstered its ability to help organizations’ challenges associated with coupling data-centric workloads running on Kubernetes through acquisitions. These include HPE’s purchases of MapR Technologies, BlueData and Cloud Technology Partners (CTP) acquisition. “There are a few companies who were pioneers in these spaces, and we’ve gotten them together,” said Vijaydev.

HPE has developed and acquired the know-how to create a software platform that enables users to think about data management as a data management problem, and not so much: “‘this is a Kubernetes problem. This is a storage problem. This is a network problem,’” said Vijaydev. “That’s where Kubernetes is right now because each one of those are very, very disconnected from a big picture point of view.”

This post is part of a larger story we're telling about Kubernetes.

Get the full story in the ebook

Get the full story in the ebook