Storage / Contributed

Surprise! Developers Are Leading the Next Storage Evolution

27 Sep 2018 6:00am, by

Irshad Raihan
Irshad Raihan is a product manager at Red Hat Storage, responsible for product strategy, messaging, and go to market activities. Previously, he held senior product marketing and product management positions at HP and IBM responsible for big data and data management products. Irshad holds a Masters in Computer Science from Clemson University, and an MBA from Carnegie Mellon University. He is based in Northern California and can be reached on Twitter @irshadraihan.

Everything evolves. People, cultures, attitudes, thinking. As a species, we are simply never meant to sit still. Enterprise IT is a perfect example.

The desire for something better ultimately led to the evolution from bare metal to virtual machines (VM), which allow organizations to better utilize resources for multiple applications with interdependencies. Still, as many modern cloud-native applications and microservices require agility, and more granular resource allocation, VMs can become unwieldy and sluggish, undermining the desire of the modern developer for greater speed and efficiency.

The advent of containers addressed many of the compromises of VMs. However, given the ephemeral nature of containers and the focus on stateless applications, one important feature got neglected: storage.

Lost in a Container

Containers were initially created as stateless tools that were not designed with storage in mind. But since containers’ inception, many developers have discovered that the desire for small, lightweight, and portable solutions that can run just about anywhere requires persistent storage.

Many applications still need to persist their states, data, and configurations. Think of what would happen if you ran a service like Lyft and, during a ride, your cloud-native application got lost in a container and had to restart somewhere else. Without persistent storage, you would lose that customer’s ride data — bad for the customer, but much worse for the driver. Indeed, any agile developer or person using a NoSQL database will recognize that persistent storage is necessary for their stateful applications.

Monoliths Were Not Built for Microservices

Unfortunately, traditional monolithic storage arrays were not made with today’s applications in mind. They are difficult to scale, and local storage capacity is limited. Traditional storage units are also expensive to maintain and upgrade. Some, depending on their age, may not be maintainable or upgradeable at all, and may have high operational costs.

Containers require highly elastic and persistent storage. That has led us to another evolutionary stage in which developers are beginning to provision the storage their applications will consume and ensuring those applications can mount and use that storage within their containers. Software-defined storage (SDS) allows applications and storage management components to operate side-by-side within the container. This brings persistent storage to the container, ensuring that even if the container or application goes away, the data remains.

A Single Control Plane? Yes, Please.

While providing containers with persistent storage is undoubtedly impressive, it is not the only innovation brought about by packaging storage and applications within the same location. Storage management has traditionally been handled by storage administrators, but having storage and applications running within the same container provides a single control plane for developers to manage both. They no longer need to worry about inconsistent persistent storage among development, test, and production environments; they have far more granular control, allowing them to maximize container performance and achieve high availability.

Traditionally, when a developer would need more storage capacity, they would need to submit a request to the storage admin ahead of time. In doing so, they would need to try to predict how much storage they would need — an inaccurate science, to be sure, as it is impossible to predict whether an application may have five or five million requests. The developer may lean toward overestimating, which could lead to the storage administrator saying there is not enough storage for their needs and that they would need to wait for the next storage upgrade. Or, they could say they have enough, which would beg the question of why the enterprise is harboring so much capacity. Either way, the request exposes the budget, efficiency, process and scalability constraints of traditional storage for modern application development.

Open SDS can empower the developer to be able to scale their storage requests based on their application’s needs. Using an orchestration framework like Kubernetes, developers can scale storage up or down without submitting a ticket into the storage administrator; what would typically take hours can be accomplished in just a few seconds. They can better optimize their capacity and avoid over-provisioning, all within a single orchestration framework for managing applications, storage, and infrastructure. And they can expect the same dynamic, resilient, agile user experience regardless of whether they choose to deploy on bare metal, VM’s, or a public or private cloud infrastructure.

The Next Evolution

Developers have typically looked upon storage with a sense of trepidation. It was perceived as important, but something that could slow them down; a necessary evil.

But the advent of containers and SDS have led many to not only embrace storage but contribute to new and exciting projects and tools that incorporate storage into the application development process. They have gone from building really big things that require a lot of planning, review cycles, and capital outlay, to creating small, agile applications and services that, working together, have become something bigger and better than what we had before — the true definition of evolution. In doing so, they are not only empowering themselves, but giving their organizations more efficient, agile, and cost-effective ways of building and managing applications and storage.

We are still at the very beginning of this evolutionary process. That means that developers have an enormous opportunity to shape where things go next, but it also means that enterprises have a choice to make when it comes to their storage processes. Will they choose the traditional storage methods they have been using for years, or will they consider making the move toward modern SDS, which will allow them to maximize their return on investment in VMs and containers?

Will they choose to evolve?

Red Hat is a sponsor of The New Stack.

Feature image via Pixabay.

A newsletter digest of the week’s most important stories & analyses.