Cloud Native / Kubernetes / Storage / Contributed

Cloud Native Apps Demand Cloud Native Storage Systems

27 May 2021 10:00am, by

Gary Ogasawara
Gary Ogasawara is Cloudian’s Chief Technology Officer, responsible for setting the company’s long-term technology vision and direction. Before assuming this role, he was Cloudian’s founding engineering leader. Prior to Cloudian, Gary led the Engineering team at eCentives, a search engine company. He also led the development of real-time commerce and advertising systems at Inktomi, an Internet infrastructure company. Gary holds a Ph.D. in Computer Science from the University of California at Berkeley, specializing in uncertainty reasoning and machine learning.

The rise of cloud native apps is fundamentally changing how organizations develop and operate software apps. But this shift isn’t just changing app development; it’s also impacting infrastructure. New technologies such as containerization, Kubernetes, serverless computing and microservices are key components of cloud native environments. To support modern, cloud native environments — and the emerging technologies that are part of them — organizations cannot rely on traditional storage infrastructure.

Instead, they must employ cloud native storage platforms. To avoid confusion, it’s important to define what “cloud native” means, as the term has been stretched to include apps that aren’t truly cloud native. The Cloud Native Computing Foundation (CNCF) describes cloud native apps as “scalable apps in modern, dynamic environments, such as public, private, and hybrid clouds.” In practice, this has meant apps that are highly distributed, easily scalable, containerized and managed using Kubernetes.

Cloud native storage systems need to be portable, scalable, and support the technologies common in cloud native settings. Put differently, these systems must be equipped to properly manage data for stateful apps, while also addressing the data storage challenges common in cloud native environments, such as obstacles associated with managing data for Kubernetes-based apps.

To do this effectively, cloud native storage must rate highly in six key operational areas: scalability, performance, consistency, durability, deployability and availability.

Scalability

Scalability is a foundational feature of cloud native apps and environments, so it only makes sense that cloud native storage must also be easily scalable. A storage platform’s scalability can be broken down into four categories: client scalability (the ability to increase the number of clients or users accessing the storage system), capacity scalability (the ability to grow storage capacity in a single deployment of storage systems), throughput scalability (the ability to run more throughput or process more data per second), and cluster scalability (the ability to grow a storage cluster by deploying additional components). Cloud native storage must be highly scalable in all four dimensions.

Performance

Performance is critical in cloud native environments. Cloud native storage platforms must offer high-level performance that is also predictable and scalable. This means that systems should be able to rapidly complete a read or write operation, execute a substantial number of storage operations per second, and deliver high data throughput that can be stored or retrieved in MB/s or GB/s.

Consistency

Consistency is also key for cloud native storage systems. Platforms can be described as consistent if read operations promptly return the correct data after it’s written, updated or deleted. If new data is immediately available for read operations by clients after it’s been changed, the system is extremely consistent. However, if there is a lag until read operations return the updated data, the system is only eventually consistent. In this case, the read delay can be considered as a recovery point objective (RPO), because it represents the maximum amount of data loss in case of component failure.

Durability

Cloud native storage needs to be durable and protect against data loss. Durability means more than just accessibility — truly durable platforms guarantee that data can be safely stored for extended time periods. These systems should include multiple layers of data protection (including support for numerous backup data copies) and multiple levels of redundancy (such as local redundancy, redundancy over regions, redundancy over public cloud availability zones and redundancy to a remote site). They also must be able to identify data corruption and automatically restore or reconstruct that data. Deteriorating storage media and failing components are common causes of data corruption.

In addition, the specific storage media that comprises a cloud native storage platform (e.g., SSDs, spinning disks and tapes) should be physically resilient.

Deployability

Cloud native apps are extremely portable and easily distributed across many locations. To support this agility, it’s critical that storage systems can be deployed or provisioned on demand. This requires a software-defined, scale-out approach, which enables organizations to immediately grow storage capacity without adding new appliances. A storage architecture that leverages a single namespace is ideal here. Since these systems connect all nodes together in a peer-to-peer global data fabric, they make it possible to add new nodes (and more capacity) on-demand across any location using the existing infrastructure.

Availability

Lastly, cloud native storage must be highly available, providing immediate access to data in the event of a failure, no matter where that failure occurs (e.g., in the transmission system, storage medium, controller, etc.). To be considered highly available, storage systems should be able to heal and restore any failed components, maintain redundant data copies on a separate device and handle failover to redundant devices.

Conclusion

Cloud native apps are no longer being deployed exclusively in DevOps environments — they are now increasingly being deployed in full production. This includes both stateless and stateful cloud native apps. With more cloud native apps supporting real-world use cases — especially stateful cloud native apps — organizations need storage infrastructure specifically designed for these environments.

Effective cloud native storage must deliver in all six of the areas outlined above. Of course, these aren’t the only considerations, as security, manageability, observability, and API-based automation are also important factors. But for organizations evaluating cloud native storage options, a checklist consisting of scalability, performance, consistency, durability, deployability and availability is the best place to start.

The New Stack is a wholly owned subsidiary of Insight Partners. TNS owner Insight Partners is an investor in the following companies: Real.

Feature image via Pixabay.

A newsletter digest of the week’s most important stories & analyses.