CI/CD / Data

Solved! Kubernetes Operators for Clustered Database Deployments

22 Apr 2019 12:00pm, by

Ravi Mayuram
As Senior Vice President of Engineering and CTO, Ravi is responsible for product development and delivery of the Couchbase Data Platform, which includes Couchbase Server and Couchbase Mobile. He came to Couchbase from Oracle, where he served as senior director of engineering and led innovation in the areas of recommender systems and social graph, search and analytics, and lightweight client frameworks. Also while at Oracle, Ravi was responsible for kickstarting the cloud collaboration platform. Previously in his career, Ravi held senior technical and management positions at BEA, Siebel, Informix, HP, and startup BroadBand Office. Ravi holds a Master of Science degree in Mathematics from University of Delhi.

As we have witnessed countless times across every industry, today’s corporate technology watch phrase is: Be Agile or Die. Enterprises, their products and services, and how they interact with their customers is undergoing a complete revolution. No sooner has a new business model or technology been successfully launched than emerges a new, improved paradigm focused on replacing it. The only practical solution is to stay agile and adapt.

Incorporating agility into the enterprise technology stack is the challenge of our time. Continuous Integration and Continuous Deployment (CI/CD) has become the primary model that leading companies are implementing as fast as they can, in order to keep up with the demands for change and agility.

On the CI side of the equation, there are hundreds of articles written and dozens of tools available to address the agility needs of application developers. Consequently, application development has moved out of traditional IT departments and into DevOps almost completely.

However, the CD side of the problem is a bit more challenging. Both virtual machine (VM) and container technologies have arisen to address continuous deployment, but with somewhat limited success. These technologies are good at encapsulating and deploying lots of things that look the same, like file servers, applications, and microservice components where each service is very self-contained. However, they fall short when it comes to deploying complex, stateful database components in a distributed, clustered environment. Most often, database deployments require complex scripts to ensure that the interdependencies are configured, deployed and operated in the correct order.

These manually coded scripts are hard pressed to include capabilities like availability zones, disaster recovery, and cascading failure scenarios, and usually lack flexibility if they cover them at all. Scripts also tend to be very cloud-provider specific, thus limiting flexibility and making cloud-agnostic deployments virtually impossible. Over the last several years this has lead to a fragmentation of CD: easily packaged and containerized enterprise services being handled by DevOps VS hard, complex, cloud-specific clustered technology (like databases) being handled by IT or other specialized groups.

But what if we could make even “complex” CD simple and automated, that allowed custom, stateful, context-sensitive deployments? Enter the Kubernetes Operator Framework. The Operator Framework is an open-source toolkit to manage Kubernetes native applications, called Operators, in an effective, automated, and scalable way. Fundamentally, they allow complex, interdependent, distributed services (like clustered databases) to describe how they should be deployed, operated and managed via simple YAML configuration files.

Automating the deployment and operation of clustered databases fulfills the goals promised with CI/CD. It reduces time-consuming, manual, error-prone intervention in database deployments, improves resource management and puts databases into the DevOps domain, making databases just another component of your agile CI/CD infrastructure. Database Administrators no longer need to focus on getting the database “up and running” and then “keeping it running,” since that is exactly what the Operator Framework will automatically do for them. This is also great news for System Architects because it simplifies, clarifies, and automates the overall application deployment architecture, making it more reliable and less brittle.

Let’s look at a specific example — the Couchbase Autonomous Operator for Kubernetes. Based on years of experience with Kubernetes containers, Couchbase engineers used the Operator Framework to create a custom Kubernetes Operator for the Couchbase Data Platform — the first NoSQL database product to do so! The Couchbase Autonomous Operator automates deployment, scale-up and scale-out operations, upgrades, cluster monitoring, node failure detection and automatic recovery, data distribution across availability zones and enables disaster recovery via automatic configuration for Cross Data Center Replication (XDCR). The Couchbase Autonomous Operator makes it easy for DevOps personnel to define the configuration and operational parameters that they need, tell Kubernetes to deploy the appropriate resources and monitor them for failures. DevOps personnel can gather status and information about the cluster and its operation from the Kubernetes or Couchbase management consoles.

Couchbase and its Autonomous Operator for Kubernetes removes the administrative friction between the application front-end and database back-end. Historically, especially with relational databases, database, schema, and application changes had to be made separately, forcing administrators to prepare, manage and deploy each component individually.

With Couchbase, you can let the application requirements and changes drive all of the modifications. The administration tasks and actual mechanics involved in updating all of the nodes that run these instances are now done by Kubernetes. DevOps only needs to specify “what” needs to happen in response to an application change, and the Kubernetes Operator(s) will take care of the “how.” This allows the DBA and “Ops” teams to be liberated from the mundane task of maintaining a database

By using Kubernetes Operators, Couchbase is able to make the Couchbase Data Platform an equal partner in the CI/CD objectives of the enterprise. DevOps gets better control over database changes and ensured flexibility in the future, as cloud platforms, application requirements, and customer interactions change. In turn, DBAs can focus on higher value aspects of database management and provide better quality of the “data” service to the applications and to the end customers.

For more on the Couchbase Autonomous Operator for Kubernetes, please follow this link.

Feature image by M Ameen from Pixabay.

A newsletter digest of the week’s most important stories & analyses.

View / Add Comments

Please stay on topic and be respectful of others. Review our Terms of Use.