Cloud Native / Sponsored

Sharded Deployments with MongoDB and Brooklyn, a Framework for Scaling

28 Apr 2015 6:33am, by

This post originally appeared on the Cloudsoft blog to demonstrate how to create services with the Apache Brooklyn Plugin for Cloud Foundry. Apache Brooklyn is a platform for integrating services across multiple data centers. It is the fourth post in a series of that covers how Brooklyn works with a wide range of services. Part one explored how the Cloud Foundry service broker works with Brookyn. Part two in the series covered how a user can deploy the Brooklyn plugin to integrate Cloud Foundry. Part three demonstrated how Apache Ambari’s deployment integration technology fits with Apache Brooklyn.

The nature of distributed architecture requires the user to think through how apps and services may run across multiple cloud services and data centers. Apps are one thing, but running a database in these types of environments carries a different level of complexity.

In this example, we are going to create a more complicated service involving MongoDB and sharding. We will then show you how you can use effectors to scale this service out or in.

Brooklyn Service Broker Creating a 3 node, 5 shard MongoDB instance

MongoDB offers some pretty advanced options for scaling, both horizontally and vertically. So, we’re going to look at how to create a sharded deployment (like that described in the figure above) with the Brooklyn plugin.

Create a Sharded Deployment Service

First, taking advantage of our new ability to specify blueprints in the application manifest, under the Brooklyn section, create a MongoDBShardedDeployment service:

Notice with the blueprint, you can also specify provisioning properties and configure the sharded deployment. We have specified a minimum of 16GB of RAM with one router, five shards and three nodes per shard in this blueprint. Simply push this blueprint …

and you have a sharded deployment of MongoDB.

MongoDB Sharded Deployment Running in Brooklyn

Examine the Sensors

As we saw in part two, you can view the sensors associated with a running service with the sensors command, so let’s do that with this service:

In this example, there are lots of entities and many more sensors, but a few are worth pointing out: service.state shows whether the entity is running; host.address and mongodb.server.port show how to connect to the individual MongoDB servers, should you need to; and mongodb.server.network.bytesIn and mongodb.server.network.bytesOut show how much traffic is coming in and out.

Scale it Back Using Effectors

So, looking at the sensors, suppose we decide we would like to scale back the number of shards to one. To find out which effector to use for that, we use the effectors command:

The one we need is cfgFazAk:resize so let’s invoke that:

Brooklyn Showing one ReplicatSet after resize

Scale it Back Out

And if we need to scale back out again, we simply run the same command again but with the parameter changed:

What about increased replication? Just look at the effectors to find the cluster you want to increase and run the same command:

Brooklyn Showing 5 mongos in the cluster

Video

To wrap up this post, once again, here is a video of our plugin in action:

Next Time

In previous posts, we saw the basics of using the Cloud Foundry Service Broker and Plugin to manage Brooklyn services, from creating services in the catalog to adding new items to the catalog; from specifying catalog items in the manifest to specifying new types of services in the catalog. And now we’ve seen a bigger service that we can control using sensors and effectors.

But Brooklyn has more, and in the next part we are going to show you how to deploy an application to Cloud Foundry with multi-cloud, polyglot persistence provided by Apache Brooklyn. See you then.

Cloudsoft is a sponsor of The New Stack.

Feature image via Flickr Creative Commons.

A newsletter digest of the week’s most important stories & analyses.

View / Add Comments

Please stay on topic and be respectful of others. Review our Terms of Use.