Cloud Foundry and Apache Brooklyn for Automating PaaS with a Service Broker

Mar 11th, 2015 8:00am by Alex Williams

This post originally appeared on the CloudSoft blog to demonstrate how Cloud Foundry integrates with Apache Brooklyn, a platform for integrating services across multiple data centers. It is the first in a series of blog posts that covers how CF is used with a wide range of Brooklyn blueprints, such as Cassandra, Couchbase, ActiveMQ and Kafka, across enterprise and cloud service environments. Part two in the series covered how to use the Brooklyn plugin to integrate Cloud Foundry with Apache Brooklyn.

Cloud Foundry is a platform as a service (PaaS), making it easy to deploy, run, and scale applications on top of a host of runtime frameworks.

Apache Brooklyn is an application blueprint and management system which supports a wide range of software and services in the cloud.

In this post we will learn about the service broker in the Cloud Foundry ecosystem. We will explore how this modular component bridges the gap between service providers and the CF tooling. The next post will cover a CF plugin that makes it simple to specify services in your application manifests, so you can create them and bind them to your application with one command. After that, we will give some examples, including a sharded MongoDB instance in your manifest for a truly scalable application and a Riak cluster.

The Service Broker

The Service Broker component in CF allows developers to decouple their apps from the services that they consume. The cf create-service command calls to the broker to create a new instance of a service. You can then include a reference to this service instance in your application’s manifest, or manually issue a bind-service, to associate it to one or more applications. CF will then automatically inject details of the service instance — such as the URL and credentials for consuming it — into your application’s runtime.

Until now, most service brokers have dealt with a single type of service, requiring a lot of up-front investment to make a new service available in CF and limiting the degree to which these services can be customized when needed for applications.

By making Apache Brooklyn available as a service broker, however, you only need one broker to expose the broad catalog of existing Brooklyn blueprints. And adding new services is as simple as POSTing a blueprint to the Brooklyn catalog.

Let’s start by installing the Brooklyn Service Broker. You’ll need Java 8, git and Gradle installed, and then:

<tt>$ git clone https://github.com/cloudfoundry-community/brooklyn-service-broker.git
$ cd brooklyn-service-broker
$ gradle clean build
</tt>

<tt>$ git clone https://github.com/cloudfoundry-community/brooklyn-service-broker.git

$ cd brooklyn-service-broker

$ gradle clean build

</tt>

This generates a file brooklyn-service-broker.war containing the service broker.

Next we’ll need an Apache Brooklyn server. If you don’t have one, it’s easy to install following the instructions here.

Once that’s running, make a note of the brooklyn-host and brooklyn-password, and install the Brooklyn service broker WAR file to /path/to/broker-war/. Choose a broker-user and broker-password, and in the YAML below replace these five tokens with the real values. Paste the resulting blueprint into the “Add Application” dialog in your Brooklyn server, and within a minute your Service Broker should be running (usually on port 8080, but see the sensors of the Tomcat if that port is already in use).

<tt>name: Brooklyn Service Broker
location: localhost
services:
- type: brooklyn.entity.webapp.tomcat.TomcatServer
  name: Tomcat Server
  war: /path/to/broker-war/brooklyn-service-broker.war
  brooklyn.config:
    java.sysprops:
      brooklyn.uri: https://brooklyn-host:8081
      brooklyn.username: admin
      brooklyn.password: brooklyn-password
      security.user.name: broker-user
      security.user.password: broker-password
      spring.data.mongodb.host: $brooklyn:component("mongodb").attributeWhenReady("host.address")
      spring.data.mongodb.port: $brooklyn:component("mongodb").attributeWhenReady("mongodb.server.port")

- type: brooklyn.entity.nosql.mongodb.MongoDBServer
  id: mongodb
  name: Service Broker Mongo DB
</tt>

<tt>name: Brooklyn Service Broker

location: localhost

services:

- type: brooklyn.entity.webapp.tomcat.TomcatServer

war: /path/to/broker-war/brooklyn-service-broker.war

brooklyn.config:

java.sysprops:

brooklyn.uri: https://brooklyn-host:8081

brooklyn.username: admin

brooklyn.password: brooklyn-password

security.user.name: broker-user

security.user.password: broker-password

spring.data.mongodb.host: $brooklyn:component("mongodb").attributeWhenReady("host.address")

spring.data.mongodb.port: $brooklyn:component("mongodb").attributeWhenReady("mongodb.server.port")

- type: brooklyn.entity.nosql.mongodb.MongoDBServer

id: mongodb

</tt>

Adding Brooklyn Services to Cloud Foundry

Next, we’ll register this Broker with Cloud Foundry. If you don’t yet have a running instance of CF, see https://docs.cloudfoundry.org/deploying/, log in to it, and then continue.

First, let’s add a service to the Brooklyn catalog so we have something to access from Cloud Foundry. In the Brooklyn GUI, select the “Catalog” tab and then click the “+” and the “entity” button and add the following service blueprint:

<tt>brooklyn.catalog:
  id: mysql
  version: 1.0
  iconUrl: classpath://mysql.png
  description: MySql is an open source relational database management system (RDBMS)
name: MySQL Database
services:
- type: brooklyn.entity.basic.BasicApplication
  brooklyn.children:
  - type: brooklyn.entity.database.mysql.MySqlNode
    id: mysql
    name: MySQL Node
</tt>

<tt>brooklyn.catalog:

id: mysql

version: 1.0

iconUrl: classpath://mysql.png

description: MySql is an open source relational database management system (RDBMS)

services:

- type: brooklyn.entity.basic.BasicApplication

brooklyn.children:

- type: brooklyn.entity.database.mysql.MySqlNode

id: mysql

</tt>

Now register the broker with Cloud Foundry:

<tt>$ cf create-service-broker brooklyn broker-user broker-password https://broker-host:8080/
Creating service broker brooklyn as admin...
OK
</tt>

<tt>$ cf create-service-broker brooklyn broker-user broker-password https://broker-host:8080/

Creating service broker brooklyn as admin...

</tt>

<tt>$ cf service-access
Getting service access as admin...
broker: brooklyn
     service          plan       access   orgs
     MySQL Database   localhost  none

</tt>

<tt>$ cf service-access

Getting service access as admin...

broker: brooklyn

service plan access orgs

MySQL Database localhost none

</tt>

The blueprint we added to Brooklyn is now visible in Cloud Foundry. It’s access is none, but all that remains is to make it available in the marketplace.

<tt>$ cf enable-service-access "MySQL Database"
Enabling access to all plans of service MySQL Database for all orgs as admin...
OK

$ cf marketplace
Getting services from marketplace in org cloudsoft / space development as admin...
OK

    service          plans       description   
    MySQL Database   localhost   MySql is an open source relational database management system (RDBMS)
</tt>

<tt>$ cf enable-service-access "MySQL Database"

Enabling access to all plans of service MySQL Database for all orgs as admin...

$ cf marketplace

Getting services from marketplace in org cloudsoft / space development as admin...

service plans description

MySQL Database localhost MySql is an open source relational database management system (RDBMS)

</tt>

To create instances of this service, simply:

<tt>$ cf create-service "MySQL Database" localhost mysql
Creating service mysql in org cloudsoft / space development as admin...
OK
$ cf services
Getting services in org cloudsoft / space development as admin...
OK

    name    service           plan           bound   apps   
    mysql   MySQL Database   localhost      
</tt>

<tt>$ cf create-service "MySQL Database" localhost mysql

Creating service mysql in org cloudsoft / space development as admin...

$ cf services

Getting services in org cloudsoft / space development as admin...

name service plan bound apps

mysql MySQL Database localhost

</tt>

To deploy to new machines, all you have to do is change the location section of the blueprint. For instance, if we wanted a 5-node 8GB-each Riak cluster in Softlayer instead of a MySQL on localhost, we could add this to the Brooklyn catalog (replacing XXX with your SL username and access key):

<tt>brooklyn.catalog:
  id: riak
  version: 1.0
  iconUrl: classpath://riak.png
  description:  Riak
name: riak-cluster
location:
  jclouds:softlayer:
    identity: XXX
    credential: XXX
services:
- type: brooklyn.entity.basic.BasicApplication
  brooklyn.children:
  - type: brooklyn.entity.nosql.riak.RiakCluster
    initialSize: 5
    provisioning.properties:
      minRam: 8GB
</tt>

<tt>brooklyn.catalog:

id: riak

version: 1.0

iconUrl: classpath://riak.png

description: Riak

location:

jclouds:softlayer:

identity: XXX

credential: XXX

services:

- type: brooklyn.entity.basic.BasicApplication

brooklyn.children:

- type: brooklyn.entity.nosql.riak.RiakCluster

initialSize: 5

provisioning.properties:

minRam: 8GB

</tt>

Refresh the catalog, and Riak is now at your service.

Binding to Your Applications

You can now bind these service instances to your applications. If your application is already deployed, use:

<tt>$ cf bind-service my-app mysql
</tt>

1 2	<tt>$ cf bind-service my-app mysql </tt>

This will make the following environment variables available to your application runtime:

<tt>"VCAP_SERVICES": {
  "MySQL Database": [
   {
    "credentials": {
     "MySqlNode:tE5W": {
      "host.address": "127.0.0.1",
      "host.name": "127.0.0.1",
      "mysql.password": null,
      "mysql.port": 3306,

…

     }
    },
    "label": "MySQL Database",
    "name": "mysql",
    "plan": "localhost",
    "tags": []
   }
  ]
 }
</tt>

<tt>"VCAP_SERVICES": {

"MySQL Database": [

{

"credentials": {

"MySqlNode:tE5W": {

"host.address": "127.0.0.1",

"host.name": "127.0.0.1",

"mysql.password": null,

"mysql.port": 3306,

…

}

"label": "MySQL Database",

"name": "mysql",

"plan": "localhost",

"tags": []

}

]

}

</tt>

Or, you can add it to your manifest:

<tt>applications:
- name: my-app
  path: path/to/app.war
  memory: 512M
  services:
  - mysql
</tt>

<tt>applications:

- name: my-app

path: path/to/app.war

memory: 512M

services:

- mysql

</tt>

And then do a cf push like normal.

Pretty cool, eh? But what if we want to specify new services to be added to the marketplace? Well, we have a plugin just for you! Stay tuned!

Service Broker image via Cloud Foundry.

Feature image via Flickr Creative Commons.

CloudSoft is a sponsor of The New Stack.

Alex Williams is founder and editor in chief of The New Stack. He's a longtime technology journalist who did stints at TechCrunch, SiliconAngle and what is now known as ReadWrite. Alex has been a journalist since the late 1980s, starting...