Joyent product manager Tim Gross contributed this tutorial, in conjunction with the Linux Foundation’s Node Interactive, taking place this week in Portland, Oregon.
Tim is a product manager for Joyent, providers of the Triton Elastic Container Service. Tim previously ran Ops at DramaFever, where he and his scrappy team ran Docker in production to serve a few million fans their daily dose of dramas, documentaries, and gross-out horror movies. In a previous life, Tim was an architect (buildings, not software). He took the leap into programming and Operations after he discovered he could automate away almost everything boring in his life.
Today I’m going to put all these pieces together in a multi-tier application that can serve as a blueprint for a microservices architecture. Follow along with the code on Github.
The application we’re going to put together is Touchbase, a Node.js application. We’ll need the following pieces.
- Touchbase, the Node.js app at the center of the stack
- Nginx, acting as a load balancer for Touchbase nodes
- Couchbase, for the data tier
- Consul, acting as a discovery service
- Containerbuddy, to help with service discovery
- CloudFlare-watcher, to update DNS
- CloudFlare DNS, to make our site accessible by domain name on the Internet
- Triton, our container-native infrastructure platform
This stack could be used for any microservices application and the individual components can be swapped out easily. Prefer HAProxy to Nginx? No problem – just update the
docker-compose.yml file with the image you want to use.
The Touchbase Node.js application was written by Couchbase Labs as a demonstration of Couchbase 4.0’s new N1QL query features. It wasn’t specially designed for a container-native world, so we’re using Containerbuddy to allow it to fulfill our requirements for service discovery.
Touchbase uses Couchbase as its data layer. We can have it serve requests directly, but we’re going to put the Nginx load balancer in front of it because we want Nginx’s ability to perform zero-downtime configuration reloads. (Also, in a production project we might have multiple applications behind Nginx.) We’re going to use a fork of Touchbase that eliminates the requirement to configure SendGrid because setting up transactional email services is beyond the scope of this article.
The Touchbase service’s Containerbuddy has an
onChange handler that calls out to
consul-template to write out a new
config.json file based on a template that stored in Consul. Unfortunately, Touchbase does not support a graceful reload, so in order to give Touchbase an initial configuration with a Couchbase cluster IP we’ll need a pre-start script that does so. Having the option to run the
onChange handler or another startup script before forking the main application would be a great feature to add to Containerbuddy and I’ll circle back on that in an upcoming post.
The Nginx virtual host config has an
upstream directive to run a least-conns load balancer for the back-end Touchbase application nodes. When Touchbase nodes come online, they’ll register themselves with Consul.
Just like in our original Containerbuddy example project, the Nginx service’s Containerbuddy has an
onChange handler that calls out to
consul-template to write out a new virtual host configuration file based a template that we’ve stored in Consul. It then fires an
nginx -s reload signal to Nginx, which causes it to gracefully reload its configuration.
Couchbase is a clustered NoSQL database. We’re going to use the blueprint for clustered Couchbase in containers written by my Joyent colleague Casey Bisson. It uses the
triton-couchbase repo for Couchbase 4.0 to get access to the new N1QL feature.
When the first Couchbase node starts, we use
docker exec to bootstrap the cluster and register the first node with Consul for discovery. We’ll then run the appropriate REST API calls to create Couchbase buckets and indexes for our application. At this point, we can add new nodes via
docker-compose scale and those nodes will pick up a Couchbase cluster IP from Consul. At that point, we hand off to Couchbase’s own self-clustering.
Just like in our dynamic DNS project earlier this week, the
cloudflare container will have a Containerbuddy
onChange handler that updates CloudFlare via their API. The handler is a bash script that queries the CloudFlare API for existing A records, and then diffs these against the IP addresses known to Consul. If there’s a change, we add the new records first and then remove any stale records.
Running the Example
You can run this entire stack using the
start.sh script found at the top of the Github repo. You’ll need a CloudFlare account and a domain for which you’ve delegated DNS to CloudFlare, but if you’d like to skip that part you can simply comment out
Once you’re ready:
- Get a Joyent account.
- Install the Docker Toolbox (including
docker-compose) on your laptop or other environments, as well as the Joyent CloudAPI CLI tools (including the
- Configure Docker and Docker Compose for use with Joyent.
- Have your CloudFlare API key handy.
At this point you can run the example on Triton:
./start.sh env # here you'll be asked to fill in the .env file ./start.sh
or in your local Docker environment (note that you may need to increase the memory available to your docker-machine VM to run the full-scale cluster):
./start.sh env ./start.sh -f docker-compose-local.yml
.env file that’s created will need to be filled in with the values described below:
CF_API_KEY=<your CloudFlare API key> CF_AUTH_EMAIL=<the email address associated with your CloudFlare account> CF_ROOT_DOMAIN=<the root domain you want to manage. ex. example.com> SERVICE=nginx <the name of the service you want to monitor> RECORD=<the A-record you want to manage. ex. my.example.com> TTL=600 <the DNS TTL you want> CB_USER=<the administrative user you want for your Couchbase cluster> CB_PASSWORD=<the password you want for that Couchbase user>
As the start script runs, it will launch the Consul web UI and the Couchbase web UI. Once Nginx is running, it will launch the login page for the Touchbase site. At this point there is only one Couchbase node, one application server and one Nginx server and you will see the message:
Touchbase cluster is launched! Try scaling it up by running: ./start scale
If you do so you’ll be running
docker-compose scale operations that add 2 more Couchbase and Touchbase nodes and 1 more Nginx node. You can watch as nodes become live by checking out the Consul and Couchbase web UIs.
The stack we’ve built here highlights the advantages of Dockerizing this application. We have an easy, repeatable deployment that we could test locally and then push the same stack to production. We have the automatic discovery and configuration that makes that deployment possible. We have easy horizontal scaling, with fine-grained control over scale of each tier. And we have global discovery, thanks to our integration of CloudFlare.
We’ve used Containerbuddy as an example of the minimal shims that are required to make arbitrary applications container-native. And now that we’ve seen a production-ready multi-tier application assembled on Triton, we can see that container-native service discovery can be agnostic to any particular scheduling framework. This made it easy to connect components that were not designed to be containerized.
Deploying on Triton made all this even easier. In an environment where application containers have their own NIC(s), as they do on Triton, we can rely on application containers updating the discovery service without a heavyweight scheduler. That means you can use simple tools like Docker Compose to deploy and link containers without any additional software. With Triton’s container-native infrastructure there’s no need to provision virtual machines, and Triton charges per container so it’s easy to keep track of how your costs will scale with your app. You can deploy on Triton in the Joyent public cloud or in your own data center (it’s open source!). Just configure Docker and press the
Docker and Joyent are sponsors of The New Stack.
Feature Image via Pixabay.