Technology / Tutorials / /

How To Detect, Map and Monitor Docker Containers with Weave Scope from Weaveworks

9 Jun 2015 9:35am, by and

If you use Docker, you probably already know about Weave’s Docker network. Weave creates a virtual, software-defined network, or SDN, across every Docker host in your infrastructure. Each container gets its own IP, allowing you to design your application topology without changing your application’s behavior. Weave Run, a set of companion tools for service discovery and address management, also contributes to an easier development experience.

Weave Scope, a new product from Weaveworks, continues in the company’s tradition of making tools that reduce complexity. Scope automatically detects and monitors every host, container and process in your infrastructure, and builds a map of their communication. Scope presents an always-up-to-date view of your infrastructure in a modern web interface, allowing you to deeply understand, monitor and explore your complex application topology. Scope does this without requiring a single change to your code or configuration, and without requiring you to make declarations about your infrastructure that will immediately become out-of-date and stale.

In this tutorial we’ll demonstrate how to deploy Scope alongside a typical three-tier web application stack, consisting of a pool of data services, a set of custom application servers and a load-balancing layer. You’ll learn how Scope works out-of-the-box with no configuration, and how it can immediately start returning useful intelligence.

While Weave Scope works brilliantly with Docker and the Weave network, neither is required. Scope can be deployed in any infrastructure, and works well in all cloud and bare-metal environments.

Getting Started

We’ll get started by deploying an example application using Docker Compose. For the demo we’ll work on a single host, but remember that Weave works equally well on multiple hosts, or even across data centers and cloud providers.

First, let’s install Docker and Docker Compose:

$ wget -qO- https://get.docker.com/ | sh
$ sudo curl -L https://github.com/docker/compose/releases/download/1.2.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose

Now, let’s use Docker Compose to launch all of the components:

$ wget -O docker-compose.yml http://git.io/scope-compose
$ docker-compose up -d

Take a look at the output of Docker ps, to make sure that everything is running:

CONTAINER ID  IMAGE            PORTS                 NAMES
fe41c10a63ca  tns_lb1:latest   0.0.0.0:8001->80/tcp  tns_lb1_1
c94005d87115  tns_lb2:latest   0.0.0.0:8002->80/tcp  tns_lb2_1
8c15a1325094  tns_app1:latest  8080/tcp              tns_app1_1
645386356a2e  tns_app2:latest  8080/tcp              tns_app2_1
e34ccea042fd  tns_db3:latest   9000/tcp              tns_db3_1
c0d53d1327b4  tns_db2:latest   9000/tcp              tns_db2_1
0a920e17818a  tns_db1:latest   9000/tcp              tns_db1_1

Verify everything is working by curling one of the tns_lb instances. By default, they’re listening on ports 8001 and 8002:

$ curl localhost:8001  # on a Mac, try: curl `boot2docker ip`:8001
lb-6d5b2352f76d4a807423ce847b80f060 via http://app1:8080
app-60fbe0a31aee9526385d8e5b44d46afb via http://db2:9000
db-e68d33ceeddbb77f4e36a447513367e8 OK

Great! In just a few commands, we’ve deployed a three-tier application stack. Now, we want to get a picture of our infrastructure, so let’s install and launch Weave Scope:

$ wget -O scope git.io/scope
$ chmod a+x ./scope
$ ./scope launch

Now, you can point your web browser to localhost:4040. (On a Mac, that might be `boot2docker ip’:4040.)

Seeing Your Infrastructure

Now that Scope is up and running, it’s time to see your infrastructure. The Scope user interface is loaded in your browser and gives you an immediate overview of your network.

weaveworks1

A simple map of your containers

Each circle of this map represents a container in your network. The circles that are connected by a line are containers communicating with each other. Use your mouse to explore the network. When you hover over a container or a connection Scope highlights the connected containers and helps you understand the topology quicker. This is especially helpful for infrastructures with lots of containers.

Let’s say we’re looking for more information about one of the database (db) containers. Click on a container with the label tns_db1_1 to find more information about it.

weaveworks2

Details and metrics of the DB container

The panel on the right shows basic information about the container, like the image and process name, as well as network metrics, like ingress/egress byte rates and number of TCP connections.

More Advanced Use Cases

Scope works great in topologies of any size. When you get too many containers on the screen at once, you can use Scope’s grouping features to make things easier to understand. For example, in the containers view, you can click the grouping button to merge all the containers that are running the same container image together.

weaveworks3

Grouped container view

In our stack, that means all of the db, app, and lb instances get merged together into single nodes, even when they exist across hosts. The result is a logical, rather than physical, lens on the infrastructure — something that application developers should intuitively understand. And detailed statistics are merged, too.

These features are great for application developers, but those with a more operations-oriented background will feel right at home in the hosts view. Here, we see the physical layout of the infrastructure, as you might find in a tool like Ganglia or Squid.

weaveworks4

The Hosts view

A click on applications changes the map again, showing all processes communicating in your network. This is the most granular view of Scope.

weaveworks5

The Applications view

Scope collects information in a flexible data model, so it’s possible to visualize nearly any possible transformation of your network.

Taking Integration a Step Further: The API

Now that you have seen how to easy it was to get visibility on an application network, it is time to integrate it into your system. There are multiple ways for you to set up Weave Scope for your environment. It is open to connect with other platforms and data sources. Out-of-the-box the user interface gives you a good overview of your application network. Use it as a starting point to troubleshoot problems or simply explore the topology. Beyond that, you can directly query its API and get a feed of container connections and metrics. This feed can act as an input to your own monitoring solution and provide topology information.

Let’s use the API to get to the same information about the database container from above (remember that this works on any host that runs Scope):

First, we see which topologies are available:

GET http://localhost:4040/api/topology
[
   {name: "Containers", url: "/api/topology/containers"},
   ...
]

We want the container topology. To get all containers we query:

GET http://localhost:4040/api/topology/containers
{
   nodes: {
      “123...”: {
         “adjacency”: [‘456...“],
         ...
      }
}

Here we see that the container topology is represented as a node collection with adjacency lists. In the example we see that node “123…” has a connection to node “456…”. Iterating over all nodes allows you to derive your own topology.

Let’s assume that node “123…” is already the database container we are looking for. To get more information on that container we can query:

GET http://localhost:4040/api/topology/containers/123…

Try it out to see what information is available in the response.

More Visibility

Use Weave Scope now to automatically generate maps and feeds to enable visibility for your Docker apps. It is especially easy to add when you are already using Weave Net to set up your Docker cluster. There is a collection of helpful guides on how to get started with Weave.

Weave Scope is open source and continuously being developed. The code is available on GitHub and is easy to set up if you like to contribute. The probes and the backend are written in Go. The user interface is a React single-page application. Follow Weaveworks on Twitter to stay in the know about latest releases. You can expect more container metrics and features to be added soon.

Weaveworks is a sponsor of The New Stack.

Feature image via Flickr Creative Commons.


A digest of the week’s most important stories & analyses.

View / Add Comments