At the end of 2013, Docker’s Jérôme Petazzoni sparked a Google Groups discussion to improve capturing more log sources and the way that log entries are consumed. “Ideally, we want to capture more log sources (syslog and regular files come to mind), and better ways to consume logs,” he wrote. The discussion then moved GitHub, with more than 25,000 interested in how best to log activity within Docker environments.
As Docker becomes more central to more organizations’ operations, so too will these organizations’ need more information about the performance of these containers. With this in mind, Docker has since been working with partners to fill out the logging ecosystem, and on Tuesday highlighted nine companies in its Ecosystem Technology Partner (ETP) program that have been specializing in logging technologies.
“We have a vibrant ecosystem out there doing a variety things in different realms, whether it’s monitoring, logging, security, networking. There’s a great breadth of smaller companies, emerging startups as well as larger organizations building technologies around Docker around the complete lifecycle. One of the focuses of the ETP program is to shine a light on companies that are using Docker and that it’s working in a Docker-native way,” said David Messina, vice president of enterprise marketing, in a statement.
While some people consider monitoring and logging two sides of the same coin, there are distinct use cases for each, Messina said.
“One of the things we’re excited about in the logging realm is that the category of companies there is incredibly broad,” he said. “From the research we’ve seen out there, the lifetime of an individual container is generally far more dynamic than that of a virtual machine, so having information in detail at the highest resolution is incredibly important.”
The ETP program allows customers to understand what the integrations are and what the Docker-specific capabilities are, Messina said.
Some of the partners, such as Amazon Web Services, with its Amazon CloudWatch service, have developed logging drivers, a key feature in the November release of Docker 1.9. Others there have developed native capabilities for predictive analytics and to relay the logging streams to third-party solutions. Among their approaches:
- Rapid7, which bought out Logentries in October, uses four Docker APIs and a the Rapid7 Logentries container to automatically collect container-level logs and metrics in near real time. Logentries previously told The New Stack it aimed to go beyond troubleshooting to provide insight for analytical methods and business metrics.
- Sematext works with SPM, an alerting tool, and Logsene, a centralized log-management tool. Logs are shipped to Logsene, where they’re made searchable while metrics and events are sent to SPM. Both are integrated into a single UI. It’s easily deployed with Tutum, which Docker acquired in October.
- Treasure Data’s Fluentd logging driver for Docker enables archiving logs into S3, sending logs to Elasticsearch or to data-processing backends and streaming logs into monitoring services.
The company in June announced the ETP program and its monitoring partners, which included AppDynamics, Datadog, New Relic, Scout, SignalFx and Sysdig. It plans further announcements in areas such as configuration management, security and others, Messina said.
Docker, Datadog, New Relic, SignalFX are sponsors of The New Stack.
Feature Image: “Skeletons from engraved title page vignette of Anatomy improv’d and illustrated (1723)” by University of Liverpool, licensed under CC BY-SA 2.0.
The New Stack is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker, Sysdig.