Merging Logs and Metrics with Grafana Labs’ Loki 1.0 Launch
Portworx sponsored The New Stack’s coverage of Kubecon+CloudNativeCon North America 2019.
Grafana Labs has released version 1.0 of the open source Loki log aggregation software at this year’s KubeCon+CloudNativeCon, calling the tool generally available and ready for production use. While Loki was first released at KubeCon 2018, Tom Wilkie, vice president of product at Grafana Labs, said that the release was quite early on and that it has come a long way in the year since.
“It was a very early announcement, very much a prototype. We’ve spent the last 12 months really iterating on it very, very quickly, with very tight release cycles, continuously deploying it and dogfooding it internally,” said Wilkie in an interview with The New Stack. “We were getting a lot of feedback from our users that they love it, that they’re using it in dev and test and staging, but there’s a lot of apprehension about pre-1.0 products.”
In the time since Loki was first released, Grafana Labs has been running it in production for the last eight months and the project has seen more than 1,000 contributions from nearly 200 contributors, and Wilkie says that they have put out at least five minor releases along the way. Beyond signaling Loki’s readiness for production use, Wilkie also points to the addition of LogQL, a “Prometheus-style query language,” as something the team wanted to highlight as part of a 1.0 release.
“It’s early days still for LogQL. We’ve built some basic functionality to start extracting metrics from the logs and be able to combine them with metrics from Prometheus. We tried to build the language so it’s almost as identical to PromQL as it can be,” said Wilkie, explaining that LogQL can be used to bring in and expose log data from legacy systems that predate Prometheus.
Grafana Labs is also the company behind the open source metrics, analytics, and monitoring visualization tool Grafana, and, in many ways, metrics and logging are two different sides of the same coin — metrics can tell you that a service is running slowly, for example, while logging can help troubleshoot why that service is running slowly. Loki differs from other logging tools in that it doesn’t index the full text of the logs, but rather indexes only the metadata, while using the same service discovery technology as Prometheus to operate in a Kubernetes environment. With Loki, metrics and logs are joined together, explained Wilkie, as Loki allows users to instantly switch contexts between metrics and logs.
“Because we’ve got this systematically consistent metadata between your metrics and your logs, we can actually automate that process and we can automatically switch from any given metrics-based query to a logs query that shows you the relevant logs for the services you were looking at. It’s really powerful. This helps drive a huge reduction in the mean time to recovery for incidents.”
In addition to the release of the open source Loki 1.0, Grafan Labs also launched Grafana Cloud, a highly-scalable, hosted Loki service with usage-based pricing that includes support and training. As for what’s next for Grafana Labs, Wilkie says that there’s a final realm that the company is planning to add to its offerings on metrics and logs — tracing.
“Distributed tracing is commonly referred to as the third pillar of observability. We started out on our logging journey at Grafana Labs a year ago and at KubeCon last week we started out on our tracing journey,” said Wilkie. “We’ve added some integrations for distributed tracing into Grafana, the ability to visualize traces from things like Jaeger and Zipkin, and over the next year I hope to be able to make some exciting announcements about various tracing integrations and projects from Grafana Labs.”
The Cloud Native Computing Foundation and KubeCon+CloudNativeCon are sponsors of The New Stack.
