Weaveworks Brings Horizontal Scaling to Prometheus
Initially developed by a team of SoundCloud engineers including Julius Volz and Brian Brazil, Prometheus has become one of the de-facto software for a monitoring microservices-based architectures. On this new episode of The New Stack Makers podcast, Weaveworks Chief Operating Officer Mathew Lodge discusses how Weaveworks new multitenant, horizontally scalable Prometheus-as-a-Service project, Weavecortex, aims to change how developers work with their data. He spoke with TNS founder Alex Williams and co-host Lee Calcote, SolarWinds cloud technology lead, at Kubecon 2016.
Weaveworks is Bringing Horizontal Scaling to Prometheus
Traditionally, monitoring as a whole has focused on servers and VMs. However, with the rise of containers and the shift to a more cloud-native development approach, many teams found themselves struggling to monitor their stack. “It didn’t map very well to highly containerized environments. A lot of monitoring tools couldn’t keep track of where a container was. The game you’d have to play is, ‘find the container,’” said Lodge.
One challenge faced by those seeking to implement Prometheus was that it stores all its data was stored on a single disk. “That’s what we sought to solve with cortex. It’s a horizontally scalable version of Prometheus. You can add instances of cortex and it will spread out load, queries, and data collection. It turns it into a horizontally scalable multitenant solution,” said Lodge.
“What we did was paralyze and scale out queries, so they will still execute quickly even though that data is spread out across multiple shards,” Lodge noted.
This multidimensional approach to data is where Prometheus shines, particularly when developers have need of a fine-grained picture of their system. “In a dynamic environment, it’s really useful to be able to correlate all those things together. Being able to move up and down that level of granularity is one of the advantages of that multi-dimensional query model,” Lodge explained.
Cisco is a sponsor of The New Stack.
