Data / Monitoring / Open Source

Grafana Adds Elasticsearch to its ‘Big Tent’ of Data Sources

10 Mar 2021 6:00am, by

Grafana’s new plugin for Elasticsearch further adds to the cloud native observability tool’s growing roster of data sources under what Grafana Labs describes as a “big tent” of supported platforms.

Before the Elasticsearch plugin became available, many Grafana users had “hacked together solutions of their own,” to work with the Elasticsearch scalable search and analytics engine, Raj Dutt, CEO and co-founder at Grafana Labs, told The New Stack. “With the help of the Grafana and Elastic open source communities, the new plugin will make that process simpler,” he said.

Grafana is a multiplatform open source analytics and interactive visualization web application, widely used for many cloud native computing deployments.

Additional benefits of the partnership include more functionality such as aggregations, broader query language support, and support for space-saving constructs such as rollups and other features.

“Elasticsearch is one of the most popular data platforms that our users currently visualize in their Grafana deployments, so collaborating with them on a native plugin is the best way to make sure our customers have a smooth, positive experience combining the two tools,” Dutt said.

The addition of Elasticsearch to the growing roster of Grafana plugins also allows for organizations to adopt several different observability tools from service providers.

“It’s really easy for them — we have nearly a hundred different plug-ins that range from free to paid (enterprise plugins),” Dutt said.

Under what Grafana describes as its “big tent,” Dutt said Grafana Labs’ main philosophy is for users “to have full control over their observability strategy, extending to the specific tools that they prefer to use,” Dutt said.

“Our role at Grafana Labs is to allow them to bring them together and understand all their data, no matter where it lives,” Dutt said.

In addition to its collaboration to develop Elasticsearch, Dutt noted now Grafana Labs has partnered with a number of other organizations, including Amazon Web Services, Redis Labs, Oracle, Azure and Google. The idea behind the development of the numerous plugins is “to take them even further so they can render and interact with data from those sources more efficiently,” Dutt said.

It is now possible, for example, for a user to combine Elasticsearch, Splunk and Loki “into a single dashboard and even run calculations across them and this is without moving the underlying data,” he said.

Tom Wilkie, vice president of product at Grafana Labs, a Prometheus maintainer and a Loki and Cortex co-creator, noted how, with Grafana, it is also now possible to visualize data from Prometheus, Loki and “almost 100 others data sources in a single dashboard and panel,” in addition to Elasticsearch.

“This is pretty unique to Grafana,” Wilkie said. “Our ‘big tent’ approach is about not playing favorites — we treat all data sources equally, and invest in Elasticsearch as well as Loki, Splunk, Google Stackdriver Logs, etc.”

Elasticsearch, in many respects, is the progenitor project for Grafana from which Grafana was forked in 2013. As Dutt noted in a blog post, Grafana Labs co-founder Torkel Ödegaard created Grafana as a fork of Kibana 3, which is the frontend of the Elasticsearch platform.

Ödegaard’s intention was to create a “time series- and graph-focused dashboarding tool,” Dutt wrote.

“While Elastic decided that Kibana’s mission would be visualizing data only from Elasticsearch, Grafana would be built to support visualization of multiple data sources. Since then, that list of data sources has expanded to include everything from Graphite and Prometheus to GitLab and Splunk — and yes, Elasticsearch,” Dutt said.

A newsletter digest of the week’s most important stories & analyses.