How FluentD Can Help Monitor Microservice Architectures Through Unified Logging

On this latest episode of The New Stack Makers podcast, we learn all about FluentD, the latest open source project to be taken under the wing of the Cloud Native Computing Foundation. Treasure Data Open Source Developer Eduardo Silva and Treasure Data CTO Kazuki Ohta sat down with TNS managing editor Joab Jackson and co-host SolarWinds Senior Director of Technology Strategy Lee Calcote to learn more about this microservices-oriented logging software, which is shepherded by Treasure Data.

How FluentD Can Help Monitor Microservice Architectures Through Unified Logging

“Collecting data at scale reliably and easily is really, really hard. What we found was, we made Fluentd as a lightweight and flexible agent where the agent had a lot of plugin mechanisms. After five years, we have around 600 plugins available for all sources and outputs,” said Ohta.

Fluentd features a Ruby gem based plugin mechanism. Rather than having to write lines upon lines of code, DevOps team members can write a Ruby gem, install it to their local Fluentd environment, and start using it.  “We have four types of plugins available. Input, output, parser, and filter. For inputs it’s a lot of protocols like net flow, receiving data from switches, HTTP, or even Twitter. Output type is a lot of data systems. Parser is known formats like JSON or inputting custom bin formats. Filter is how you filter data. DevOps people will write plugins for a specific system which is newly released or that already exists,” said Ohta.

Ultimately, becoming a part of the CNCF has brought Fluentd and Treasure Data much needed funding for improving documentation, access to the CNCF Community Cluster so that its team can start testing and measuring different use cases, and more.

“Our user typically has thousands of servers. The moment you need Fluentd is the moment you have a lot of servers.”

Cisco sponsored this podcast.