New Relic Eases Log Management into Observability
New Relic sponsored this post. TNS owner Insight Partners is an investor in New Relic.
New Relic has expanded its flagship New Relic One platform to include logging capabilities as well, setting the stage for tracing the performance and digging out potential trouble spots of a distributed application running across multiple servers, databases, and other supporting software.
At the company’s FutureStack event in New York last month, the company unveiled a new log aggregation and management tool for New Relic One, called New Relic Logs. The service has been released for general availability (GA) and the company has already opened public beta testing for the next version.
The idea behind New Relic Logs is to make it easier for New Relic One users to incorporate log data into overall system observability, said Julian Giuca, New Relic product general manager and principal software engineer. It would allow them to identify meaningful trends in their log data, correlating the log data to other sources of performance data and events.
Developers and DevOps “want to get to the ‘why’ faster: why is my server falling over? Why am I being woken up at 2 A.M. in the morning,” Giuca said. “So we built a tool that is from us talking with our customers for the past year and a half about what their pains are.” Customers wanted something fast, scalable, connected, and easy-to-manage, he said. They didn’t want to manage indexes, for instance.
Pretty much every piece of software in the data center or cloud creates logs of its performance: firewalls, servers, databases, and load balancers. They all write events to a log file, and the information they contain can be valuable for troubleshooting. Log data aggregation could be particularly useful when dealing with microservices, were applications can be composed of hundreds of microservices, and were built from multiple programming languages. Each one may have a different logging framework that would need to be integrated into an observability platform.
New Relic Logs correlates log messages with error reporting and event data from sources such as New Relic APM. In this example, a user can correlate log data to a stack trace in APM — without being forced to shift context or to move between multiple tools.
Why enter the market of log management software and services, which is already served by technologies such as Logstash, Datadog, Fluentd, and Fluent Bit? There are a number of drawbacks to using these best-of-breed packages, New Relic asserts. They can be difficult to learn how to use, difficult to customize, often lag in performance, and take time to integrate with other monitoring tools.
In this light, New Relic Logs offers a number of advantages. Number one, it is fast, as it stores data on the company’s legendarily speedy NRDB (New Relic Database). It also works well with other logging tools, such as Sumo Logic and AWS CloudWatch. A single line configuration can split the data so it goes to both Sumo Logic, for instance, as well as to New Relic One. All the data generated from other log management software is formatted into JSON payloads, so they can be easily programmed against.
The best part though is the integration with New Relic One, which can offer a lot more context about each and every transaction. New Relic One assigns a unique identifier that gets attached to every transaction, and so each transaction can be traced through the various application executing it.
“Every function that gets called, New Relic pays attention to the timing of that ID. And every logline that gets called New Relic is paying attention to that ID. So you get that correlation. You can see all the functions being evoked, that whole code stack across multiple servers and applications,” Giuca said.
