IBM Opts for Humio’s Scalable Logging to Boost Struggling ELK Cloud Deployments
When it comes to logging, there’s a fine line to be tread between getting just what you need while saving on space and missing out on something important because you were trying to conserve storage. Traditional logging tools can be data hogs and modern distributed applications at scale only exacerbate the issue. Humio, however, offers index-free logging that the company says allows it to “log everything, miss nothing,” and we recently caught up with them at KubeCon + Cloud NativeCon in San Diego to talk about its recent partnership with IBM.
Mike Mallo, offering manager with IBM, says that IBM turned to Humio because customers were simply running into scale issues with the company’s Elasticsearch Logstash Kibana (ELK) stack.
“We brought in Humio as a partner for capturing logs for our Cloud Pak for Multicloud Management solution because we have clients that are using the ELK that we provide today in our solution and really outgrowing it from a scale perspective,” said Mallo. “From an ELK stack perspective, just running ELK highly available is an expense, because you’re basically having to take on all those operational costs. It’s the same thing with Prometheus. It only runs for about 24 hours worth of data and scale until you have to start backing up with another system.”
The partnership between IBM and Humio consists of a collaboration to improve observability in IBM’s Cloud Pak for Multicloud Management with the integration of Humio’s real-time and scalable log management platform, which Humio executive vice president Morton Gram says “aims to solve any pain points associated with highly distributed systems and modern applications.” For Gram, taking users off the tightrope balancing act of storage and scale versus comprehensive logging opens up new possibilities.
Humio offers a service to store an unlimited amount of unstructured log information, so it can be easily searched. The Humio engine was built to work with up to terabytes of log data with no latency, according to the company. Beyond its ability to log without concern for storage, Humio also offers real-time observability, which Mallo focuses on as the only way to see into ephemeral cloud native deployments.
“Once you start removing the constraints, whether that is complexity or hardware costs or license costs, and customers can just start ingesting without having to think about these parameters, their behavior changes completely. Suddenly they don’t spend time on filtering their logs, they just start logging everything. When they start doing that, they get a different visibility into their environments,” said Gram. “The big challenge is actually your incidents. They never happen where you’re looking, they happen where you’re not looking. Your time to recover is based on what data you have available. If you don’t have the data available, your recovery time is going to be so much longer.”
“Logs are really the only way you’ll see it in production when you’re in cloud native because the container may only last for a minute and then it gets killed because there’s a problem and it drops some important logs,” said Mallo. “If you can’t go back and look at those over time, you don’t really see what happened. The ephemeral nature of the container means logs is really your observability aspect, your metrics and everything’s probably gone at that point.”
For Gram, however, real-time logs provide the ability for real-time alerts, as well as extracting value that was previously locked away in business intelligence (BI) tools.
“If you look in your application logs, you have so much business value that’s actually in your application logs. If you really want to do real-time analytics to change the way you run your processes in a business, the best way to go is through your logs,” said Gram. “If you look at the flexibility that you have in a platform like Humio to ask anything, it’s actually an easy way for a business to get a really good dashboard of what’s going on in real-time. How many times have you had a CEO ask a question and you can’t get it out of his BI tool because it’s not predefined?”
As part of the partnership, Humio is available in the IBM Cloud Pak for Multicloud Management as a service or on-premise, running securely behind a client’s firewall on Red Hat OpenShift, as well as under a single contract.
In addition to IBM, Humio services are also used by SpareBank and Netlify:
TNS Editorial and Marketing Director Libby Clark contributed to this post.
Humio and OpenShift are sponsors of The New Stack.
Feature image by David Mark from Pixabay.
