What news from AWS re:Invent last week will have the most impact on you?
Amazon Q, an AI chatbot for explaining how AWS works.
Super-fast S3 Express storage.
New Graviton 4 processor instances.
Emily Freeman leaving AWS.
I don't use AWS, so none of this will affect me.
Kubernetes / Observability

Kubernetes Troubleshooting Starts with DNA-like Data Swirling

The Sosivio troubleshooting platform looks at a series of actions in a Kubernetes cluster as a “strand” of DNA, which can be used to predict what the full DNA sequence will look like.
Aug 31st, 2022 10:00am by and
Featued image for: Kubernetes Troubleshooting Starts with DNA-like Data Swirling

Brandon Landry
Brandon Landry is spearheading Sosivio’s Business Development and Partnership initiatives while supporting marketing campaigns. Brandon’s background paving the way in the automotive-connected car industry positions him perfectly to understand maturing markets and identify the root cause of customer pain points while still having a strong grasp on technology.

Sifting through the sheer volume of information for Kubernetes troubleshooting, even with the help of today’s observability tools, is still like finding a needle in a haystack. Troubleshooting and optimizing Kubernetes’ environment typically looks like this: collect, ship, and store large amounts of metrics, logs, and traces and rely on an expert or outside counsel to sift through the waves of data to analyze for a root cause of an issue.

Traditional monitoring and observability tools using open source data collectors are inherently flawed as the data feeding them is incomplete. Failures generally consist of many individual events (more on this below), building on each other to make up a larger failure sequence.

Without all the links in the failure sequence, the true root cause is allusive. Not to mention, these tools are reactive, prompting the user if there is an issue, after something went wrong. This leaves teams spending excessive amounts of time doing the troubleshooting work or leaning on expensive outside expertise to find the root cause of failures; neither of which are ideal. How do we break through this legacy approach and evolve along with modern technology, ensuring that we can successfully get insights into our environments? Data Swirling.

Stephen Thorn
Stephen Thorn is the technical frontline for Sosivio hosting multiple webinars, office hours, and technical sessions. Stephen previously comes from the Open Source space now applying his experience to Kubernetes troubleshooting. Stephen is the primary driver for customer success ensuring customers are maximizing their return by utilizing all of the Sosivio features aiding organizations with their cloud native migration.

Data Swirling is a methodology, developed by Sosivio, to analyze mountains of data from multiple layers of the stack, on-the-fly, in real-time, without having to send anything outside the cluster. This requires ultra-granular and ultra-accurate data. Sosivio recognized the current challenges with today’s data collectors and opted to build custom data collectors, optimized to collect very granular metrics and information from the entire infrastructure stack (kernel signals, OS logs, process signals, application logs, container runtime events, network traffic, Kubernetes events) that in turn fuel the ML/AI engines.

As one event in a sequence occurs in a Kubernetes cluster, a “strand” of the DNA is filled out. Looking at a DNA sequence being filled out, one can start to predict what the full DNA sequence will look like.

Sosivio uses a “Data Swirling” machine learning engine for each phase, which passively collects data, compresses and translates everything to a unified language, correlating the data to form a clear picture of what is happening inside the cluster, and then recommends a resolution that can be applied to fix the failure.

Sosivio Cluster Overview showing A.I. Predicted Failures Graph, Cluster Health Scores, and Application Profiling.

When there is an issue or failure in Kubernetes, it is a combination of correlated singular events. The combination of these events can be thought of similarly to a DNA sequence. As one event in a sequence occurs in a Kubernetes cluster, a “strand” of the DNA is filled out. Looking at a DNA sequence being filled out, one can start to predict what the full DNA sequence will look like. As these singular events build toward a failure sequence, Sosivio’s ML prediction engine assigns a context and severity and continuously displays impending issues before they materialize into a catastrophe.

Sosivio’s advantage of seeing these events as they unfold in real time allows the prediction engine to detect what is going to happen in a failure sequence. Sosivio only presents relevant data and actionable insights to avoid compounding alert fatigue brought on by today’s tools. Data is processed and analyzed 100% in memory, removing the added delays of sending and receiving data to and from disk. And because data is lightweight, Sosivio can analyze a massive amount of data without being a burden on resources.

The Sosivio platform runs as Yet-Another-Application on your Kubernetes clusters which means a non-intrusive solution that works in completely disconnected/Air-Gapped environments. Sosivio taps into signals and data from all layers of the environment (OS, Network, Kernel, K8s, Apps and more) and can be run on any Cloud Platform (AWS, GCP, Azure, AKS, EKS, etc…) or in On-Premise deployments.

Sosivio’s always free Community version includes Real-time Metrics, Cluster Health Checks, and Application Insights.

Sosivio will also be at Kubecon North America in Detroit, MI from Oct. 25-28, 2022. In addition to their exhibition booth, Sosivio is holding a Breakfast Bar Social from 8-9 a.m. each day prior to the event at The Apparatus Room across from Huntington Place. Each morning, Sosivio’s CTO Liran Cohen will be partaking in a mini-roundtable light discussion to start the day with industry experts including Head of Global Engineering from an F100 financial service provider. Learn from elite Kubernetes experts for free! Enjoy coffee, bagels, donuts and more while also networking with the executive team at Sosivio. RSVP here.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Real.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.