Open Source Mizu Unblocks View into Kubernetes API Traffic
The latest open source project from microservice testing platform UP9 is called Mizu, a utility that enables developers to see all their Kubernetes API traffic in a local web application.
You can view all the API communication between microservices in Kubernetes just as you would use Google ChromeDevTools to view the traffic of web apps.
“In the past, troubleshooting through traffic inspection was table stakes, it was easy. With Kubernetes, it’s not easy anymore,” said Alon Girmonsky, Up9 founder and CEO.
“Today, when a developer experiences a problem, and they want to troubleshoot, there’s no traffic, for many reasons — traffic is distributed, traffic requires permissions, traffic is under layers of encryption and new protocols that need deciphering before understanding kind of what’s going on.
“So traffic is something that’s very hard to come by. And if you look at other tools in the industry, they all require code instrumentation. Right? You have to install an SDK, an agent or change your code in order to be able, when the time comes, to actually look inside the communication to figure out what’s going on.”
In contrast, a developer can download the 40-megabyte utility Mizu and view the traffic.
Mizu is an observability tool that injects a container that performs a tcpdump-like operation at the node level of a Kubernetes cluster. This operation can be performed on-demand via a CLI and used without preparation. After use, the container is removed.
While running, Mizu passively observes traffic at the network level and is not a proxy. Mizu does not require code instrumentation.
The CLI is built in Golang, and can be downloaded and run without installation. Mizu uses kubectl, so can run on any node through which kubectl is configured.
Mizu is available to download as an executable binary or as source code directly from its GitHub repository.
Using regular expressions, you can observe all traffic or specific pods. While Mizu runs, pods with names that match the regular expression will be observed and their traffic logged.
The observability tool supports HTTP/1.x, HTTP/2, AMQP, Apache Kafka and Redis protocols. A Kubernetes server version of 1.16.0 or higher is required.
Digging into Traffic
It’s not enough for a modern cloud developer who is working with APIs to simply see that TCP packets were delivered.“
Developers will open the browser and see the actual traffic that goes inside the Kubernetes cluster,” Girmonsky said. They want to dig deeper: They know Kafa, gRPC, REST, “the protocols that are carried on top of the entire network stack.”
“If I were a web developer, I would use Chrome DevTools to inspect and see the traffic. But in Kubernetes, you don’t have Chrome DevTools; you don’t have even tcpdump of the past, you had to have permissions to run it. And then it would give you only the TCP packets, which … everything is kind of obfuscated, to a point in the eyes of the developer.
“Using Mizu. I get permissions, I tap, I translate, I see everything in front of me in a nice UI. And I can start digging in and trying to find the relics to a problem that will lead me to the root cause and potentially fix the problem.”
That is, if the developer has authorization within the company to view the traffic. The tool allows companies to set different levels of authorization according to user roles.
Mizu includes a filtering system built atop a database software called Basenine. It’s schema-free and built for the fastest possible write speed and a read speed that scales linearly. It accepts JSON as the record format and allows only create and read access.
The filtering system employs Basenine Filter Language (BFL) inside Basenine database server to enable efficient and precise querying of traffic logs.
With the new filtering syntax, many UI elements are queryable and results can be timestamped.
Test Automation
In 2011, Girmonsky created Blazemeter, a load-testing cloud based on open source Apache JMeter. It was acquired by CA Technologies and later by Broadcom and more recently by Perforce last September.
During the time with Broadcom, Girmonsky and his current co-founders Alex Haiut, Andrey Pokhilko and Refael Botbol Weiss encountered a recurring problem: New releases would break features that previously worked seamlessly and angry calls from customers would ensue.
The ongoing problem led them to conclude that software testing is broken in the microservices age. They’re not the only ones saying so.
They created UP9, a Palo Alto, California-based startup that offers delivers out-of-the-box test-automation for microservices, Kubernetes and cloud native technologies. Incorporating machine learning and artificial intelligence, it’s an on-premises technology, rather than SaaS.
The tools enable developers to unpack API protocols such as Kafka, REST, gRPC, Redis and RabbitMQ and continuously monitor API payloads to detect vulnerabilities that drift into production.
The company announced in December the acquisition of U.K.-based MockLab Ltd., creator of the open source API mocking tool WireMock.
A simulator for HTTP-based APIs, it enables developers to prototype an API that doesn’t yet exist or isn’t complete. You can use it to test edge cases and failure modes that the real API won’t reliably produce.
With the acquisition, WireMock creator Tom Akehurst was named UP9’s chief technology officer.
