Iter8 Unifies Performance Validation for gRPC and HTTP
gRPC is an open source remote procedure call (RPC) system that is becoming increasingly popular for connecting microservices and connecting mobile/web clients to backend services. Benchmarking and performance validation is an essential building block in the continuous integration and delivery (CI/CD) of robust gRPC services.
In this hands-on article, we show how Iter8 enables you to rapidly launch gRPC benchmarking and validation experiments, and configure the service-level objectives (SLOs) used to validate the gRPC service, the load profile used for benchmarking, and the data and metadata supplied as part of the call requests, all in a matter of seconds. We also discuss how Iter8 unifies performance validation for HTTP and gRPC services.
What Is Iter8?
Iter8 is the open source Kubernetes release optimizer built for DevOps, MLOps, site reliability engineers (SRE) and data science teams. Iter8 introduces a primitive called “Experiment” that makes it simple to collect performance and business metrics, assess, compare and validate multiple app/machine learning model versions, and safely promote the winning version of the app/ML model. To enable easy reuse, Iter8 experiments are packaged and released in the form of experiment charts, which are Helm charts with a specific structure. For example, the load-test-grpc experiment enables benchmarking and performance validation for gRPC services. This experiment is the main focus of this article and is illustrated below.
Why Iter8?
Iter8 is packed with powerful features that simultaneously enhance and simplify your release engineering/CI/CD workflows. We highlight three of them below.
- Iter8 provides a unified way to benchmark and validate both HTTP and gRPC services. This enables organizations to evolve a common set of CI/CD best practices and design patterns that are applicable to both these types of services.
- Iter8’s CLI generates intuitive reports that describe the results of an experiment. Reports are ideal for promoting human understanding of experiment outcomes through visual insights. Iter8 CLI also provides a simple way to assert conditions, such as whether the target app satisfies the SLOs specified in the experiment. Assertions are ideal from an automation perspective, since they allow CI/CD pipelines and/or scripts to branch off into different paths depending on whether the assertions are true.
- Iter8’s experiment charts make it possible to launch experiments in seconds by supplying a few relevant parameter values, which conceptually and operationally simplify even advanced benchmarking and validation tasks.
Benchmark and Validate gRPC
Get started with gRPC benchmarking and validation by installing the Iter8 CLI.
|
1 2 3 |
brew tap iter8-tools/iter8 brew install iter8@0.9 |
Alternatively, you can also use prebuilt binaries for various platforms, or Go 1.17+ for Iter8 CLI installation.
Sample App
Run the sample gRPC service in a separate terminal.
docker run -p 50051:50051 docker.io/grpc/java-example-hostname:latest
Iter8 Experiment
The iter8 launch subcommand downloads an experiment chart from Iter8 hub (a Helm repo containing Iter8 experiment charts) and generates a fully formed experiment (experiment.yaml) by combining the chart with user-supplied values, runs the experiment and outputs the results of the experiment run (result.yaml). Try Iter8’s load-test-grpc experiment chart to test the sample gRPC service as follows. In this experiment launch, you are supplying the host information, the fully qualified method name (call) and the URL of the protocol buffer file (protoURL) for the gRPC service.
|
1 2 3 4 5 6 7 |
iter8 launch -c load-test-grpc \ --set host="127.0.0.1:50051" \ --set call="helloworld.Greeter.SayHello" \ --set protoURL="https://raw.githubusercontent.com/grpc/grpc-go/master/examples/helloworld/helloworld/helloworld.proto" |
Report
The iter8 report subcommand generates a report of an experiment in HTML or text formats. Generate an HTML report and open report.html using a browser.
|
1 |
Iter8 report -o html > report.html |
The experiment report looks as follows.
Metrics and SLOs
The ability to collect built-in latency and error-related metrics, and automatically validate SLOs, is one of the powerful features of Iter8. Modify the iter8 launch command used above by configuring the following additional parameters.
|
1 2 3 4 5 6 7 |
--set SLOs.grpc/error-rate=0 \ --set SLOs.grpc/latency/mean=50 \ --set SLOs.grpc/latency/p90=100 \ --set SLOs.grpc/latency/p'97\.5'=200 |
The above configuration validates that the app satisfies the following SLOs.
- Error rate is 0.
- Mean latency is under 50 milliseconds.
- 90th percentile latency is under 100 msec.
- 97.5th percentile latency is under 200 msec.
Generate and view the experiment report, which now contains SLO assessments in addition to metric values.
Assert Conditions
Use the iter8 assert subcommand to check whether the experiment completed without failures and if all the SLOs are satisfied. This command will exit with code 0 if the assert conditions are satisfied and with code 1 otherwise. Assertions are especially useful within CI/CD/GitOps pipelines.
|
1 |
iter8 assert -c completed -c finished -c slos |
Load Profile
Control the characteristics of the load generated by the load-test-grpc experiment by configuring the number of requests (total), the number of requests per second (rps), number of connections to use (connections) and the number of concurrent request workers to use in each connection (concurrency). To do so, modify the iter8 launch command used above by configuring the following additional parameters.
|
1 2 3 4 5 6 7 |
--set total=500 \ --set rps=25 \ --set concurrency=50 \ --set connections=10 |
Advanced Parameters
The load-test-grpc chart supports a number of advanced parameters in addition to those illustrated in the basic examples above. You can use these parameters to supply call data information in the form of JSON or binary files that may be local, or hosted at a remote URL; supply call metadata information at the command line, or as JSON input as a local file or a remote file hosted at a URL; supply proto files or precompiled protoset files or have Iter8 attempt to use server reflection in their absence, and use client streaming, server streaming or bidirectional streaming call patterns.
To learn more about all of the above extensions and other Iter8 capabilities, please visit https://iter8.tools.
Benchmark and Validate HTTP
For performance validation of HTTP services, Iter8 provides the load-test-http chart, which is illustrated below.
Try the load-test-http experiment as follows.
|
1 2 3 4 5 6 7 8 9 |
iter8 launch -c load-test-http \ --set url="https://httpbin.org/post" \ --set numQueries=200 \ --set payloadStr="abc123" --set SLOs.latency-mean=50 |
The load-test-http experiment supports validation of both GET and POST endpoints. Like its gRPC counterpart, this experiment also supports configuring a payload as part of the requests, either through a string value, local files, or remote URLs, configuring the content type, configuring the number of queries, queries per second, the number of parallel connections used, and configuring error and latency related SLOs. As in any Iter8 experiment, you can use the iter8 report command to view text or HTML reports, and use the iter8 assert command to assert experiment outcomes.
Conclusion
Benchmarking and validating SLOs is an agile practice that enables DevOps engineers to deliver applications at a high frequency, along with the confidence that they will not compromise the end-user experience in the process. A simple no-code CLI-driven solution for benchmarking and SLO validation that supports both gRPC and HTTP in a unified manner, with flexible specification of SLOs, load profile, data and metadata, goes a long way toward simplifying the application delivery process and freeing up the DevOps engineer for other business-critical tasks.
Iter8 provides this solution.