Kubernetes / Tools / Sponsored / Contributed

Simple HTTP Load Testing with SLOs

9 Feb 2022 10:27am, by

Srinivasan Parthasarathy
Sri is an applied machine learning researcher with a track record of creating scalable AI/ML/advanced optimization-based enterprise solutions for hybrid cloud, cybersecurity and data-exploration problem domains. A co-founder of Iter8, he has presented at Kubecon 2020 and 2021, and at community meetups like Knative and KFServing.

Load testing is an essential building block in the continuous integration and delivery (CI/CD) of HTTP services. During a load test, a stream of HTTP requests is sent to the target service and responses are evaluated for error and latency-related violations. From a developer’s perspective, setting up a load test involves three key dimensions — namely, i) the load-related characteristics of the request stream, such as the request rate; ii) the shape of the requests, in particular, whether the HTTP endpoint requires a payload to be sent as part of its requests; and iii) the service-level objectives (SLOs) used to validate the quality of the target service.

This article shows how you can use Iter8, the open source Kubernetes-friendly release engineering platform, to flexibly set up a load test with precise control over all of the above, within seconds.

Iter8’s command-line interface (CLI) makes it simple and easy to set up load tests for HTTP services with SLO specifications, verify that the target service meets the SLOs, and create a visual report of the load test as shown below.

Screenshot of Iter8 Experiment Report

Prerequisites: Get Iter8 CLI and Run Sample App

  1. Install the Iter8 CLI using one of the following methods.
    • Using Brew

    • Using precompiled binaries

Precompiled Iter8 binaries for many platforms are available here. Uncompress the iter8-X-Y.tar.gz archive for your platform, and move the iter8 binary to any folder in your PATH.

    • Using Go 1.16+

You can now run iter8 from your gopath bin/ directory.

2. We will use httpbin as the target service. Run it in a separate terminal as follows.

Part 1: Basic Load Test with SLOs

Iter8 provides a predefined and highly customizable experiment chart for load testing. Similar to a Helm chart, Iter8’s experiment charts can be combined with values to create fully specified experiments that can be run by Iter8.

Download the load test HTTP experiment chart as follows.

The sample app (httpbin) implements an HTTP endpoint http://127.0.0.1/get that accepts HTTP GET requests. The following example shows how to run the load test for this endpoint.

The above command sends a steady stream of HTTP GET requests to the specified URL and built-in error and latency-related metrics. By default, the following metrics are collected: error-count, error-rate, latency-mean, latency-max, latency-stddev, latency-min and latency percentiles in the list [50.0, 75.0, 90.0, 95.0, 99.0, 99.9]. In addition, any other latency percentiles that are specified as part of SLOs are also collected. In the above example:

  1. Latency percentiles [25.0, 50.0, 75.0, 90.0, 95.0, 97.5, 99.0, 99.9] are collected and reported.
  2. The following SLOs are validated.
    1. error rate is 0
    2. mean latency is under 50 msec
    3. 90th percentile latency is under 100 msec
    4. 97.5th percentile latency is under 200 msec

You can use the iter8 assert subcommand to assert that the experiment completed without any failures and all the SLOs are satisfied (as follows).

Assuming the assertions are satisfied, the above subcommand exits with exit code 0. It exits with exit code 1 otherwise.

You can use the iter8 report subcommand to view the results of an experiment in HTML or text formats. View a report of the experiment in HTML, as follows.

Open report.html with a browser. In MacOS, you can run open report.html to do so.

Part 2: Load Characteristics

Iter8 makes it easy to control the load-related characteristics of the request stream that is generated by Iter8. In particular, you can specify the number of queries/duration of the load test, the number of queries sent per second and the number of parallel connections used to send requests. The following example shows how to set the total number of requests sent to 200 (numQueries), number of requests per second to 10 (qps) and the number of parallel connections to five (connections). Replace the iter8 run command used in Part 1 with the following.

Part 3: Shape of the Requests

HTTP services with POST endpoints may accept payloads as part of requests. Iter8 makes it easy to send various types of content as payload during the load test.

In the following example, Iter8 downloads a JSON object and uses the downloaded JSON as the payload in POST requests; the type of payload is set to “application/json”.

In the following example, Iter8 downloads an image from Pixabay and uses the downloaded image as the payload in POST requests; the type of payload is set to “image/jpeg”.

In the following example, the payload is not downloaded from a URL but is directly supplied as a string:

Conclusion: Iter8

Iter8’s load test http experiment chart and its simple CLI enable you to rapidly set up load experiments and validate HTTP services. The experiments can be ad hoc, i.e., run on the command line manually. They are also scriptable and can be embedded easily into CI/CD/GitOps pipelines for greater automation, with the outcome of the iter8 assert command influencing the execution of the pipeline.

Complete documentation for Iter8 is available.

Featured image via Pixabay