TNS
VOXPOP
How has the recent turmoil within the OpenAI offices changed your plans to use GPT in a business process or product in 2024?
Increased uncertainty means we are more likely to evaluate alternative AI chatbots and LLMs.
0%
No change in plans, though we will keep an eye on the situation.
0%
With Sam Altman back in charge, we are more likely to go all-in with GPT and LLMs.
0%
What recent turmoil?
0%
Software Testing

Simple Load Testing with GitHub Actions

Mar 16th, 2022 8:07am by
Featued image for: Simple Load Testing with GitHub Actions
Feature image via Pixabay

Michael Kalantar
Michael is a senior software engineer who has contributed to the design and development of a number of  scalable distributed and cloud-based enterprise systems. He is a co-founder of the Iter8 project.

In this article, we show how to use GitHub Actions to load-test, benchmark and validate HTTP and gRPC services with service-level objectives (SLOs).

When developing a new version of an HTTP or gRPC service, it is desirable to benchmark its performance and to validate that it satisfies desired service-level objectives (SLOs) before upgrading the current version. We describe a no-code approach based on GitHub Actions that can be used to automate such testing at any point in a continuous integration/continuous delivery (CI/CD) pipeline. For example, at build time it can be used to validate the new version as soon as possible. Alternatively, at deployment time it can be used to validate SLOs in the production environment.

HTTP Load Testing with the Iter8 GitHub Action

The Iter8 GitHub Action, iter8-tools/iter8-action@v1, enables automated Iter8 experiments in a GitHub workflow. To use the action, specify an experiment chart and its configuration via a Helm valuesFile. No programming is necessary — all configuration is declarative. Typical use is to define the test configuration in a values.yaml file and call the Iter8 action passing the configuration. For example, to run a load test against an HTTP service, use the built-in load-test-http chart. For example:

Example use of Iter8 action to run a load test against HTTP endpoint http://localhost:8080 and validate three service-level objectives: Error rate must be 0, mean latency must be below 100 milliseconds and the 97.5th percentile latency must be below 200 milliseconds.

All available options for the load-test-http experiment chart are documented in the chart readme.

Additional examples of the Iter8 action for testing HTTP services can be found in the Iter8 end-to-end test cases. To run these test cases:

  1. Fork the Iter8 docs repo: https://github.com/iter8-tools/docs to your organization, myorg.
  2. If necessary, enable workflows on your forked repository (by navigating to https://github.com/myorg/docs/actions).
  3. Select the workflow end-to-end tests.
  4. Run the workflow using the Run workflow button.
  5. When the workflow is completed, you can inspect the test definition (test.yaml) and logs for the load tests against a local httpbin service, local httpbin tests.

Benchmarking

When the Iter8 action executes, it logs a list of benchmarks that can be inspected later. A sample benchmarking report is:

SLO Validation

By default, the Iter8 action validates any specified SLOs and the action succeeds only if the SLOs are validated. In this way, a workflow using this action can be certain that it is safe to promote the new service. However, if the goal is only to benchmark the service, the optional flag validateSLOs can be set to false. In this case, the action will not validate SLOs and will succeed when the experiment completes.

gRPC Load Testing with the Iter8 GitHub Action

gRPC services can be tested using the built-in load-test-grpc chart. The Iter8 end-to-end test cases contain an example. Run as described above, but inspect the logs for local grpc tests. The full range of configuration options for this chart is documented in the chart readme for the chart.

Conclusion

The Iter8 GitHub action simplifies the execution of performance testing for HTTP and gRPC services. Configuration is declarative; no programming is necessary. The action generates and applies load to a target service. It measures and records performance metrics that are used to benchmark the service and to validate desired SLOs. In a GitHub workflow, it can be used to ensure that new versions are promoted only when it is safe to do so.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.