API Management / Open Source / What Is DevOps? / Sponsored / Contributed

How to Test External APIs with Nock

8 Dec 2021 9:50am, by
knock

Darin Spivey
Darin is a senior software engineer at LogDNA, where he works on product architecture and performance, advanced testing frameworks and LogDNA's open source projects.

Testing external APIs can be tricky, especially if your company does not own the service/endpoint. Still, this testing is an important part of having good integration tests. Mocks are never good for testing such things because it makes too many assumptions that can, and will, change later to cause tests that pass when they should fail. So, what are some good approaches to testing external APIs?

Just spin up the service locally.

The most thorough way is to spin up the service locally using a Docker image of it. Granted, the setup is a little more complicated, but the correct way is seldom the easiest. Doing this ensures that the calls are real, and the only concern is bumping the image periodically to stay current with the master.

Nock, Nock, Who’s There?

If you can’t use the actual service, then the next best thing is something that can intercept HTTP traffic and put the control in the hands of the person writing the tests. A package I’ve grown to love is called nock, and it lets you do just that. Using nock, your HTTP calls will be handled by the code’s HTTP agent as written; the only difference is that it’s not actually talking to anything real. This allows the user to control returned status codes, inspect POST bodies, query strings and just about anything a good test will want to do. For me, the fact that the real agent is actually used is a big win. A response from axios is far different from a request response, so nock allows the user to see how the agent handles situations created by the tester so that proper assertions can be written.

How it Works

nock is given a URL (or URI), then told what patterns to intercept by using chained methods. Note that things like path parameters and query strings must be exact matches, or nock won’t work. Thankfully, nock accepts regular expressions as well as function handlers to assist with this, and the tester can make it as flexible or rigid as they want. My point is though, that if you have something like a query string in the API call, it can’t be ignored, a query handler will have to be set up for it. For added protection, nock.disableNetConnect() can be used to turn off all external calls, just in case a mistake was made and a real HTTP call would be made.

A Real Example

Recently, I worked on some of our client code where batches of log lines are posted to the ingester. Since this is a public package, spinning up the service isn’t an option for security reasons, but I still needed to test how lines are POSTed to our service. Using nock, I was able to test things such as retries, HTTP timeouts, error conditions and whether the proper payloads were being sent, all while using the axios agent to process the responses.

This is a happy path test. Note that there is a query handler to accept anything since the query string contains values that will change with every test. Returning true means it’s a valid match. Here, we examine that the payload is as expected, then we reply with a 200 success. Replies can be JSON too!

For this client, user errors are not considered to be failure, and exponential backoff will NOT be used. Use persist() to make the interceptor last for more than one call (the default).

Now, let’s get fancy! How about something that tests exponential backoff logic upon getting an HTTP timeout?

Best Practices

  • nock will return a reference to the interceptor definition. .isDone() (returns a bool) can be used to easily tell if it was used in a test. You can use scope.done() to accomplish the same thing, but that does an assertion that will fail a test if it hasn’t been called.
  • Each interceptor will only be used once by default, but .persist() can be used to keep them alive. Because of that, each test should probably clear all interceptors to avoid cross-contamination of tests.t.on('end', async () => { nock.cleanAll() })
  • If you need something more low-level (like testing TCP traffic), the Man in the Middle (mitm) package can help.

The New Stack is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker, Real.

Photo by cottonbro from Pexels.