Testing has always been a must at Adjust. Serving tens of thousands of requests per second, we can’t afford tiny oopsies caused by some pointer being
null when we didn’t expect it or a good old side effect in a presumably clean function. Tests are considered first-class citizens in our code base and are mandatory for any pull request that adds or changes functionality.
Just like tests, any dependency that we bring into our projects becomes a part of our code base. Since no one likes maintaining code written by some unknown dude (do you?), we try to reduce the number of third-party packages we use to an absolute minimum and prefer writing or own tailor-made libraries.
This is why we decided to stick with the minimalistic but decent
testing package that Go provides as a part of its standard library.
Every test, regardless of programming language, contains four major parts: setup, invocation, assertion, and teardown. Besides providing the framework to run tests using the
testing provides only one of these things–assertion–and leaves the rest up to its user. While the invocation part is usually very specific to implementation and there is not much that could be generalized there, setup and teardown are tightly coupled and can often be split into small reusable blocks that mock external services, provide data fixtures, etc. So we’ll focus on these two parts, specifically on mocking HTTP requests to external services.
Mocking an external HTTP API in tests
A common task in our tests is to verify the interaction with an HTTP API provided by a third party. We don’t want to send all the requests we’re making in our unit tests to an actual server for several reasons:
- This would add an extra dependency on the network state between our CI server and the target service
- It would be difficult to reproduce the error responses
- Any API change would cause our tests to fail. Some would say that this is good rather than bad, but we believe that test failures should be induced by only our changes.
- We’re nice people and wouldn’t want to bother the API providers unless we really have to
Thus we came up with a fairly simple idea to mock
1 2 3 4 5 6 7 8 9 10 11 12
Now we only needed to mock the expected requests with canned responses:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1 2 3 4 5 6 7 8 9 10 11
This approach worked well for a while, but soon we found ourselves adding more and more functionality to
testutils.ClientMock. Sometimes we’d need to add additional cookies to the response, send requests using different HTTP methods, or provide a different response depending on what was sent in the request. The mock became so complex, that we started thinking about writing tests for it.
Nobody was smitten with the idea of writing tests for tests, so we had to rethink our approach. By that time, our client mock looked almost like a limited
http.Server without the transport part, so we decided to leave the honorable task of testing mocks to the Go team and came up with the following approach, which is currently used in most of our tests these days:
1 2 3 4 5 6 7 8 9 10
And the code turned into something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
We’re now using the standard library
*http.Client, and, instead of mocking its request methods, we override the host and port of the API server. This way we can redirect every HTTP request to our server mock:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Mocking an HTTPS server
For tests that explicitly require HTTPS, we added a similar mock that creates an instance of
httptest.Server by calling
httptest.StartTLSServer() instead of
httptest.StartServer(), while the rest of the code is completely the same as in
1 2 3 4 5
But this made
http.Client complain about a bad certificate:
Since we did not provide any certificates at all while creating an
httptest.Server instance, there should have been some default one hidden in
net/http/httptest. It turned out that the Go standard library contains a self-signed certificate and a private key used by the
httptest package. So we needed to make the
http.Client trust this certificate:
1 2 3 4 5 6 7 8 9 10 11
If you’ve already upgraded to Go 1.9, then you don’t need
x509.ParseCertificate() anymore. An instance of
httptest.Server now has a
Certificate() method that returns an
*x509.Certificate used by this server.
All that was left now was to replace the system-default
http.Client certificate pool with our own one, which held our server mock certificate:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
So what about HTTP/2?
Since Go v1.6,
http.Server supports HTTP/2 out of the box, and we naturally assumed
httptest.Server would, too. However, once we configured
http.Client to use
golang.org/x/net/http2, our tests that used
testutil.ServerMock() began to fail with an unexpected error:
1 2 3
A quick note on ALPN ALPN or Application-Layer Protocol Negotiation is a TLS extension that allows parties to agree on which protocol should be handled over a secure connection. HTTP/2 uses this feature to avoid additional round trips, and, hence, TLS handshakes, by agreeing on an application protocol during the hello phase. The client provides a list of protocols it supports and the server is expected to choose one and send it back.
unexpected ALPN protocol ""; want "h2" meant that our server did not know it now supported HTTP/2. There is a method in the
http2 library to configure an existing server to support HTTP/2, but it expects an instance of
http.Server as an argument, whereas we only had
(httptest.Server).Config as an argument to
http2.ConfigureServer() wouldn’t work, because
Config to serve incoming connections using an already existing
tls.Listener that is created when
(*httptest.Server).StartTLS() gets called, and ALPN support is implemented by
crypto/tls. Thus we needed a way to configure the
httptest.Server listener to support
"h2" as an application-level protocol.
1 2 3 4 5 6 7 8
Looks exactly like what we’re looking for! What was left was to apply the same configuration changes as
http2.ConfigureServer() does and we’d have a nicely working HTTP/2 mock using Go standard library only:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
http2.NextProtoTLS is a constant for the
"h2" string we were looking for and
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 is a cipher suite required by the HTTP/2 specification.
Instead of mocking
http.Client, mock the server it talks to. The Go standard library offers a very convenient
net/http/httptest package that allows you to spawn an HTTP server with only a few lines of code, which can be easily configured to handle HTTP/2 requests.