Isolating Wolfram
We’d like to keep our Wolfram tests isolated, but we have a problem. Our code makes an HTTP request to the WolframAlpha API, which isn’t something we want to perform within our test suite. You might be thinking, “Let’s write a bunch of mocks!”
Within the Elixir community, we want to avoid mocking whenever possible. Most mocking libraries, including dynamic stubbing libraries, end up changing global behavior—for example, by replacing a function in the HTTP client library to return some particular result. These function replacements are global, so a change in one place would change all code running at the same time. That means tests written in this way can no longer run concurrently. These kinds of strategies can snowball, requiring more and more mocking until the dependencies among components are completely hidden.
The better strategy is to identify code that’s difficult to test live, and to build a configurable, replaceable testing implementation rather than a dynamic mock. We’ll make our HTTP service pluggable. Our development and production code will use our simple :httpc client, and our testing code can instead use a stub that we’ll call as part of our tests. Let’s update our Wolfram backend to accept an HTTP client from the application configuration, or a default of :httpc. Update rumbl_umbrella/apps/info_sys/lib/info_sys/wolfram.ex with this code:
| 1: | @http Application.get_env(:info_sys, :wolfram)[:http_client] || :httpc |
| 2: | defp fetch_xml(query) do |
| 3: | {:ok, {_, _, body}} = @http.request(String.to_charlist(url(query))) |
| 4: | |
| 5: | body |
| 6: | end |
We have made only a minor change to this file. First, we look up an :http_client module from our mix configuration and default it to the :httpc module. We bake that module into an @http module attribute at compile time for speedy runtime use. Next, we replace our :httpc.request call with an @http.request invocation.
The result is simple and elegant. We simply call the function as before, using our environment’s HTTP client instead of hardcoding the HTTP client. This way, our behavior remains unchanged from before, but we can now stub our HTTP client as desired.
Now let’s update our test configuration to use our stubbed client. Update the config/test.exs file at the umbrella root, like this:
| | config :info_sys, :wolfram, |
| | app_id: "1234", |
| | http_client: InfoSys.Test.HTTPClient |
This bit of configuration sets two configuration keys for Wolfram. One key is the as-yet unwritten module for our test backend. The other is a fake configuration key that we can replace if we need to do some direct testing—for example, as we’re creating data for our stub.
Now on to the tests. To test our stubbed WolframAlpha API results, we need an example XML payload. Wolfram conveniently includes an API explorer[34] that accepts a search query and displays the XML response. We’ve grabbed a result for you for a query of "1 + 1". Keep in mind that this file is incomplete. You will need to use the Wolfram service to build your own or copy our version from the sample code for our book. Either way, place the entire XML response into a new rumbl_umbrella/apps/info_sys/test/fixtures/ directory and save it as wolfram.xml:
| | <?xml version='1.0' encoding='UTF-8'?> |
| | <queryresult success='true' |
| | error='false' |
| | numpods='6' |
| | ... |
With our fixture in place, now we need a stubbed HTTP client, one that returns fake XML results using our fixture. Create a new rumbl_umbrella/apps/info_sys/test/backends/ directory and add the following module to a new rumbl_umbrella/apps/info_sys/test/backends/http_client.exs file:
| | defmodule InfoSys.Test.HTTPClient do |
| | @wolfram_xml File.read!("test/fixtures/wolfram.xml") |
| | def request(url) do |
| | url = to_string(url) |
| | cond do |
| | String.contains?(url, "1+%2B+1") -> {:ok, {[], [], @wolfram_xml}} |
| | true -> {:ok, {[], [], "<queryresult></queryresult>"}} |
| | end |
| | end |
| | end |
We define an InfoSys.Test.HTTPClient module that stubs our request function and returns fake Wolfram results. We cheat as we did before. We check the fetched url for the URI-encoded "1 + 1" string. If it matches, we simply return the XML contents of our wolfram.xml fixture. For any other case, we return a fake request for empty XML results.
Our goal isn’t to test the Wolfram service, but make sure we can parse the data Wolfram provides. This code elegantly lets us write tests at any time that return a result. To confirm our HTTPClient module is loaded before our tests, add the following line to the top of your rumbl_umbrella/apps/info_sys/test/test_helper.exs:
| | Code.require_file "../../info_sys/test/backends/http_client.exs", __DIR__ |
| | ExUnit.start() |
With our HTTP client in place, create a new rumbl_umbrella/apps/info_sys/test/backends/wolfram_test.exs file with the following contents:
| | defmodule InfoSys.Backends.WolframTest do |
| | use ExUnit.Case, async: true |
| | |
| | test "makes request, reports results, then terminates" do |
| | actual = hd InfoSys.compute("1 + 1", []) |
| | assert actual.text == "2" |
| | end |
| | |
| | test "no query results reports an empty list" do |
| | assert InfoSys.compute("none", []) |
| | end |
| | end |
Since we’ve put in the hard work for testing the cache and generic InfoSys this test will be light, and that’s exactly how we want tests that must consider external interfaces. Using our stubbed HTTP client, we add test cases to handle requests with and without results.
Now let’s run the test:
| | $ mix test |
| | .. |
| | |
| | Finished in 0.2 seconds (0.1s on load, 0.09s on tests) |
| | 5 tests, 0 failures |
And they pass. Since we’re handling the rest of the edge cases in our base info_sys tests, that should wrap up the Wolfram tests!
For the WolframAlpha API case, we chose to create a stub that replaces the :httpc module. However, you might not be comfortable with skipping the whole HTTP stack during the test. You’ll have to decide the best place to stub the HTTP layer. No single strategy works for every case. It depends on your team’s confidence and the code being tested. For example, if the communication with the endpoint requires passing headers and handling different responses, you might want to make sure that all of those parameters are sent correctly.
One possible solution is the Bypass[35] project. Bypass allows us to create a mock HTTP server that our code can access during tests without resorting to dynamic mocking techniques that introduce global changes and complicate the testing stack.
Our tests are all green, and they’ll be consistently green because we make sure that our measurements await the completion of our tests.
You may have noticed that these tests are more involved than the typical single-process tests you might be used to. But by using the specific helpers that ExUnit provides and thinking through possible outcomes and orderings, you’ll quickly get the hang of writing tests that aren’t too much more difficult than synchronous ones. When you’re done, you’ll have one major advantage. Your tests will run concurrently, meaning they’ll finish much more quickly than their synchronous counterparts.
With our Wolfram backend covered, it’s time to move on to the last part of our application: the channels. We’re ready to use the testing tools from Phoenix.ChannelTest to set up your tests and finish out the rest of our tests.