Writing Property Tests with ExUnit
Elixir makes testing a breeze using it’s testing framework ExUnit. ExUnit is a framework for writing unit tests that’s built in to every Elixir project. Every time you create a new project using mix new, Elixir will automatically create a test directory for you and populate it with a skeleton for writing unit tests for your project. ExUnit is already packaged with Elixir, so you don’t need to install any dependencies.
StreamData is an Elixir library for writing property-based tests. Property-based tests test the properties of a function to ensure your code meets some specified properties every time. Property-based tests are especially useful when your functions contain randomness, because you can write tests that run hundreds of times and ensure your functions meet specified properties and behaviors. For example, if you had a function that generated lists of length 10 of random integers between 1 and 10, you could run the function hundreds of times and ensure that every time it ran, the list it generated contained 10 elements and all of the elements were integers between 1 and 10. StreamData offers utilities for creating streams of data that can be combined with it’s ExUnitProperties module to write property-based tests.
This section will walk you through an example property-based test for Toolbox.Crossover.single_point/2. You can repeat this process with the rest of your functions in your framework. To learn more about property-based testing, check out Property-Based Testing with PropEr, Erlang, and Elixir [Heb19] or Testing Elixir [LM21].
To get started, you first need to add StreamData to your dependencies, like this:
| | defp deps do |
| | ... |
| | {:stream_data, "~> 0.5", only: :test} |
| | end |
The only: :test property specifies that you only need this dependency in a test environment. Run mix deps.get to ensure StreamData is loaded.
Next, create a new file in tests called crossover_test.exs. In that file, add the following:
| | def CrossoverTest do |
| | use ExUnit.Case |
| | use ExUnitProperties |
| | alias Types.Chromosome |
| | end |
Here you define a module that will contain your test and ensure it uses the ExUnit.Case suite of tools. You then use ExUnitProperties to import the macros for writing property-based tests. Finally, you alias Types.Chromosome, because you’ll be creating chromosomes to test your function with later on.
Now, consider what properties the function Toolbox.Crossover.single_point/2 needs to maintain. Recall from Chapter 6, Generating New Solutions, single-point crossover takes in two chromosomes and swaps slices of genes at a random crossover point. It maintains the size of both chromosomes, and produces two unique chromosomes at the end. Your test should check to ensure that the size of the chromosomes produced by Toolbox.Crossover.single_point/2 is the same as the input chromosomes. You can try to enforce other properties later on; for now you’ll just ensure that single_point/2 maintains the size of the input chromosomes.
Define your test like this:
| | property "single_point/2 maintains the size of input chromosomes" do |
| | check all size <- integer(0..100), |
| | gene_1 <- list_of(integer(), length: size), |
| | gene_2 <- list_of(integer(), length: size) do |
| | p1 = %Chromosome{genes: gene_1, size: size} |
| | p2 = %Chromosome{genes: gene_2, size: size} |
| | {c1, c2} = Toolbox.Crossover.single_point(p1, p2) |
| | assert c1.size == size and c2.size == size |
| | end |
| | end |
Here you define a property using the property macro. You then use the check all macro to generate some data to test your function with. StreamData comes with a number of helpful generators that make creating data easy. In this example you use them to generate genes for two parent chromosomes. In the body of the test, you define two parent chromosomes and then run Toolbox.Crossover.single_point/2 to get two children. Finally, you assert that the size of the children is equal to the original size of both of the original sets of genes.
Next, run mix test, like this:
| | $ mix test |
| | ... |
| | Finished in 0.08 seconds |
| | 1 property, 1 failure |
After running, your test might have failed; but why? If you inspect the output of the test, you’ll notice that the function doesn’t behave well when running on chromosomes with empty genes. While it’s not likely you’ll run into this problem in practice, you should fix your function to handle empty genes just in case. Open up crossover.ex in toolbox and add the following above single_point/2:
| | def single_point(c1 = %Chromosome{genes: []}, |
| | c2 = %Chromosome{genes: []}), do: {c1, c2} |
This function uses pattern matching to check if c1 and c2 are empty. If they are, it returns the original chromosomes. Now, run mix test again:
| | $ mix test |
| | ... |
| | Finished in 0.08 seconds |
| | 1 property, 0 failures |
And your problem is fixed. Property-based testing is a useful tool for quickly identifying bugs in your code, especially when trying to test stochastic functions. You may have written a test to explicitly handle the case of empty chromosomes, but the property-based test here caught it for you automatically. You can implement tests like this example for most of the functions in your framework to identify any similar bugs in your code.
Now that you know how to test, in the next section you’ll learn how to clean up your code using the static analysis tool, credo.