Having a complete and consistent test suite that consists of unit, integration, or functional tests is essential in ensuring a good overall quality of your software development. However, sometimes, such a suite is not enough. While testing for example-specific data structures, it often happens that there are too many possible values to test with, which means that there is a very large amount of mocking or production of test data. Automated property-based testing is the aim of ScalaCheck, a Scala library inspired by Haskell that allows generating, more or less randomly, the test data to verify some properties about the code you are testing. This library can be applied to Scala as well as to Java projects.

To get up and running quickly with ScalaCheck, you can include the appropriate library in the build.sbt file, as we have often done till now. This is shown as follows:

resolver += Resolver.sonatypeRepo("releases")

libraryDependencies ++= Seq(
"org.scalacheck" %% "scalacheck" % "1.11.0" % "test")

From the SBT prompt, you may type reload instead of exiting and relaunching SBT, to get a fresh version of the build file, and then type update to fetch the new dependency. Once this is done, you may also type eclipse to update your project with the dependency so that it will be a part of your classpath, and the editor will recognize the ScalaCheck classes.

Let's first run the StringSpecification test that is proposed by the Quick start page available at www.scalacheck.org:

import org.scalacheck.Properties
import org.scalacheck.Prop.forAll

object StringSpecification extends Properties("String") {

  property("startsWith") = forAll { (a: String, b: String) =>
    (a+b).startsWith(a)
  }

  property("concatenate") = forAll { (a: String, b: String) =>
    (a+b).length > a.length && (a+b).length > b.length
  }

  property("substring") = forAll { (a: String, b: String, c: String) =>
    (a+b+c).substring(a.length, a.length+b.length) == b
  }

}

In this code snippet, ScalaCheck produces (randomly) a number of strings and verifies that the properties are correct; the first one is straightforward; it should verify that adding two strings a and b should produce a string that starts with a. It probably sounds obvious that this test will pass, no matter what the values of the strings are, but the second property that verifies the length of the concatenation of the two strings is not always true; feeding both a and b with the empty value "" is a counter example that shows that the property is not verified. We can illustrate that by running the test via SBT as follows:

> test-only se.chap4.StringSpecification
[info] + String.startsWith: OK, passed 100 tests.
[info] ! String.concatenate: Falsified after 0 passed tests.
[info] > ARG_0: ""
[info] > ARG_1: ""
[info] + String.substring: OK, passed 100 tests.
[error] Failed: : Total 3, Failed 1, Errors 0, Passed 2, Skipped 0
[error] Failed tests:
[error] 	se.chap4.StringSpecification
[error] (test:test-only) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 1 s, completed Nov 19, 2013 4:30:37 PM
>

ScalaCheck conveniently outputs a counter example, ARG_0: "" and ARG_1: "" that makes the test fail.

We can add a few more tests on more complex objects than just strings. Let's add a new test class named ConverterSpecification as part of our test suite, to test the Converter that we have created in the Mocking with ScalaMock section:

package se.chap4

import org.scalacheck._
import Arbitrary._
import Gen._
import Prop.forAll

object ConverterSpecification extends Properties("Converter") with Converter {

  val currencies = Gen.oneOf("EUR","GBP","SEK","JPY")

  lazy val conversions: Gen[(BigDecimal,String,String)] = for {
    amt <- arbitrary[Int] suchThat {_ >= 0}
    from <- currencies
    to <- currencies
  } yield (amt,from,to)

  property("Conversion to same value") = forAll(currencies) { c:String =>
    val amount = BigDecimal(200)
    val convertedAmount = convert(amount,c,c)
    convertedAmount == amount
  }

  property("Various currencies") = forAll(conversions) { c =>
    val convertedAmount = convert(c._1,c._2,c._3)
    convertedAmount >= 0
  }
}

If we run the test in SBT, the following output is displayed:

> ~test-only se.chap4.ConverterSpecification
[info] + Converter.Conversion to same value: OK, passed 100 tests.
[info] + Converter.Various currencies: OK, passed 100 tests.
[info] Passed: : Total 2, Failed 0, Errors 0, Passed 2, Skipped 0
[success] Total time: 1 s, completed Nov 19, 2013 9:40:40 PM
1. Waiting for source changes... (press enter to interrupt)

In this specification, we added two specific generators; the first one named currencies is able to generate only a few strings taken from a list of valid currencies that we want to test, as otherwise, a randomly generated string would produce strings that are not part of the Map. Let's add an invalid item "DUMMY" to the generated list to verify that the test is failing:

val currencies = Gen.oneOf("EUR","GBP","SEK","JPY","DUMMY")

On saving this, the tests are rerun automatically as we specified the ~ sign in front of test-only. This is shown as follows:

[info] ! Converter.Conversion to same value: Exception raised on property evaluation.
[info] > ARG_0: "DUMMY"
[info] > Exception: java.util.NoSuchElementException: key not found: DUMMY
[info] ! Converter.Various currencies: Exception raised on property evaluation.
[info] > ARG_0: (1,,)
[info] > ARG_0_ORIGINAL: (1,DUMMY,SEK)
[info] > Exception: java.util.NoSuchElementException: key not found: 
[error] Error: Total 2, Failed 0, Errors 2, Passed 0, Skipped 0
[error] Error during tests:
[error] 	se.chap4.ConverterSpecification
[error] (test:test-only) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 1 s, completed Nov 19, 2013 9:48:36 PM
2. Waiting for source changes... (press enter to interrupt)

The second generator named conversions illustrates the construction of a more complex generator that takes advantage of the power of for comprehensions. In particular, notice the suchThat {_ >= 0} filter method that makes sure that the arbitrary chosen integer has a positive value. This generator returns a Tuple3 triplet that contains all the necessary values to test the Converter.convert method.