The ScalaCheck test harness is the Prop, and the testing class is a collection of Properties. To create a ScalaCheck test, create an object that extends Properties. The following is a basic test object for a ScalaCheck test. The argument passed to the constructor of the super class is the name that describes the test, Simple Math.

src/test/scala/com/oreilly/testingscala/BasicScalaCheckProperties.scala. 

package com.oreilly.testingscala

import org.scalacheck.{Prop, Properties}

object BasicScalaCheckProperties extends Properties("Simple Math"){
  property("Sum is greater than its parts") = Prop.forAll {(x:Int, y:Int) => x+y > x && x+y > y}
}

The title is sent to the superclass as a String parameter. A Properties test must be an object (a Scala singleton) and not a class. Otherwise, the test will not run.

Each property takes a String to describe the purpose of the test. The example shown tests whether the sum of two numbers is greater than the sum of each number by itself. (Hint: it isn’t always.) The left side of the assignment statement is referred to as the property specifier, while the right side is the actual Prop test. Prop.forAll takes, as its parameter, a function whose arguments describe the data that the test developer wants automatically generated for a test. In this example, the Prop will provide two integers, and ScalaCheck will test the assertion that the sum is greater than the first created number (x), then that it is greater than the second created number (y).

Running the test using either test or test-only in SBT renders the following output.

src/test/scala/com/oreilly/testingscala/BasicScalaCheckProperties.scala. 

> ~test-only com.oreilly.testingscala.BasicScalaCheckProperties
[info] ! String.Sum is greater than its parts: Falsified after 0 passed tests.
[info] > ARG_0: 0
[info] > ARG_1: 0
[error] Failed: : Total 1, Failed 1, Errors 0, Passed 0, Skipped 0
[error] Failed tests:
[error]     com.oreilly.testingscala.BasicScalaCheckProperties
[error] {file:/home/danno/testing_scala_book.git/testingscala/}default-cef86a/test:test-only: Tests unsuccessful
[error] Total time: 0 s, completed Jan 11, 2012 10:26:08 PM
1. Waiting for source changes... (press enter to interrupt)

It didn’t take long for the test to fail, and ScalaCheck is nice enough to let us in on why it failed to pass. In the output, the Property check failed when both arguments, ARG_0 and ARG_1, were zero.

Let’s change the test to something a bit more successful.

src/test/scala/com/oreilly/testingscala/BasicScalaCheckProperties.scala. 

package com.oreilly.testingscala

import org.scalacheck.{Prop, Properties}

object BasicScalaCheckProperties extends Properties("Simple Math"){
  property("Sums are associative") = Prop.forAll {(x:Int, y:Int) => x+y == y+x}
}

Running the test now reports success. This success shows how thoroughly this method was tested by reporting that Sums are associative passed 100 tests. This is very worth repeating: 100 separate combinations of integers were tested, without the test developer typing in any data table or data provider, often times, with data that the developer did not consider. In the case of integers that can include a 0, MAX_VALUE, and MIN_VALUE. For floating-point numbers, that can include NaN, Infinity, and -Infinity.

> ~test-only com.oreilly.testingscala.BasicScalaCheckProperties
[info] Compiling 1 Scala source to /home/danno/testing_scala_book.git/testingscala/target/scala-2.9.0-1/test-classes...
[info] + Simple Math.Sums are associative: OK, passed 100 tests.
[info] Passed: : Total 1, Failed 0, Errors 0, Passed 1, Skipped 0
[success] Total time: 21 s, completed Jan 11, 2012 10:42:06 PM
1. Waiting for source changes... (press enter to interrupt)

The next few examples test some of the music-based code that was developed in earlier chapters, to give our production code some good vetting. First, we’ll generate some first, middle, and last names in the style that we tested in AcceptanceSpecification. We’ve designed the string so that the middle name is optional, so the following definition of the property contains two cases. ScalaCheck will generate some strings with a middle name and some without.

/src/test/scala/com/oreilly/testingscala/ArtistScalaCheckProperties.scala

package com.oreilly.testingscala

import org.scalacheck.{Prop, Properties}


object ArtistScalaCheckProperties extends Properties("Testing Artists Thoroughly") {
  property("middleNames") =
    Prop.forAll {
      (firstName: String, middleName: Option[String], lastName: String) =>
        middleName match {
          case Some(x) =>
            val artist = new Artist(firstName, x, lastName)
            artist.fullName == firstName + " " + x + " " + lastName
          case _ =>
            val artist = new Artist(firstName, lastName)
            artist.fullName == firstName + " " + lastName
        }
    }
}

ScalaCheck is very thorough with string testing. In the property we just defined, some of the values created are pretty random characters, not limited to those in the Latin character code. A sample follows of a first name, middle name, and last name set created by ScalaCheck.

(ฦꝒܷ㾯琗ꨮ攖쎁䕗뙋ʹᛵƹỳ섢⁛˞ᘦ䉽᫃ᗤ沔㳵험쮮뱆৊뇀叜칩㫂ឝ魯
ࢶ姅ᾘ懹컚ꣀ舶푫㈏᳧䥳ᶑ⩎ই댗溿ឪ䖙湹클뢵흰烐帘꣈攒′敪㤼驹깶滷⻌,Some(氷ࣈﱊ衾銠ხ뻃ᢉ
趼嬓ᗚ벇돡퓑昁䛄ᄧ偵੼馿嵢떀ꥆꬷ踱왾侊霳焗铻뫮ᗣ잵詩ꢳ⢟솖搾ꤗ챭튊擓勱䓜秤枴솸㭊꣐㠒Ṍ黗墘侺
⮞﹛눏쇖㚉﹧葻ﶬ⾐),ᛎ適趹뫊鶵鎺ⶸ햮쁿瀌仹喢輕걑鱳멟廕爆령鲌˒둯꡾댁Ꟙ㴳뤶)

In fact, come to think of it, this doesn’t contain many Latin/English characters at all! ScalaCheck does well at providing even the most esoteric data, which will add to the confidence level of any test. In contrast, it is highly unlikely that a test developer would even consider randomly chosen Unicode characters for a string, unless they require a very internationalized application.

Of course, that’s a lot of weird data for most tests, and the data sometimes needs to be constrained to give a more realistic test set. If you know that a test requires only alphanumeric characters, only positive integers, etc, ScalaCheck makes it easy to customize the data provided.

The next sample object uses additional Gen parameters to change the type of data given to the test. Prop.forAll now takes three additional parameters that correspond to the first name, middle name, and last name it will create. The first parameter requests that the first element be an alphabetical string. The second allows either Some with an alphabetical string or None for the second element. The third parameter, like the first, chooses an alphabetical string. The println statement is included just to print a sample of the data that ScalaCheck is creating, so you can verify that all the data consists of ASCII letters. The rest of the code is the same as before.

src/test/scala/com/oreilly/testingscala/ArtistScalaCheckProperties.scala. 

object ArtistScalaCheckProperties extends Properties("Testing Artists Thoroughly") {
  property("middleNames") =
    Prop.forAll (Gen.alphaStr, Gen.oneOf(Gen.alphaStr.sample, None), Gen.alphaStr) {
      (firstName: String, middleName: Option[String], lastName: String) =>
        println(firstName, middleName, lastName)
        middleName match {
          case Some(x) =>
            val artist = new Artist(firstName, x, lastName)
            artist.fullName == firstName + " " + x + " " + lastName
          case _ =>
            val artist = new Artist(firstName, lastName)
            artist.fullName == firstName + " " + lastName
        }
    }
}

A resulting data looks like the following, which is vastly different from some of the varying global character sets. The first sample shown has a first and last name, with no middle name. The second example is the same but with longer string length, and finally comes a case with all three names: first, middle, and last.

(tvzoTzJppo,None,gygoprpyzw)
(OzebmzjbbovreytrmsfwuwfbsmlvjkzutwcbbfspqJhrjqqwdaveArsel,None,gbuxlswkfhyeyplrtzKkasfklrzkjaktygrzucftfhlfeeuxlleoin)
(zfvtoVki,Some(ggohbctymlkjsrmmprcRigdiqmygtfDmsknwcoikzbhzrfwuoNgrNwjcjmohccznrbzldiRcmcGscambzaporrmnc),noodpvKwusaRwzimZxujgqvknnlfgqVq)

The end result of the test is pure success with 100 varying tests.

[info] + Testing Artists Thoroughly.middleNames: OK, passed 100 tests.
[info] Passed: : Total 1, Failed 0, Errors 0, Passed 1, Skipped 0
[success] Total time: 4 s, completed Jan 12, 2012 12:10:35 AM

A different way to constrain a test is by using a conditional property. Conditional properties are essentially filters where you stipulate what is considered good data for the test. In the next example, we may want to test numbers against particularly high or particularly low values.

The following example limits the year an album was made to within the 20th and 21st centuries. Our constraints would need a number from 1900 to maybe 3000, since we may have already taken care of validating the year of an album’s creation. The year 3000 was created arbitrarily.

Using constraints in ScalaCheck requires only the use of a new ==> operator, an implication operator that divides the filtering logic from the test itself.

src/test/scala/com/oreilly/testingscala/AlbumScalaCheckProperties.scala. 

package com.oreilly.testingscala

import org.scalacheck.{Prop, Properties}
import org.scalacheck.Prop._

object AlbumScalaCheckProperties extends Properties("Album Creation") {
  property("album can be created using a year from 1900 to 3000") =
    Prop.forAll {
      (title: String, year: Int, firstName: String, lastName: String) =>
        (year > 1900 || year < 3000) ==> {
          val album = new Album(title, year, new Artist(firstName, lastName));
          album.year == year
          album.title == title
        }
    }
}

This example runs through a list of random string data, and a year that is constrained between 1900 and 3000 is stipulated by the implication operator.

[info] Compiling 1 Scala source to /home/danno/testing_scala_book.git/testingscala/target/scala-2.9.0-1/test-classes...
[info] + Album Creation.album can be created using a year from 1900 to 3000: OK, passed 100 tests.
[info] Passed: : Total 1, Failed 0, Errors 0, Passed 1, Skipped 0
[success] Total time: 4 s, completed Jan 12, 2012 12:47:03 AM

A Properties object can contain one or more property tests. Prop objects can also be combined to craft a larger composed Prop. The following example employs fairly simple tests and creates stringsOnly, which creates String objects and tests if the size of the string is greater than 0. positiveNumbersOnly uses a positive number Gen to provide positive numbers and a positiveNumbers2 generator to do the same thing. alwaysPass and wontPass are Props that generate tests that always pass and always fail. Each Prop is assigned to a variable, and each variable is then combined with &&, ||, and == to determine if the tests pass or not.

src/test/scala/com/oreilly/testingscala/ArtistScalaCheckProperties. 

package com.oreilly.testingscala

import org.scalacheck.{Gen, Prop, Properties}
import org.scalacheck.Prop._

object CombiningGenScalaCheckProperties extends Properties("Combining Properties") {
  val stringsOnly = Prop.forAll(Gen.alphaStr) {
    x: String => (x != "") ==> x.size >= 0
  }
  val positiveNumbersOnly = Prop.forAll(Gen.posNum[Int]) {
    x: Int => x >= 0
  }
  val positiveNumbers2Only = Prop.forAll(Gen.posNum[Int]) {
    x: Int => x > 0
  }

  val alwaysPass = Prop.forAll {
    x: Int => true
  }

  val wontPass = Prop.forAll((x: Int, y: Int) => x + y > 0)

  property("And") = stringsOnly && positiveNumbersOnly
  property("Or") = stringsOnly || wontPass
}

When run, property("And") passed, since stringsOnly and positiveNumbersOnly passed. property("Or") passed, since stringsOnly passed and wontPass didn’t. Of course, since we are using an or operator with the || short circuit operator becomes a successful test.

The ArtistScalaCheckProperties object in the previous section contained three Gen objects. Here are some more Gen methods that give you a wide range of options.

Gen.value merely returns the value that it contains. For instance, suppose that a test should succeed when handed a String of Orinoco Flow. This can be useful when you wish to generate a fixed value as part of a larger random data sample. The following overly simple test implements this.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.value("Orinoco Flow")) { _ == "Orinoco Flow"}

Gen.chooses provides a value in a range between two items, inclusively. In the following example, a number is chosen randomly between 1 and 52, so that the value can be used in a test. What gets selected in Gen.choose has to make sense.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.choose(1, 52)) {
   card => card < 53 && card > 0
}

Gen.choose with two Strings would not work out of the box, since there is there is no set of rules established on how to create words—say, between Foo and Grok. That is not to say it is impossible. If you want to use your operating system’s dictionary (e.g. /usr/share/dict in Linux, contains an American-English dictionary), you can plug that into ScalaCheck to create random English words to test your code. We will see later some options you can use to customize your test harness to make that possible.

Gen.chooses requires values that have some sort of beginning and end. For a String we can use ScalaCheck’s Choose implicit object to customize the behavior.

Gen.oneOf wraps other Gen objects and randomly selects one for each data set generated. The following small example can generate either a number from 0 to 3 or the string Aretha Franklin.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.oneOf(Gen.choose(-2, 3), Gen.value("Aretha Franklin"))) {
  _ match {
    case y: Int => (0 to 3).contains(y)
    case z: String => z == "Aretha Franklin"
  }
}

Gen.listOfN generates a list of the size provided in its first parameter, containing random values. This is perfect for randomly generated strings that must all be the same size. The following example creates a swath of random lists containing numbers from 20 to 60, but each list will have only four elements.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.listOfN(4, Gen.choose(20, 60))) {
   x => (x.size == 4) && (x.sum < 240)
}

Gen.listOf create a random lists with random sizes, giving production code a great workout if it involves List. The following example creates varying-sized lists with numbers ranging from 20 to 60.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.listOf(Gen.choose(20, 60))) {
   x =>
     if (x.size > 0) x(0) > 19 && x(0) < 61
     else true
}

The previous test is fairly benign, since it doesn’t quite test anything and is just shown to illustrate the listOf method. If the list provided by ScalaCheck contains more than one element, we just make sure the first element is within the range, otherwise true.

Prop has a nifty method called classify that can be used in such cases to show the distribution of the test data. classify takes a Boolean value and a label that will be displayed in the test output when the generated data matches the Boolean value. Rewriting the previous example as a classified test makes it a bit more useful.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.listOf(Gen.choose(20, 60))) {
  x =>
    classify((x.size >= 0) && (x.size < 50), "0 to 50") {
      classify((x.size >= 50) && (x.size < 100), "50 - 100") {
        classify((x.size >= 100), "100 or more") {
            true
        }
      }
    }
}

The report will break down the test into categories used to decipher the test data. This can give you an idea of the distribution of values used in the test. The previous example used three classify methods to break the output data down by length of list. Sample output follows. It turns out that this test run didn’t generate any lists containing more than 100 elements.

[info] + Various Gen Properties.Gen.ListOf (Random) and classified: OK, passed 100 tests.
[info] > Collected test data:
[info] 84% 0 to 50
[info] 16% 50 - 100

The previous list methods can create empty lists, which is important in many tests. But Gen.listOf1 creates a randomly generated list with at least one element.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.listOf1(Gen.choose(20, 60))) {
  _.size > 0
}

The assertion in the test proves that each list generated contains more than zero elements.

Gen supports other containers besides lists. In the next example, instead of a List, Gen.containerOf is used to create randomly sized Sets. The Gen.containerOf method takes a collection-type parameter to specify what type of container is needed for the test. All the listOf methods defer internally to containerOf for their construction.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.containerOf[Set, Int](Gen.choose(1, 5))) {
  x => true
}

Prop.forAll(Gen.containerOf1[Set, Int](Gen.choose(1, 5))) {
  _.size > 0
}

Prop.forAll(Gen.containerOfN[Set, Int](4, Gen.choose(1, 5))) {
  x => (x.size <= 4) && (x.sum < 240)
}

The last of the examples specifies a Set of Int with another Gen constricting the values used between 1 and 5, inclusive. The sample asserts that the size is less than or equal to 4.

containerOfN specifies the size of each container generated, and containerOf1 specifies a container that has at least one element.

You can also use a function as a generator through Gen.resultOf. It provides random data of the types you specify as parameters to the function, which will be used later inside the test block for assertions. For instance, the following contrived example contains a Gen.resultOf function that accepts an Int parameter and a String parameter. The parameters are generated by ScalaCheck. resultOf in turn creates the Map[Int, String] that is used inside the test block. In the test block, an assertion is made that that the function parameter provided is indeed a test of Map[Int, String].

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.resultOf((x: Int, y: String) => Map(x -> y))) {
  p => println(p); p.isInstanceOf[Map[_,_]]
}

The above test will create a range of testing data that looks like Map(1029697308 -> 鯁垞펕돀)

If you want complete control over the distribution of the test data, ScalaCheck provides a frequency generator so that the data given to the test block is of the right proportion.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.frequency(
  (3, Gen.value("Phoenix")),
  (2, Gen.value("LCD Soundsystem")),
  (5, Gen.value("JJ")))) { ... }

The previous example creates a test that distributes the data according to the weighted calculation of the frequency. The first frequency method parameter, (3, Gen.value("Phoenix")), gives a weight of 3 to the value "Phoenix“. (2, Gen.value("LCD Soundsystem")) assigns a weight of 2 to the value "LCD Soundsystem“, while (9, Gen.value("JJ")) assigns a weight of 9 to the value "JJ“. In other words, you want three times as many values of 10 as you want values of 4. The weight is merely a suggestion, and not an exact percentage of the resulting data sets. Next we will classify the frequencies, so we can see an example of the actual distribution of the data.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.frequency(
  (3, Gen.value("Phoenix")),
  (2, Gen.value("LCD Soundsystem")),
  (5, Gen.value("JJ")))) {

  x =>
    classify(x == "Phoenix", "Phoenix") {
      classify(x == "LCD Soundsystem", "LCD Soundsystem") {
        classify(x == "JJ", "JJ") {
          true
        }
      }
    }

Running the sample provides the following test output. As you can see, we’ve come close to the percentages requested, but we didn’t hit them on the nose.

[info] 47% JJ
[info] 34% Phoenix
[info] 19% LCD Soundsystem

ScalaCheck has various options that can be used either at the command prompt or in SBT to change the way ScalaCheck behaves. Table 6-1 shows some of these options.

The actual number of data sets is specified by -maxSize and -minSize. -minSize defaults to zero (so ScalaCheck could theoretically tell you everything ran fine without testing any data sets at all) while -maxSize defaults to 100. Although these values are set on the command line, you can access them within the test through the Gen.sized generator.

src/test/scala/com/oreilly/testingscala/VariousGenCheckProperties.scala. 

Prop.forAll(Gen.sized(x => Gen.listOfN(x, Gen.value("*")))) {
   x => println(x.size + " " + x); true
}

This test will use the maxSize and minSize variable information as part of generating the data. This example creates a list of asterisks with a size; if this were run as is without maxSize and minSize, there would be mess in the console showing a whole lot of stars! If it were run with some minSize and a maxSize value of something small, like 4 or 5, then it would show some short lists with a maximum size of 5 and a minimum of 4. Confused? If so, run the command in SBT:

~test-only com.oreilly.testingscala.VariousGenCheckProperties -- -minSize 3 -maxSize 5

The result is you will get 100 tests. You will always get 100 tests. But the kind of tests that you will get will be one of the following combinations:

3 List(*, *, *)
4 List(*, *, *, *)
5 List(*, *, *, *, *)

Of course, if you run some of these examples, you’ll notice that they still generate 100 samples. At the time of writing for this book, there is nothing that can be done about that. There will always be a sample of 100, and the developer would have to bear with it until a new version comes around, when perhaps that number can be constrained as we choose.

val gen = for {
  x <- Gen.choose(3, 300)
  y <- Gen.alphaStr
} yield Map(x -> y)

Prop.forAll(gen) {...}

Gen.choose(3, 300).flatMap(x => Gen.alphaStr.map( y => Map(x -> y)))

val entries = for {
  y <- Gen.alphaStr
  x <- Gen.choose(3, 300)
} yield (x -> y)

val maps = for {
  x <- Gen.listOfN(4, entries)
} yield (Map(x: _*))

Prop.forAll(maps) {...}

val albums = for {
  a <- Gen.alphaStr
  b <- Gen.choose(1900, 2012)
  c <- Gen.alphaStr
} yield (new Album(a, b, new Band(c)))

val listOfAlbums = Gen.listOf(albums)
Prop.forAll(listOfAlbums) {
  albums =>
    val jukebox = new JukeBox(Some(albums))
    jukebox.albums.get.size == albums.size
}

Generators, as seen, can do quite a bit, but once a developer has settled on a particular generator it would be nice not to need to declare or specify the generator every time you want to use it. Arbitrary uses Scala’s implicit variables and methods so these values can always be available without any added programming.

For example, say that in the company that makes Album objects and JukeBox objects, a Gen[Album] has already been established and is located in some object named com.oreilly.testingscala.AlbumGen. Using Arbitrary, AlbumGen only needs to import com.oreilly.testingscala.AlbumGen._ to make use of the new Gen. Once that is done, the test-driven developer merely has to create a Prop.forAll with no Gen programming, and test randomly distinct data whenever necessary. The following example shows how it is done.

property("Arbitrary Creating Objects") = {
  implicit val album: Arbitrary[Album] = Arbitrary {
    for {
      a <- Gen.alphaStr
      b <- Gen.choose(1900, 2012)
      c <- Gen.alphaStr
    } yield (new Album(a, b, new Band(c)))
  }

  Prop.forAll {album: Album => album.ageFrom(2012) == (2012 - album.year)}
}

The variable album is implicit, which means that it is available in scope for anything that requires that type of signature. Since the variable album is an Arbitrary[Album], if any Prop.forAll requires an Album it will automatically pull from the implicit declaration without any extra work. The benefit gained is shown in the Prop.forAll statement at the end of the example. The test block requires an album variable, and no Gen had to be specified.

Tests in ScalaCheck can have labels appended to each assertion. The benefit is that ScalaCheck can reference the actual assertion by its label. Consider the following example without a label.

Prop.forAll {
  (x:Int, y:Int) => (x > 0 && y > 0) ==> {
      (x + y) != 0 && (x+y) > 0 && (x+y) < (x+y)
  }
}

The test block in the last line of code contains three very different assertions. The last assertion, (x+y) < (x+y), will fail. As it stands, there is no way of knowing which assertion is the one that failed. Running the example will render the test as a failure and say which arguments have failed, but not state which assertion actually failed.

[info] ! Various Gen Properties.Compound assertions without labels: Falsified after 0 passed tests.
[info] > ARG_0: 6
[info] > ARG_1: 1

ARG_0 specifies that x is 6, and ARG_1 specifies that y is 1. That’s all ScalaCheck provides without the use of labels. For better reporting, apply a ScalaCheck label to each of the assertions and give it a name.

Prop.forAll {
  (x: Int, y: Int) => (x > 0 && y > 0) ==> {
      ((x + y) != 0)   :| "(x+y) equals (y+x)" &&
      ((x+y) > 0)      :| "(x+y) > 0" &&
      ((x+y) < (x+y))  :| "(x+y) < (x+y)"
  }
}

Now there is some clarity as to which of the assertions has failed.

[info] ! Various Gen Properties.Compound assertions with labels: Falsified after 0 passed tests.
[info] > Labels of failing property:
[info] (x+y) < (x+y)
[info] > ARG_0: 48
[info] > ARG_1: 1

Here’s a bit more about the use of the label tags. The : will always associate with the assertion, and the | will always associate with the label. && is irrelevant to the label, since, as previously seen, it is used to join each of the Boolean assertions. If you want to have the label come first instead of the assertion, use |:. Which to use is up to you.

Prop.forAll {
  (x: Int, y: Int) => (x > 0 && y > 0) ==> {
    ("(x+y) equals (y+x)" |: ((x + y) != 0))  &&
    ("(x+y) > 0"          |: ((x+y) > 0))     &&
    ("(x+y) < (x+y)"      |: ((x+y) < (x+y)))
  }
}

Note that in this example, parentheses are required around the combination of assertion and label.

If you need evidence of which values were actually used, or, if you need to make any intermediate calculations to answer the question “How did we get to this point?”, assign the intermediate calculation to a variable. Returning to the previous simple example, suppose you wish to inquire what (x+y) equals before running the test. The test can be refactored to give a label to the intermediate result so you can trace where the problem in the test occurred.

Prop.forAll {
  (x: Int, y: Int) => ((x > 0) && (y > 0)) ==> {
    val result = x + y  //intermediate result
      ("result = " + result) |: all(
      ("(x+y) equals (y+x)"  |: (result != 0))    &&
      ("(x+y) > 0"           |: (result > 0))     &&
      ("(x+y) < (x+y)"       |: (result < result))
    )
  }
}

This is essentially the same test, with two generated values constrained so they are both positive. But this time a result is calculated as an extra variable, result. This lets you track down the intermediate result when investigating a failed test. A string printing result is attached to a series of other labeled assertions using the |: all (..) construct. Given this method, result can displayed along with each test.

[info] ! Various Gen Properties.Compound assertion labelling with evidence: Falsified after 0 passed tests.
[info] > Labels of failing property:
[info] (x+y) < (x+y)
[info] result = 27
[info] > ARG_0: 26
[info] > ARG_1: 1

ScalaCheck is an indispensable tool. Using it, we no longer need to come up with data for classes. Mocks and ScalaTest are all we need to quickly generate tests and keep moving. Next we’ll look at ScalaCheck in use with ScalaTest, and explore some of the enhancements made to accompany ScalaCheck.

ScalaTest offers some sugar to make ScalaCheck code a bit more fluent and readable. Using any one of ScalaTest’s Specs, extend the GeneratorDrivenPropertyChecks trait to integrate ScalaCheck testing.

package com.oreilly.testingscala

import org.scalatest.matchers.ShouldMatchers
import org.scalatest.Spec
import org.scalatest.prop.GeneratorDrivenPropertyChecks
import org.scalacheck.Gen

class ScalaTestWithScalaCheck extends Spec with ShouldMatchers with GeneratorDrivenPropertyChecks {
}

The following example shows a basic ScalaCheck test used inside of ScalaTest.

class ScalaTestWithScalaCheck extends Spec with ShouldMatchers with GeneratorDrivenPropertyChecks {
  describe("We can use test data from Scala check") {
    it("runs the same but with different constructs") {
      forAll {
        (a: Int, b: Int) =>
          (a + b) should be(b + a)
      }
    }
 }

There isn’t much difference between this ScalaTest code and a plain ScalaCheck test. The forAll method in this example is a ScalaTest method, not a ScalaCheck method, but as far as usability, it remains nearly the same. What is different is how ScalaTest deals with constraining some of these properties.

class ScalaTestWithScalaCheck extends Spec with ShouldMatchers with GeneratorDrivenPropertyChecks {
  describe("We can use test data from Scala check") {
   it("runs constraints but differently") {
      forAll {
        (a: Int, b: Int) =>
          whenever(b > 14) {(a + b) should be(b + a)}
      }
    }
 }

In this example, forAll is used like a ScalaCheck conditional property. This time, though, the term whenever replaces the ==> overloaded operator. This test will run for all integers except when b is greater than 14.

ScalaTest cleans up a lot of the funky operators when it deals with failure labels. Previously, labels used either |: or :| operators to label each assertion, so a test could report which of them had failed. ScalaTest offers an alternative that eliminates the need to write explicit labels. Instead, ScalaTest relies on its built-in reporting. First, as a refresher, here is the test that was used in ScalaCheck with labels.

Prop.forAll {
 (x: Int, y: Int) => (x > 0 && y > 0) ==> {
     ((x + y) != 0)   :| "(x+y) equals (y+x)" &&
     ((x+y) > 0)      :| "(x+y) > 0" &&
     ((x+y) < (x+y))  :| "(x+y) < (x+y)"
 }
}

ScalaTest uses its own reporting mechanism instead of labels. The only thing required is to use either a MustMatcher or a ShouldMatcher with a compound and or or operator.

forAll {
  (x: Int, y: Int) =>
    whenever (x > 0 && y > 0) {
      (x + y) should (not be (0) and ((be > 0) and (be < (x+y))))
  }
}

As seen in the following result, ScalaTest provides more-than-adequate information to dissect the issues with the test. The message line includes a clause for each of the checks separated in the code by and operators.

[info] - no need for test labels *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]   Message: 2 was not equal to 0, but 2 was greater than 0, but 2 was not less than 2
[info]   Location: (ScalaTestWithScalaCheck.scala:30)
[info]   Occurred when passed generated values (
[info]     arg0 = 1,
[info]     arg1 = 1 // 29 shrinks
[info]   )

ScalaTest has another nice feature: labeling properties. In the previous output, arg0 and arg1 describe which values violated the test. ScalaTest with ScalaCheck gives the developer the ability to rewrite the test with variable labels. The following example extends the previous example to show this.

forAll ("x", "y") {
  (x: Int, y: Int) =>
    whenever (x > 0 && y > 0) {
      (x + y) should (not be (0) and ((be > 0) and (be < (x+y))))
    }
}

forAll uses two variables called x and y, and the names provide a nice way to display the output of the test, giving the developer a better chance at finding issues. Compare the following output with the output of the previous example. Instead of arg_0 it shows the name of the variable, x, and instead of arg_1 it shows y. Glorious!

[info]   TestFailedException was thrown during property evaluation.
[info]   Message: 9 was not equal to 0, but 9 was greater than 0, but 9 was not less than 9
[info]   Location: (ScalaTestWithScalaCheck.scala:39)
[info]   Occurred when passed generated values (
[info]     x = 8, // 39 shrinks
[info]     y = 1 // 29 shrinks
[info]   )

Shrinks in ScalaCheck and in this example are minimizations of the values that fail the test. Why? Because it is easier to determine that a test failed with 8 and 1 than with 10,321 and 948. You can create your own minimization strategies in ScalaCheck if you wish. Due to constraints on the size of this book, it will not be covered.

forAll(Gen.choose(10, 20), Gen.choose(30, 40)) {
    (a: Int, b: Int) =>
       (a + b) should equal((a + b)) // Should fail
}

forAll((Gen.choose(10, 20), "a"), (Gen.choose(30, 40), "b")) {
  (a: Int, b: Int) =>
    (a + b) should equal(a + b + 1)
}

[info] - runs with labels and generators *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]   Message: 44 was not equal to 45
[info]   Location: (ScalaTestWithScalaCheck.scala:62)
[info]   Occurred when passed generated values (
[info]     a = 15, // 1 shrink
[info]     b = 29 // 1 shrink
[info]   )

import com.oreilly.testingscala.AlbumGen._
forAll {(a:Album, b:Album) =>
   a.title should not be (b.title + "ddd")
}

Specs2 allows ScalaCheck to live within its specifications, and offers some slight modifications to work with properties. First let’s look at a specification.

package com.oreilly.testingscala

import org.specs2.ScalaCheck
import org.specs2.mutable.Specification
import org.scalacheck.Prop._
import com.oreilly.testingscala.AlbumGen._
import org.scalacheck.{Arbitrary, Gen, Prop}

class Specs2WithScalaCheck extends Specification with ScalaCheck {

  "Using Specs2 With ScalaCheck".title ^
  "Can be used with the check method" ! usePlainCheck

  def usePlainCheck = check((x: Int, y: Int) => {
    (x + y) must be_==(y + x)
  })

This is the shell required for using ScalaCheck within Specs2. Most information about the specification has already been covered in Specs2. The only difference is that the ScalaCheck trait is included to use some of the sugars that ease testing.

The code shows a simple test named usePlainCheck, called from the specification. It also uses a Specs2 method called check instead of Prop.forAll to run the very simple test. As with ScalaTest, in Specs2 it is preferred for the assertion to use the Specs2 matcher syntax. The use of check is not mandatory, and Prop.forAll can also be used, which is the preferred way to include generators in a test.

Constraints in Specs2 use the ==> operator, just as in a plain ScalaCheck property.

package com.oreilly.testingscala

import org.specs2.ScalaCheck
import org.specs2.mutable.Specification
import org.scalacheck.Prop._
import com.oreilly.testingscala.AlbumGen._
import org.scalacheck.{Arbitrary, Gen, Prop}

class Specs2WithScalaCheck extends Specification with ScalaCheck {

  "Using Specs2 With ScalaCheck".title ^
  "Can be used with the check method" ! usePlainCheck
  "Can be used with constraints" ! useCheckWithConstraints

  def useCheckWithConstraints = check {
    (x: Int, y: Int) => ((x > 0) && (y > 10)) ==> {
      (x + y) must be_==(y + x)
    }
  }
}

There is nothing different in this example, except that it uses check instead of a Prop.forAll.

When using generators, on the other hand, stick to forAll instead of check. The reason is that forAll supports Gen parameters, whereas check does not.

package com.oreilly.testingscala

import org.specs2.ScalaCheck
import org.specs2.mutable.Specification
import org.scalacheck.Prop._
import com.oreilly.testingscala.AlbumGen._
import org.scalacheck.{Arbitrary, Gen, Prop}

class Specs2WithScalaCheck extends Specification with ScalaCheck {

  "Using Specs2 With ScalaCheck".title ^
  "Can be used with the check method" ! usePlainCheck
  "Can be used with constraints" ! useCheckWithConstraints
  "Can be used with generators" ! useGenerators

  //This is a workaround
  implicit val foo3: (Unit => Prop) = (x: Unit) => Prop(Prop.Result(Prop.True))

  //Code removed for brevity

  def useGenerators = forAll(Gen.containerOfN[Set, Int](4, Gen.choose(20, 60))) {
    x => x.size must be_<= (4) and (x.sum must be_< (240))
  }
}

Arbitrary can be used in Specs2 in much the same way as in a ScalaCheck property.

package com.oreilly.testingscala

import org.specs2.ScalaCheck
import org.specs2.mutable.Specification
import org.scalacheck.Prop._
import com.oreilly.testingscala.AlbumGen._
import org.scalacheck.{Arbitrary, Gen, Prop}

class Specs2WithScalaCheck extends Specification with ScalaCheck {

  "Using Specs2 With ScalaCheck".title ^
  "Can be used with the check method" ! usePlainCheck
  "Can be used with constraints" ! useCheckWithConstraints
  "Can be used with generators" ! useGenerators
  "Can be used with Arbitrary in the same way" ! useArbitrary

  //Code omitted for brevity
  def useArbitrary = check((album: Album) => album.ageFrom(2012) must be_==(2012 - album.year))
}

The Arbitrary[Album], as before, is an imported method from the AlbumGen object, so there is no need for a Gen object in the test. This example uses the Specs2 check method; the Arbitrary[Album] will take care of the rest.

Specs2 has an alternate way to use an Arbitrary object in a test. The alternative way is to use the Arbitrary object in place of forAll or check. This makes the Arbitrary object the test method itself. Consider the following example.

package com.oreilly.testingscala

import org.specs2.ScalaCheck
import org.specs2.mutable.Specification
import org.scalacheck.Prop._
import com.oreilly.testingscala.AlbumGen._
import org.scalacheck.{Arbitrary, Gen, Prop}

class Specs2WithScalaCheck extends Specification with ScalaCheck {

  "Using Specs2 With ScalaCheck".title ^
  "Can be used with the check method" ! usePlainCheck
  "Can be used with constraints" ! useCheckWithConstraints
  "Can be used with generators" ! useGenerators
  "Can be used with Arbitrary in the same way" ! useArbitrary
  "Can be used with Arbitrary in a clever way" ! useAnArbitraryInACleverWay

  val mapIntString = Arbitrary {
    for {
      x <- Gen.choose(3, 300)
      y <- Gen.alphaStr
    } yield Map(x -> y)
  }

  //previous code removed for brevity
  def useAnArbitraryInACleverWay = mapIntString {
    (x: Map[Int, String]) => x.size must be_==(1)
  }
}

This example creates an Arbitrary object of type Map[Int, String] and does not bind implicitly in the scope. This can provide some benefit, because there is no need to commit an implicit object to scope, and you can use Arbitrary freely by creating a variable and using it in a test method. In other words, one Arbitrary of a particular type can be used for one method, and another Arbitrary of the same type but a different implementation can be used for a subsequent method.

Specs2’s and ScalaTest’s use of ScalaCheck makes testing and creating fake data a breeze, possibly cutting down the time required to create the data. Whether you choose Specs2 or ScalaTest is based on your preference. Overall, Scala’s testing frameworks make testing Scala and Java a pleasure. May we never make up fake data again by letting ScalaCheck construct all that we need.