Chapter 3
ScalaCheck Fundamentals
The two most fundamental concepts in ScalaCheck are properties and generators. This chapter will introduce the classes that represent properties in ScalaCheck, and bring up some technical details about the API. A following chapter will then present the multitude of different methods that exists in ScalaCheck's API for constructing properties.
Generators are the other important part of ScalaCheck's core. A generator is responsible for producing the data passed as input to a property during ScalaCheck's verification phase. Up until now we have sort of glanced over how ScalaCheck actually comes up with the values for the abstract arguments that your properties state truths about. The second part of this chapter will show what a generator is and demonstrate situations where you can make more explicit use of them.
The final section will talk about ScalaCheck's test case simplification feature, that was briefly mentioned in Chapter 1.
3.1 The Prop and Properties classes
A single property in ScalaCheck is the smallest testable unit. It is always represented by an instance of the org.scalacheck.Prop class.
The common way of creating property instances is by using the various methods from the org.scalacheck.Prop module. Here are some ways of defining property instances:
import org.scalacheck.Prop
val propStringLength = Prop.forAll { s: String => val len = s.length (s+s).length == len+len }
val propDivByZero = Prop.throws(classOf[ArithmeticException]) { 1/0 }
val propListIdxOutOfBounds = Prop.forAll { xs: List[Int] => Prop.throws(classOf[IndexOutOfBoundsException]) { xs(xs.length+1) } }
The first property is created by using the Prop.forAll method that you have seen several times before in this book. The second property uses Prop.throws that creates a property that tries to run a given statement each time the property is evaulated. Only if the statement throws the specified exception the property passes. The property propListIdxOutOfBounds in the example above shows that Prop.forAll not only accepts boolean conditions, but you can also return another property that then must hold for all argument instances.
The property values above are instances of the Prop class, and you can give the values to ScalaCheck's testing methods to figure out whether or not they pass.
When defining several related properties, ScalaCheck also has a class named org.scalacheck.Properties that can be used to group a bunch of properties together. It provides a way to label the individual property instances, and makes it easier for ScalaCheck to present the test results in a nice way. Using the Properties class is the preferred way of defining properties for your code. The code below shows how to use Properties to define a set of properties.
import org.scalacheck.Properties import org.scalacheck.Prop.{forAll, throws}
object MySpec extends Properties("MySpec") {
property("list tail") = forAll { (x: Int, xs: List[Int]) => (x::xs).tail == xs }
property("list head") = forAll { xs: List[Int] => if (xs.isEmpty) throws(classOf[NoSuchElementException]) { xs.head } else xs.head == xs(0) }
}
The Properties.property method is used to add named properties to the set. If we check the property collection in the Scala console we can see the names printed:
scala> MySpec.check + MySpec.list tail: OK, passed 100 tests. + MySpec.list head: OK, passed 100 tests.
You mostly don't need to handle individual property instances, but sometimes it can be useful to reuse parts of properties, or combine several properties into one. For example, there is a && operator that creates a new property out of two other property instances. All the operators and methods that can be used to create properties are defined in the org.scalacheck.Prop module, and most of them are described in Chapter 5.
3.2 Generators
Up until now, we have never been concerned with how data is generated for our properties. Through the Prop.forAll method, we have simply told ScalaCheck to give us arbitrary strings, integers, lists, and so on, and ScalaCheck has happily served us the data when the properties have been evaluated.
However, sometimes we want a bit more control over the test case generation. Or we want to generate values of types that ScalaCheck know nothing about. This section will introduce the generators and show how you can make more explicit use of them.
The Gen class
A generator can be described simply as a function that takes some generation parameters and produces a value. In ScalaCheck, generators are represented by the Gen class, and the essence of this class looks like this:
class Gen[+T] { def apply(prms: Gen.Params): Option[T] }
As you can see, a generator is parameterized over the type of values it produces. In ScalaCheck, there are default Gen instances for each supported type (Gen[String], Gen[Int], Gen[List], etc.). You can also see that the Gen.apply method returns the generated value wrapped in an Option instance. The reason for this is that sometimes a generator might fail to generate a value. In such cases, None will be returned. I will get back to why generators might fail in Chapter 6.
Normally, you don't deal with the Gen class explicitly, even when creating custom generator instances. Instead, you use one or more of the many methods in the module org.scalacheck.Gen. This module is quite independent from the other parts of ScalaCheck, so if you want you can use the generators in a project of your own just for data generation purposes, not only in the ScalaCheck properties you specify.
Let's fire up the Scala interpreter, define a generator, and see how to generate a value with it:
scala> import org.scalacheck.Gen import org.scalacheck.Gen
scala> val myGen = Gen.choose(1,10) myGen: org.scalacheck.Gen[Int] = Gen()
scala> myGen(Gen.Params()) res0: Option[Int] = Some(7)
scala> myGen.sample res1: Option[Int] = Some(5)
First, we imported the Gen module. Then we created a generator, myGen, using the Gen.choose method. This method creates generators that will generate random numbers in the given inclusive range. We can see from the type Gen[Int] of myGen that it will generate integers.
Finally, we used myGen to generate values in two different ways. In the first example, we can see how closely a generator resembles a function. We just apply the default generation parameters that are defined in Gen, and we get the generated value in return. In the second example, we use the sample method that exists on every generator; it is a convenient way of doing exactly the same thing.
In the example, you can also see that the generator returns its value as an Option type, which was mentioned previously. The generators you can create by using the Gen.choose method will never fail, but will always deliver a Some-instance containing a value.
The parameters a generator uses to generate data contain information about which random number generator should be used and how large the generated data should be. Chapter 6 will describe the parameters more closely; for now, you can just use Gen.Parameters() or the sample method as shown previously.
Defining custom generators
As I've mentioned, there are many methods you can use to create your own generators in the Gen module. These methods are called combinators, since you can use them as basic building blocks for generating more complex structures and classes. To combine them together, you use Scala's versatile for statement, which is mostly used in loop constructs but in fact is much more general. Here is an example of its use with generators:
import org.scalacheck.Gen.choose
val myGen = for { n <- choose(1, 50) m <- choose(n, 2*n) } yield (n, m)
In this example, myGen generates randomized tuples of integers, where the second integer always is larger than or equal to the first, but not more than twice as large. With the sample method, we can check that it is working as expected:
scala> myGen.sample res0: Option[(Int, Int)] = Some((45,60))
scala> myGen.sample res1: Option[(Int, Int)] = Some((29,37))
You can define generators to build any structure. Consider the following simple types that model shapes and color:
trait Color case object Red extends Color case object Green extends Color
trait Shape { def color: Color } case class Line(val color: Color) extends Shape case class Circle(val color: Color) extends Shape case class Box(val color: Color, val boxed: Shape) extends Shape
We can now define generators for the Color and Shape types:
import org.scalacheck.Gen
val genColor = Gen.oneOf(Red, Green)
val genLine = for { color <- genColor } yield Line(color) val genCircle = for { color <- genColor } yield Circle(color) val genBox = for { color <- genColor shape <- genShape } yield Box(color, shape)
val genShape: Gen[Shape] = Gen.oneOf(genLine, genCircle, genBox)
In this example, we used Gen.oneOf, which takes an arbitrary number of generators (or plain values) and creates a new generator that will use one of the provided generators at random when it is evaluated. As you can see, genBox and genShape are recursive generators. There are some things you should be aware of when defining recursive generators in order to not cause infinite recursions and huge data structures. This will be covered in Chapter 6. However, the above generator definition should be just fine, because it converges quickly as we can see when we try it out:
scala> genShape.sample res0: Option[Shape] = Some(Line(Green))
scala> genShape.sample res1: Option[Shape] = Some(Box(Blue,Box(Red,Circle(Green))))
As I've said, data generators are not exclusively related to properties; you can use the Gen module as an API for defining data generators for any setting really. Chapter 6 will provide reference information about most of the methods in Gen, and also show how to use the generator parameters, both when evaluating generators and when defining them.
Making explicit use of generators in properties
In most of the properties shown earlier, ScalaCheck has automatically picked suitable generator instances and used them behind the scenes when evaluating the properties. However, you can instruct ScalaCheck explicitly to use a certain generator in a property definition. You can use Prop.forAll with one extra parameter to inform ScalaCheck which generator to use:
import org.scalacheck.{Gen, Prop}
val evenInt = for { n <- Gen.choose(-1000, 1000) } yield 2*n
val propDivide = Prop.forAll(evenInt) { n: Int => val half = n/2 n == 2*half }
You can also specify several explicit generators for one property:
import org.scalacheck.Prop.forAll import org.scalacheck.Gen.{posNum, negNum}
val p = forAll(posNum[Int], negNum[Int]) { (n,m) => n*m <= 0 }
Another common usage of explicit generators is to nest forAll invocations, and let the inner one use an explicit generator that is defined in terms of the generated value in the outer forAll:
import org.scalacheck.Prop.forAll import org.scalacheck.Gen.choose
val propPrefix = forAll { s: String => forAll(choose(0, s.length)) { n => val prefix = s.substring(0, n) s.startsWith(s) } }
Instead of nesting forAll calls, we could have defined a custom generator in the following way:
import org.scalacheck.Arbitrary.arbitrary import org.scalacheck.Gen.choose
val genStringWithN = for { s <- arbitrary[String] n <- choose(0, s.length) } yield (s,n)
We can now specify the property with only one forAll call:
import org.scalacheck.Prop.forAll
val propPrefix = forAll(genStringWithN) { case (s,n) => val prefix = s.substring(0, n) s.startsWith(s) }
Notice that we have to use a case-expression since our property takes one tuple as its argument, not two separate arguments.
Whether you use nested forAll calls or custom generators is largely a matter of taste. If you have a lot of input arguments to your properties, putting them in a separate generator can make the property easier to read.
Adding implicit support for custom generators
I gave you a quick introduction to defining generators and then using them with the Prop.forAll method. However, you can also add implicit support for your own generators so you can write properties for your own classes in exactly the same way you would for the standard types, without explicitly specifying which generator to use in every property.
The key to this lies in Scala's built-in support for implicit methods and values. ScalaCheck can pick up default generators for any type if an implicit instance of the Arbitrary class for the given type exists. The Arbitrary class is simply a factory that provides a generator for a given type. In the example below, we first define a generator for a simple type and then make an implicit Arbitrary instance for it by using the Arbitrary module.
import org.scalacheck.Gen.{choose, oneOf}
case class Person ( firstName: String, lastName: String, age: Int ) { def isTeenager = age >= 13 && age <= 19 }
val genPerson = for { firstName <- oneOf("Alan", "Ada", "Alonzo") lastName <- oneOf("Lovelace", "Turing", "Church") age <- choose(1,100) } yield Person(firstName, lastName, age)
Given this Person generator, making an implicit Arbitrary[Person] instance is simple:
scala> import org.scalacheck.Arbitrary import org.scalacheck.Arbitrary
scala> implicit val arbPerson = Arbitrary(genPerson) arbPerson: org.scalacheck.Arbitrary[Person] = org.scalacheck.Arbitrary$$anon$1@1391f61c
As long as arbPerson is in scope, we can now write properties like this:
scala> import org.scalacheck.Prop.forAll import org.scalacheck.Prop.forAll
scala> val propPerson = forAll { p: Person => p.isTeenager == (p.age >= 13 && p.age <= 19) }
3.3 Test case simplification
As soon as ScalaCheck manages to falsify a property, it will try to simplify, or shrink, the arguments that made the property false. Then it will re-evaluate the property with the simplified arguments. If the property still fails, simplification will continue. In the end, the smallest possible test case that makes the property false will be presented along with the the original arguments that caused the initial failure. We can demonstrate this by defining a property that is intentionally wrong, to trigger the simplification mechanism in ScalaCheck:
import org.scalacheck.Prop.forAll import org.scalacheck.Gen.choose
val propNoPairs = forAll { xs: List[Byte] => forAll(choose(0, xs.length-2)) { i => xs(i) != xs(i+1) } }
The property states that there never exists a pair of equal items in a random list, which simply is false. Let's see what happens if we check the property:
scala> propNoPairs.check ! Falsified after 11 passed tests. > ARG_0: List("127", "127") > ARG_0_ORIGINAL: List("-104", "127", "127", "-1", "89") > ARG_1: 0
ScalaCheck correctly finds a test case (ARG_0_ORIGINAL) that makes the property false. Then this value is repeatedly simplified until ARG_0 remains, that still makes the property false.
ScalaCheck has the ability to simplify most data types for which it has implicit generators. There is no guarantee that the simplification will be perfect in all cases, but they are helpful in many situations. Where ScalaCheck has no built-in simplification support, you can add it yourself, just as you can add implicit generators for custom types. Therefore, you can give your own types and classes exactly the same level of support as the standard ones in ScalaCheck. In Chapter 6, you will be shown how to define such custom simplifiers for your own types.
3.4 Conclusion
This chapter has presented the fundamental parts of ScalaCheck, getting you ready to use it in your own projects. The next chapter will focus less on the technical details of ScalaCheck, and instead provide general techniques and ways to think when coming up with properties for your code. Later chapters will then revisit the topics of this chapter, digging deeper into the details of the API.