Chapter 4. Testing Tools

No matter which language you are programming in, testing should be performed with great care, as it will not only document your code in a consistent way but will also be of great help for refactoring and maintenance activities, such as fixing bugs.

The Scala ecosystem largely follows Java trends towards testing at all levels, but with some differences. At many places, we will see that Scala is using DSLs (Domain Specific Languages), which makes the testing code very clear to read and understand. As a matter of fact, testing can be a good area to start with when introducing Scala, migrating progressively from an existing Java project.

In this chapter, we are going to cover some of the major testing tools and their usage through a number of code examples. We have already written a tiny JUnit-like test in Scala in Chapter 3, Understanding the Scala Ecosystem, so we will go from here and focus on BDD-style tests that belong to Behavior Driven Development (BDD). Agnostic to which technology stack is used, BDD has emerged in these past few years as a compliant choice for writing clear specifications in the Gherkin language (which is part of the cucumber framework and is explained at http://cukes.info/gherkin.html) on how code should behave. Already used in Java and many other languages, tests written in that style are often easier to understand and maintain as they are closer to plain English. They are one step closer to the true adoption of BDD that aims at making the business analysts write the test specifications in a structured way, which the programs can understand and implement. They often represent the sole documentation; it is therefore very important to keep them up to date and close to the domain.

Scala primarily offers two frameworks to write tests, ScalaTest (www.scalatest.org) and Specs2 (etorreborre.github.io/specs2/). As they are quite similar to each other, we are only going to cover ScalaTest, and interested readers can look through the Specs2 documentation to understand their differences. Moreover, we will take a look at automated property-based testing using the ScalaCheck framework (www.scalacheck.org).

Writing tests with ScalaTest

To be able to quickly start visualizing some of the tests that can be written with ScalaTest, we can take advantage of the test-patterns-scala template from the Typesafe Activator that we introduced in the previous chapter. It consists of a number of examples that essentially target the ScalaTest framework.

Setting up the test-patterns-scala activator project only requires you to go to the directory where you installed the Typesafe Activator, as we did earlier, and then, either start the GUI through the > activator ui command, or type > activator new to create a new project and select the appropriate template when prompted.

The template project already contains the sbteclipse plugin; therefore, you can generate eclipse-related files by simply entering from a command prompt in the root directory of the project, as follows:

> activator eclipse

Once the eclipse project is successfully created, you may import it into your IDE workspace by selecting File | Import... | General |Existing Projects. As a reminder from the previous chapter, you can also create project files for IntelliJ or other IDEs since the Typesafe Activator is just a customized version of SBT.

You can look into the various test cases in src/test/scala. As some of the tests use frameworks such as Akka, Spray, or Slick, which we haven't covered yet, we will skip these for now to concentrate on the most straightforward ones.

In its simplest form, a ScalaTest class (which, by the way, might also test Java code and not just Scala code) can be declared by extending org.scalatest.FunSuite. Each test is represented as a function value, and this is implemented in the Test01.scala class, as shown in the following code:

package scalatest
import org.scalatest.FunSuite

class Test01 extends FunSuite {
  test("Very Basic") {
    assert(1 == 1)
  }
  test("Another Very Basic") {
    assert("Hello World" == "Hello World")
  }
}

To execute only this single test class, you should enter the following command in the command prompt:

> activator
> test-only <full name of the class to execute>

In our case, this command will be as follows:

> test-only scalatest.Test01   (or scalatest.Test01.scala)
[info] Test01:
[info] - Very Basic (38 milliseconds)
[info] - Another Very Basic (0 milliseconds)
[info] ScalaTest
[info] Run completed in 912 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 9 s, completed Nov 11, 2013 6:12:14 PM

The example given under src/test/scala/scalatest/Test02.scala within the test-patterns-scala project is very similar, but the extra === instead of == will give you additional info when the test fails. This is shown as follows:

class Test02 extends FunSuite {
  test("pass") {
    assert("abc" === "abc")
  }
  test("fail and show diff") {
    assert("abc" === "abcd") // provide reporting info
  }
}

Once again running the test can be done by entering the following command:

> test-only scalatest.Test02
[info] Test02:
[info] - pass (15 milliseconds)
[info] - fail and show diff *** FAILED *** (6 milliseconds)
[info]   "abc[]" did not equal "abc[d]" (Test02.scala:10)
[info] …
[info] *** 1 TEST FAILED ***
[error] Failed: Total 2, Failed 1, Errors 0, Passed 1

Before fixing the failing test, this time, we can execute the test in the continuous mode, using the ~ character in front of test-only (from the activator prompt), as follows:

>~test-only scalatest.Test02

The continuous mode will make SBT rerun the test-only command each time the Test02 class is edited and saved. This feature of SBT can make you save a significant amount of time by running in the background tests or just programs without having to explicitly write the command. On the first execution of Test02, you can see some red text indicating "abc[]" did not equal "abc[d]" (Test02.scala:10).

As soon as you correct the abdc string and save the file, SBT will automatically re-execute the test in the background, and you can see the text turning green.

The continuous mode works for the other SBT commands as well, such as ~run or ~test.

Test03 shows you how to expect or catch exceptions:

class Test03 extends FunSuite {
  test("Exception expected, does not fire, FAIL") {
    val msg = "hello"
    intercept[IndexOutOfBoundsException] {
      msg.charAt(0)
    }
  }
  test("Exception expected, fires, PASS") {
    val msg = "hello"
    intercept[IndexOutOfBoundsException] {
      msg.charAt(-1)
    }
  }
}

The first scenario fails as it was expecting an IndexOutOfBoundsException, but the code is indeed returning a valid h, the character at index 0 of the hello string.

To be able to run ScalaTest test suites as JUnit test suites (for example, to run them within the IDE or when extending an existing JUnit-based project that is already built in Maven, or when reporting to a build server), we can use the available JUnitRunner class along with the @RunWith annotation, as shown in the following sample:

import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner
import org.scalatest.FunSuite
@RunWith(classOf[JUnitRunner])
class MyTestSuite extends FunSuite {
  // ...
}

BDD-style testing

Test06 is an example of a test written in a different style, namely BDD. In short, you specify some kind of a user story in almost plain English that describes the behavior of the scenario you want to test. This can be seen in the following code:

class Test06 extends FeatureSpec with GivenWhenThen {

  feature("The user can pop an element off the top of the stack") 
  {
info("As a programmer")
  info("I want to be able to pop items off the stack")
  info("So that I can get them in last-in-first-out order")

  scenario("pop is invoked on a non-empty stack") {

    given("a non-empty stack")
    val stack = new Stack[Int]
    stack.push(1)
    stack.push(2)
    val oldSize = stack.size

  when("when pop is invoked on the stack")
  val result = stack.pop()

  then("the most recently pushed element should be returned")
  assert(result === 2)

  and("the stack should have one less item than before")
  assert(stack.size === oldSize - 1)
  }

  scenario("pop is invoked on an empty stack") {

    given("an empty stack")
    val emptyStack = new Stack[Int]

    when("when pop is invoked on the stack")
    then("NoSuchElementException should be thrown")
    intercept[NoSuchElementException] {
    emptyStack.pop()
    }

  and("the stack should still be empty")
  assert(emptyStack.isEmpty)
  }
}
}

BDD-style tests represent a higher level of abstraction than JUnit tests, and are more suitable for integration and acceptance testing as well as documentation, for people knowledgeable about the domain. You just need to extend the FeatureSpec class, optionally with a GivenWhenThen trait, to describe acceptance requirements. More details about BDD-style tests can be found at http://en.wikipedia.org/wiki/Behavior-driven_development. We just want to illustrate here that it is possible to write the BDD-style tests in Scala, but we won't go further into their details as they are already largely documented for Java and other programming languages.

ScalaTest provides a convenient DSL to write assertions in a way close to plain English. The org.scalatest.matchers.Matchers trait contains many possible assertions and you should look at its ScalaDoc documentation to see many usage examples. Test07.scala expresses a very simple matcher, as shown in the following code:

package scalatest

import org.scalatest._
import org.scalatest.Matchers

class Test07 extends FlatSpec with Matchers {
"This test" should "pass" in {
    true should be === true
  }
}

Note

Although built with Version 2.0 of ScalaTest, the original sample given in the activator project uses the now deprecated org.scalatest.matchers.ShouldMatchers trait; the preceding code sample achieves the same behavior but is more up to date.

Let's write a few more assertions using a Scala Worksheet. Right-click on the scalatest package that contains all the test files that were previously reviewed and then select new | Scala Worksheet. We will name this worksheet as ShouldWork. We can then write and evaluate matchers by extending a FlatSpec specification with the Matchers trait, as shown in the following code:

package scalatest
import org.scalatest._
object ShouldWork extends FlatSpec with Matchers {

  true should be === true

}

Saving this worksheet will not produce any output as the matcher passes the test. However, try to make it fail by changing one true to false. This is shown in the following code:

package scalatest
import org.scalatest._

object ShouldWork extends FlatSpec with Matchers {

  true should be === false

}

This time, we get a full stack trace as part of the evaluation, as shown in the following screenshot:

BDD-style testing

We can start evaluating many more should matchers, as shown in the following code:

package scalatest
import org.scalatest._

object ShouldMatchers extends FlatSpec with Matchers {

  true should be === true

  List(1,2,3,4) should have length(4)

  List.empty should be (Nil)

  Map(1->"Value 1", 2->"Value 2") should contain key (2)
  Map(1->"Java", 2->"Scala") should contain value ("Scala")

  Map(1->"Java", 2->"Scala") get 1 should be (Some("Java"))

  Map(1->"Java", 2->"Scala") should (contain key (2) and not contain value ("Clojure"))

  3 should (be > (0) and be <= (5))

  new java.io.File(".") should (exist)
}

The evaluation of the worksheet stops whenever we encounter a test failure. Therefore, we have to fix it in order to be able to progress in the test. This is identical to running the whole testsuite with the SBT test command, as we did previously, and as shown in the following code:

object ShouldMatchers extends FlatSpec with Matchers {

"Hello" should be ("Hello")

"Hello" should (equal ("Hej")
               or equal ("Hell")) //> org.scalatest.exceptions.TestFailedException:

"Hello" should not be ("Hello")
}

In the previous example, the last statement (which is the opposite of the first one) should fail; instead, it is not evaluated.

Functional testing

ScalaTest is well integrated with Selenium (it is a tool for automating testing in browsers and is available at www.seleniumhq.org) by providing a complete DSL, making it straightforward to write functional tests. Test08 is a clear example of this integration:

class Test08 extends FlatSpec with Matchers with WebBrowser {

  implicit val webDriver: WebDriver = new HtmlUnitDriver
go to "http://www.amazon.com"
click on "twotabsearchtextbox"
textField("twotabsearchtextbox").value = "Scala"
submit()
pageTitle should be ("Amazon.com: Scala")
pageSource should include("Scala Cookbook: Recipes")
}

Let's try to run a similar invocation directly into a worksheet. As worksheets give feedback on every statement evaluation, they are very convenient to directly identify what the problem is, for instance, if a link, a button, or content is not found as expected.

Just create another worksheet called Functional next to the ShouldWork worksheet that is already present. Right-click on the scalatest package and select New | Scala Worksheet.

The worksheet can be filled as follows:

package scalatest
import org.scalatest._
import org.scalatest.selenium.WebBrowser
import org.openqa.selenium.htmlunit.HtmlUnitDriver
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.WebDriver
object Functional extends FlatSpec with Matchers with WebBrowser {
implicit val webDriver: WebDriver = new HtmlUnitDriver
  go to "http://www.packtpub.com/"
  textField("keys").value = "Scala"
  submit()
  pageTitle should be ("Search | Packt Publishing")
  pageSource should include("Akka")
}

Upon the save operation (Ctrl + S), the worksheet will be evaluated and should probably display some output information for every statement, except for the last two lines with should matchers, as they should evaluate to true.

Try to change ("Search | Packt Publishing") to a different value, such as Results or just Packt Publishing, and notice how the console output provides handy information on what does not match. This is shown in the following screenshot:

Functional testing

This functional test just scratches the surface of what's possible. As we are using the Java Selenium library, in Scala, you can inherit the power of the Selenium framework that is available in Java.

Mocking with ScalaMock

Mocking is a technique by which you can test code without requiring all of its dependencies in place. Java offers several frameworks for mocking objects when writing tests. The most well known are JMock, EasyMock, and Mockito. As the Scala language introduces new elements such as traits and functions, the Java-based mocking frameworks are not enough, and this is where ScalaMock (www.scalamock.org) comes into play.

ScalaMock is a native Scala-mocking framework that is typically used within ScalaTest (or Specs2), by importing the following dependencies into the SBT (build.sbt) file:

libraryDependencies +="org.scalamock" %% "scalamock-scalatest-support" % "3.0.1" % "test"

Within Specs2, the following dependencies need to be imported:

libraryDependencies +=
"org.scalamock" %% "scalamock-specs2-support" % "3.0.1" % "test"

Since the release of the Scala Version 2.10, ScalaMock has been rewritten, and the ScalaMock Version 3.x is the version that we are going to cover briefly by going through an example of mocking a trait.

Let's first define the code that we are going to test. It consists of a tiny currency converter (available at http://www.luketebbs.com/?p=58) that fetches currency rates from the European Central Bank. Retrieving and parsing the XML file of currency rates is only a matter of a few lines of code, as follows:

trait Currency {
  lazy val rates : Map[String,BigDecimal] = {
  val exchangeRates =
    "http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml"
  for (
    elem <- xml.XML.load(exchangeRates)"Cube""Cube""Cube")
  yield
    (elem"@currency").text -> BigDecimal((elem"@rate").text)
  }.toMap ++ Map[String,BigDecimal]("EUR" -> 1)

  def convert(amount:BigDecimal,from:String,to:String) =
    amount / rates(from) * rates(to)
}

In this example, the currency rates are fetched from a URL using the xml.XML.load method. As XML is part of the Scala standard library, there is no need for imports here. The load method parses and returns the XML rates as an immutable structure of type Elem, which is a case class that represents XML elements. This is shown in the following code:

<gesmes:Envelope xmlns:gesmes="http://www.gesmes.org/xml/2002-08-01"
xmlns="http://www.ecb.int/vocabulary/2002-08-01/eurofxref">
  <gesmes:subject>Reference rates</gesmes:subject>
    <gesmes:Sender>
      <gesmes:name>European Central Bank</gesmes:name>
    </gesmes:Sender>
    <Cube>
      <Cube time="2013-11-15">
      <Cube currency="USD" rate="1.3460"/>
      <Cube currency="JPY" rate="134.99"/>
      <Cube currency="BGN" rate="1.9558"/>
      <Cube currency="CZK" rate="27.155"/>
      <Cube currency="DKK" rate="7.4588"/>
      <Cube currency="GBP" rate="0.83770"/>
           ...
         ...
    </Cube>
  </Cube>
</gesmes:Envelope>

Accessing the list of currency rates from this XML document is done through an XPath expression by navigating inside the Cube nodes, hence the xml.XML.load(exchangeRates) "Cube" "Cube" "Cube" expression. A single for comprehension (the for (…) yield (…) construct that we introduced in the previous chapter) is required to loop over the currency rates and return a collection of key -> value pairs where, in our case, a key will be a string that represents the currency name, and value will be a BigDecimal value that represents the rate. Notice how the information is extracted from <Cube currency="USD" rate="1.3460"/> by writing (elem "@currency").text to capture the currency attribute and (elem "@rate").text to capture the rate respectively. The latter will be further processed by creating a new BigDecimal value from the given string.

In the end, we get a Map[String, BigDecimal] that contains all our currencies with their rates. To this value, we add the mapping for the currency EUR (Euros) that will represent the reference rate one; this is why we use the ++ operator to merge two maps, that is, the one we just created together with a new map containing only one key -> value element, Map[String,BigDecimal]("EUR"-> 1).

Before mocking, let's write a regular test using ScalaTest with FlatSpec and Matchers. We will make use of our Converter trait, by integrating it into the following MoneyService class:

package se.chap4

class MoneyService(converter:Converter ) {

  def sendMoneyToSweden(amount:BigDecimal,from:String): BigDecimal = {
    val convertedAmount = converter.convert(amount,from,"SEK")
    println(s" $convertedAmount SEK are on their way...")
    convertedAmount
  }

  def sendMoneyToSwedenViaEngland(amount:BigDecimal,from:String): BigDecimal = {
    val englishAmount = converter.convert(amount,from,"GBP")
    println(s" $englishAmount GBP are on their way...")
    val swedishAmount = converter.convert(englishAmount,"GBP","SEK")
    println(s" $swedishAmount SEK are on their way...")
    swedishAmount
  }
}

A possible test specification derived from the MoneyService class is as follows:

package se.chap4

import org.scalatest._
import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner

@RunWith(classOf[JUnitRunner])
class MoneyServiceTest extends FlatSpec with Matchers {

"Sending money to Sweden" should "convert into SEK" in {
    val moneyService = 
      new MoneyService(new ECBConverter)
    val amount = 200
    val from = "EUR"
    val result = moneyService.sendMoneyToSweden(amount, from)
    result.toInt should (be > (1700) and be <= (1800))
  }

"Sending money to Sweden via England" should "convert into GBP then SEK" in {
    val moneyService = 
      new MoneyService(new ECBConverter)
    val amount = 200
    val from = "EUR"
    val result = moneyService.sendMoneyToSwedenViaEngland(amount, from)
    result.toInt should (be > (1700) and be <= (1800))
  }
}

To be able to instantiate the Converter trait, we use an ECBConverter class defined in the Converter.scala file as follows:

class ECBConverter extends Converter

If we execute the test from the SBT command prompt or directly within Eclipse (as a JUnit), we get the following output:

> test
[info] Compiling 1 Scala source to /Users/thomas/projects/internal/HttpSamples/target/scala-2.10/test-classes...
 1792.2600 SEK are on their way...
 167.70000 GBP are on their way...
 1792.2600 SEK are on their way...
[info] MoneyServiceTest:
[info] Sending money to Sweden
[info] - should convert into SEK
[info] Sending money to Sweden via England
[info] - should convert into GBP then SEK
[info] Passed: : Total 2, Failed 0, Errors 0, Passed 2, Skipped 0
[success] Total time: 1 s, completed

If the URL from which we are retrieving the currency rates is not always available, or if the currency rates have changed a lot on one particular day and the resulting amount of the conversion is not in the given interval of the assertion should (be > (1700) and be <= (1800)), then our test might fail. In that case, mocking the converter in our test seems appropriate, and can be done as follows:

package se.chap4

import org.scalatest._
import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner
import org.scalamock.scalatest.MockFactory

@RunWith(classOf[JUnitRunner])
class MockMoneyServiceTest extends FlatSpec with MockFactory with Matchers {

"Sending money to Sweden" should "convert into SEK" in {

    val converter = mock[Converter]
    val moneyService = new MoneyService(converter)

    (converter.convert _).expects(BigDecimal("200"),"EUR","SEK").returning(BigDecimal(1750))

    val amount = 200
    val from = "EUR"
    val result = moneyService.sendMoneyToSweden(amount, from)
    result.toInt should be (1750)
  }
}

The expects method contains the arguments that we expect when our code should invoke the convert method, and the returning method contains our expected output in place of the real return result.

ScalaMock has many variations on how to apply the mocking code, and is planning to enhance the mocking syntax using the Macros in future releases. In short, Macros are functions that are called by the compiler during compilation. It is an experimental feature added in Scala from Version 2.10 that makes it possible for the developer to access the compiler APIs and apply transformations to the AST (Abstract Syntax Tree), that is, the tree representation of a program. Macros are out of the scope of this book, but among other things, they are useful for the Code Generation and DSLs. Their usage will improve the ScalaMock syntax; for instance, you can apply your mock expectations within inSequence {… } or the inAnyOrder {… } blocks of code or in nested combinations of these blocks, as illustrated in their documentation, which is available at scalamock.org. ScalaMock also supports a more Mockito-like style with a Record-then-Verify cycle rather than the Expectations-First style, which we have been using.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.129.24.180