Data-Driven Tests

As stated earlier, these are tests that hook into data to throw them against a system in your codebase. Consider the object HorseOfCourse:

object HorseOfCourse{

  def isHorseOrMrEd: Animal => String = {
    case PaintHorse => "horse"
    case MrEd => "mr ed"
    case _ => "not a horse or mr ed"
  }

  def apply(x: Animal) = isHorseOrMrEd(x)

}

You can set up a few tests that ensure the proper test validation for horses and Mr. Ed:

val allHorses = Array(PaintHorse, PaintHorse, MrEd)

"A Horse" should "always be a horse" in {
  for(horse <- allHorses.filter(_ == PaintHorse)){
    horse should be (PaintHorse)
  }
}

it should "unless its the famous Mr. Ed" in {
  val mrEd = allHorses contains MrEd
  mrEd should be (true)
}

it should "understand the difference between horses" in {
  for(horse <- allHorses){
    val isItAHorse = HorseOfCourse(horse)
    horse match {
      case PaintHorse => isItAHorse should be("horse")
      case MrEd => isItAHorse should be("mr ed")
      case _ => isItAHorse should be("not a horse or mr ed")
    }
  }
} 

These sorts of tests can be bit cumbersome, since you build up more input from end users. However, that's part of the benefit, since you're using all that user input to then drive the tests in your suite. Along those same lines, property testing has huge value when creating data-driven tests in lieu of real data from logs or QA, since property tests can generate the inputs, leaving you only having to worry about the tests and underlying code (and possibly tweaking the generator). Let's take a quick look over property testing in ScalaTest. Given a simple class that generates the sum of a series:

import scala.language.postfixOps

object MathFun {
  def sumOfSeries(range: Int): Double = {
    1 to range map (x => 1.0 / (x * x)) sum
  }
}

Using this simple sum, you can then create a property test to determine if there are any holes in the logic.

import org.scalatest._
import org.scalatest.prop.GeneratorDrivenPropertyChecks

class MathFunSpec extends WordSpec with GeneratorDrivenPropertyChecks
  with Matchers {

  implicit override val generatorDrivenConfig = PropertyCheckConfig
   (minSuccessful = 3)

  "A series of numbers" should {
    forAll { (n: Int) =>
      whenever (n > 1) { MathFun.sumOfSeries(n) should be > 0.0 }
    }
  }
}

You'll note that the minSuccessful is set to a small value for a successful threshold, but this test should quickly fail. The sum of ranges has no protection for an extremely large number or a zero, neither of which we prepared for, which is the beauty of using property-based checking (this can also give your CPU a run for its money).

def sumOfSeries(range: Int): Double = {
  if(range <= 0) 0.0
  else if (range >= 32767) 0.0
  else {
    1 to range map (x => 1.0 / (x * x)) sum
  }
}

While these are not the best defaults, they will accommodate the edge cases that property checking has found.

Performance Testing

Usually, when it comes to micro benchmarking you shouldn't try to over optimize your code until the system/application has been completed. When it comes to all of the different libraries and approaches for getting something done in Scala, there should be no fundamental issue getting an idea of what the performance of each approach produces, even if those results vary to some degree based on hardware. To those ends, a micro benchmarking framework such as ScalaMeter (https://scalameter.github.io/) can be incredibly useful in finding inefficiencies in code, and help you start taking a granular approach to the logic you build in your systems.

To get started, add the following to your build.sbt:

resolvers += "Sonatype OSS Snapshots" at
  "https://oss.sonatype.org/content/repositories/snapshots"

libraryDependencies += "com.storm-enroute" %% "scalameter" % "0.8-SNAPSHOT"

testFrameworks += new TestFramework("org.scalameter.ScalaMeterFramework")

logBuffered := false

You can use the 0.7 ScalaMeter version if you prefer a stable version, but always check Maven, because these versions may have changed. You will also want to disable parallel execution in tests, which we usually disable in the SBT console before running benchmarking tests. You can also have the build.sbt file set to disable this by default using the following setting:

parallelExecution in Test := false

After that setup you can begin writing some inline benchmarking. A good feature that has been introduced in 0.7 is to wrap any function or method with the measure method and instantly get feedback on the performance of that code. This feature can be a great asset while writing code that can be executed. For a quick example, let's borrow for the ScalaMeter examples:

import org.scalatest.FunSuite
import org.scalameter._

class InlineBenchmarkTest extends FunSuite {
  test("Should correctly execute an inline benchmark") {
    val time = measure {
      for (i <- 0 until 100000) yield i
    }
    println(s"Total time: $time")
  }
}

Hopefully this gives you a small look into how to integrate ScalaMeter into your project, because that integration can give you some fantastic looks into bottlenecks and other performance-based issues early on in your coding.

Acceptance Testing

Acceptance testing is a methodology that also has concrete support in ScalaTest. Usually, the best way to think about acceptance testing is the same way you would think about black box testing, where you need to test an external API, a web page, or really anywhere you want to simulate actions in your system as if it were in production. Using a tool like Selenium is one way you can create a phantom user to interact with your application or external API—collect results and then provide feedback in terms of a real user running through the acceptance criteria of your application.

To start, you may want to take a look at the Selenium documentation from ScalaTest that is available (http://www.scalatest.org/user_guide/using_selenium). To get started using Selenium we'll need to modify the library dependencies in the build.sbt file.

libraryDependencies += "org.seleniumhq.selenium" % "selenium-java"
  % "2.35.0" % "test"

You can then reload SBT and compile it to fetch that dependency. Next, you can simulate an API to then test with Selenium. Let's use http4s with the blaze server to set up a quick API.

import org.http4s.HttpService
import org.http4s.dsl._
import org.http4s.server.blaze.BlazeBuilder

object Api extends App{
  val service = HttpService {
    case GET -> Root / "testRoute" =>
      Ok("This is a test route")
  }
    BlazeBuilder.bindHttp(8080)
    .mountService(service, "/")
    .run
    .awaitShutdown()
}

Go ahead and start that up in SBT and check it out in your browser by going to http://localhost:8080/testRoute, where you should see the phrase “This is a test route.” You can automate this test by creating the Selenium Test as follows:

import org.scalatest.{ShouldMatchers, FlatSpec}
import org.scalatest.selenium.HtmlUnit

class ApiSpec extends FlatSpec with ShouldMatchers with HtmlUnit {

  val host = "http://localhost:8080/"

  "This is a test route" should "should have the test content" in {
    go to (host + "testRoute")
    pageSource should be ("This is a test route")
  }
}

The DSL for Selenium makes it incredibly easy to read the logic within this test. Basically “go to” a URL, then check that the source is the proper phrase. While this is somewhat simplistic, you can do some fairly impressive testing. In the next example, you will do a search on Google for the rotten tomatoes score for Groundhog Day, use the cssSelector to find the rating field from the search results, and make sure it's about a 90% rating.

"This is a google search for Groundhog Day" should "should be greater than 90%"
  in {
  go to "http://www.google.com"
  click on "q"
  textField("q").value = "Groundhog Day Rotten Tomatoes"
  submit()
  eventually {
    val rawRating = find(cssSelector("div.f.slp")).get.underlying.getText
    val rating = rawRating.replaceAll("
","").replaceAll(" Rating:
       ","").replaceAll("%.*$","").toInt
    rating should be > 90
  }
}

You'll need to add the Eventually trait, since the search takes a moment for HtmlUnit to complete.

import org.scalatest.concurrent.Eventually

You should now have an idea of just how easy it is to get some feedback for the internal and external APIs within your system, and you can start using Selenium to make sure the results you see are within normal working parameters.

Mocks

Given how common they are in testing, let's examine stubs and mocks. Stubs are a simulation of an object's behavior, and mocks are an expectation of what a stub will produce. With ScalaTest and a mockito dependency, you can achieve simple testing of traits, classes, and objects by “training” those asserts to see what expectations you can get out of them.

In terms of how stub/mock testing works in Scala, let's use the mockito library and implement some basic tests for a bowling league service.

First, add the following library dependency to the build.sbt:

"org.mockito" % "mockito-all" % "1.10.19"

Then, let's add the trait for the bowling league service:

trait BowlingLeague {
  val leagueName: String
  val leagueRules: LeagueRules

  def addTeamToLeague(team: Team): Future[Boolean
  def removeTeamFromLeague(team: Team): Future[Boolean

  def teamStandings() : Future[Standings
}

You can start “mocking” out the adding and removing of leagues from the service:

class BowlingLeagueSpec
  extends FunSuite
    with ShouldMatchers
    with MockitoSugar
    with ScalaFutures {

  val players: Seq[Player] = Seq(
    Player("Charlie", "Brown", 100),
    Player("Linus", "Pelt", 105),
    Player("Lucy", "Pelt", 125),
    Player("Sally", "Brown", 85)
  )

  val testTeam = Team(
    "Pea Shooters",
    players
  )

  test("Adding a team") {
    val bowlingLeague = mock[BowlingLeague

    when(bowlingLeague.addTeamToLeague(testTeam)).thenReturn(Future(true))

    val addTeam = bowlingLeague.addTeamToLeague(testTeam)

    addTeam.futureValue should equal(true)
  }

  test("Removing a team") {
    val bowlingLeague = mock[BowlingLeague

    when(bowlingLeague.addTeamToLeague(testTeam)).thenReturn(Future(true))
    when(bowlingLeague.removeTeamFromLeague(testTeam)).thenReturn(Future(true))

    val remTeam = bowlingLeague.removeTeamFromLeague(testTeam)

    remTeam.futureValue should equal(true)
  }
}

By creating a mock of the bowling service, you can create an expectation of what adding and removing a team from league methods does. In the above code, you add an expectation that adding a team will return a future of true. This is a great way to skip having a hermetic server that is actually connecting to a database for a bowling league service, and just ensures that your interface is working properly.