IMMUTABILITY

Programming languages have a different approach to immutability: some of them embrace it and don't allow any mutable state, and others allow it only through constants that, arguably, are semantically different from a simple immutable value. Scala allows both approaches through val and var keywords. Let's look at an abstract example of swapping values in variables that makes mutable state more obvious:

var a = 1
var b = 2
// … other code goes here, but something happens, we need to swap
var c = a // a = 1 b = 2
a = b // a = 2 b = 2
b = c // a = 2 b = 1

If you have not seen this pattern before, this code may seem cryptic to you. Also, there is a state of uncertainty at the line with a = b, where a thread accessing the values of a and b will see a = 2 and b = 2. This is something you wouldn't expect to see (it should be either a = 1 b = 2 at the ­beginning or a = 2 b = 1 after the swapping).

So, how can you better handle this situation? It may seem obvious, but wouldn't it be preferable to use additional values:

// todo: find real world example with meaningful names
val a = 1
val b = 2
// something happens, we need to swap
val c = a
val d = b

Instead of reusing old variables, you can create new values with a new semantic meaning (from new value's name) and immutability. As these are now constant values, you will remain assured that all the way through the method, the values of a and b will stay the same, no matter how you use them. But what about a more real life example:

var users = dao.findAll()
// … here goes some code that uses users variable
// so that the next line cannot be directly chained …
users = users.filter(_.age >= 18)
// … reusing the same users variable through the rest of the method … 

In this code adult users are selected. A small schema of what this operation looks like is shown in Figure 2.1.

The darker part of the rectangle is the part that was filtered out of the users variable. As you can see (see Figure 2.2), the developer didn't take time to create another variable with a more appropriate name, because it wasn't needed, but if the immutable values were used, it would have been a requirement.

val users = dao.findAll()
// … here goes some code that uses users variable
// so that the next line cannot be directly chained …
val adults = users.filter(_.age >= 18)
// reusing adults that stays the same during the rest of the method

Here you may see that even after the transformation the user variable has the same value and meaning as before, and the result is stored in a new variable, with a name that is semantically correct.

As a rule of thumb, whenever you have a computation that may be given a name, assign it to a new value with that name. It's far better than a commentary, because it makes your code more readable. This is especially useful when breaking chained methods into meaningful pieces.

But val alone does not guarantee immutability. In the following code only val is used, but the value is not the same during the execution:

val string = new StringBuilder()
string.append("ab")
println(string) // "ab"
string.append("cd")
println(string) // "abcd"

When you are using mutable structures, you lose all of the benefits of val for some small improvements in performance that will be negated by the time spent debugging your code. You can read the “mutable vs immutable collections” explanation in Chapter 1.

When everything is stateless (no variables or mutable structures) there is also another benefit: your code becomes thread-safe, and it may be called from different threads without any synchronization needed.

Although avoiding var at all cost may be great (you can use tools like http://www.scalastyle.org/ to enforce this rule), there may be some cases where var may seem like a necessity. But before going with var, look to see if there is no alternative implementation in the part of code you are working on: the forced use of a var (outside of Scala library internals) is usually a sign of a “code smell.”

PURE FUNCTIONS

Pure functions, also known as “side-effect free,” are functions that take some value as an input and return a value without using any outside scope for the computations. Such functions are also called “referentially transparent.” That means a call to a side-effect free function may be directly replaced with its returned value. How can you find out if a function is not pure? If it accesses an outside scope or database, mutates input values, prints anything, or returns Unit, it has side effects. As an example of such functions, consider this:

def helloWorld(): Int = {
    println("Hello")
    println("World")
    42
}

The function is not side-effect free, because it can't be replaced with its return value, a number 42, as there are still calls to a println() that do something, such as showing a string in the terminal. This is why it is not referentially transparent. This function, however, is pure:

def add(a: Int, b: Int): Int = a + b

It doesn't use any outside scope and it can be directly replaced with its return value, add(1, 2), which can be replaced with 3 without any consequences. Otherwise, if the add() function stores something in the database, it can't be possible to replace the call with the results so easily.

The benefits of pure functions are:

• Improved testability: Your code doesn't need any mocks or dependency injections. A simple call to the function with predefined input values will always return the same result no matter what happens outside of it.
• Improved readability: Methods are loosely coupled because there is no shared scope between them. You will not break the functionality by creating another pure function. Also method signatures are more explicit in side-effect free functions.
• Improved performance: If your pure function is computation heavy, you may add caching to prevent it from recomputing the same value twice.
• Improved concurrency: Side-effect free functions are thread safe, because they don't use any shared scope. This is especially important in concurrent code, as you will see in Chapter 11. But for instance, imagine calling the helloWorld() function above by many threads at the same time. It would be impossible to predict in what order the words “Hello” and “World” would be printed!

Though pure functions are great, it's often impossible to create an application without using functions with side effects. For instance, every access to a database is a side effect. The solution to this problem is to extract side-effect free code into separate functions with meaningful names and keep them as small as possible. For example, instead of:

def extractAdultUserNames(): List[String] = {
       val users = dao.findAll()
       users.filter(_.age >= 18).map(_.name)
}

You can do:

def extractAdultUserNames(users: List[User]): List[String] =
users.filter(_.age >= 18).map(_.name)

val users = dao.findAll()
extractAdultUserNames(users)

As you may see, in this oversimplified example, we refactored the extractAdultUserNames method so that it is now side-effect free. As a benefit, this method is now far less painful to test (no need to mock dao dependency injection); it may be used in concurrent structures (in a map() method of a parallel collection for example). It also has a more accurate and meaningful signature (just judging by the input type and the output type we can guess how the method works).

Because of the abundance of the mutable state in imperative languages, very few talk about side-effect free methods outside of functional programming. Even if the concept of pure function is hard to gasp at first, it may greatly improve your code once applied.

RECURSION

Recursion is rarely used in imperative languages, except for some canonical cases like tree traversal or for writing a Fibonacci function during your job interview. It seems that such functions are often feared because of the complexity they bring, and the potential problems with stack overflow. Surprisingly enough in functional languages, recursion is one of the main ways of doing unbounded loops, because these languages do not have a while construct. Scala doesn't remove while from your toolbox; instead it gives you tools to handle recursion and use it to your own advantage.

Let's see why while is considered a bad practice, with this minimalistic example:

var flag = true

while (flag){
       println(flag)
       flag = false
}

Easy, right? Also very familiar. So, what's wrong with this code? First of all, it's mutable: there is a var that cannot be directly replaced with a val. Secondly, the block inside the while is not side-effect free, it should somehow modify the scope outside of it; otherwise the condition will always be true and the while loop will never end. These two issues were discussed previously in this chapter. Lastly, it is difficult to extract the block inside while into a separate method due to the closure (the link to the external flag variable). Otherwise, you need to mutate the variable that is passed in parameters, which is generally considered a poor practice.

So, doing while is bad, but is recursion better? Let's take for a fact that any while can be written in a recursive way, so you can prove it by creating whileFunc(), which does exactly the same thing as while, but recursively:

def whileFunc[T](block: => T, condition: => Boolean): Unit = {
    if (condition) {
        block
        whileFunc(block, condition)
    }
}

// and an example of usage:
var i = 0

whileFunc({
        println(i)
        i = i + 1
}, i < 10) // the boolean statement will be re-evaluated as it is a call-by-name

This code will print numbers from 0 to 9. It is a challenge to convince anyone that this code is better than the 4 lines in the while loop. Not only this, but the code will still get stack overflow if you increase the condition number. So, how can you deal with this problem? You can use a tail recursion: a recursive function that has a call to itself as the last call. Here is a classical example of a recursive function representing a factorial:

def fact(n: Int): Int = {
    if (n < 1) 1
    else n * fact(n - 1)
}

But this is not tail recursive: as you can see, the last call to that function isn't a call to fact(); instead it's a call to *. To make it a tail recursive you need to modify its signature:

@tailrec
def fact(n: Int, acc: Int = 1): Int = {
       if (n < 1) acc
       else fact(n - 1, acc * n)
}
fact(6) // returns 720

This last call is to the fact() function, and the @tailrec annotation verifies that the function is tail-recursive during compilation. If it is, the compiler can transform it into its while version, so that the “stack overflow” exception is no longer an issue. To better see the benefits of a recursive function, consider this code:

val ids = List(0, 3, 4, 7, 9) // generally those Ids are stored in the database
var generatedId = 0 // or null, doesn't matter

do {
  generatedId = scala.util.Random.nextInt(10)
} while (ids.contains(generatedId))

println(generatedId)

The code is not hard, but everything is here: mutable state, side effects, and a variable declaration outside of the while block. Let's refactor it in a functional way:

val ids = List(0, 3, 4, 7, 9) // generally those Ids are stored in the database

@tailrec
def generateId(currentIds: List[Int]): Int = {
  val id = scala.util.Random.nextInt(10)

  if (currentIds.contains(id)) generateId(currentIds)
  else id
}

println(generateId(ids))

Even if you only use recursion once in a blue moon, this code shouldn't be a problem. There is no modification of outside scope, which is an elegant execution. Every time you enter the body of generateId() function, you have a clean state without needing to look outside of the scope.

But sadly, not all recursive functions are easily transformed into tail recursive ones. This is a Fibonacci function:

def fibo( n : Int): Int = {
   if (n < 2) n
   else fibo(n-1) + fibo(n-2)
}

Its definition is a sum of the previous two values in the Fibonacci sequence. Here is its tail-recursive version:

@tailrec
def fibo(n: Int, a: Int = 0, b: Int = 1): Int = {
  if (n < 1) a
  else fibo(n-1, b, a+b)
}

It takes some time to understand how it works. But even if it still compiles to a while loop internally, it is still better than the actual Fibonacci's while version:

def fiboWhile( n : Int ) : Int = {
  var first = 0
  var second = 1
  var i = 0

  while( i < n ) {
    val result = first + second
    first = second
    second = result
    i = i + 1
  }
  first
}

So, as a rule of thumb, when dealing with while in your code, try to see if there is an alternative using Collection API, which is discussed later in this chapter. Otherwise, find a tail-recursive function that will do the job, and don't forget about the @tailrec annotation, because it will do all of the complicated checking for you. It will be hard at first, but soon recursion won't be a mystery anymore.

HIGHER-ORDER FUNCTIONS

Higher-order functions accept another function as a parameter and/or return a new one. Purely functional programming languages don't have any objects; instead they have functions as first-class citizens. This means that functions are not different from a normal value like a String or an Int. And applications are made by combining them into one top-level function. As for a hybrid language like Scala, higher order functions are most useful for removing code repetitions. Consider this simplified example of a stateful backend:

val authenticatedUsers = List("Alex", "Sam")

// example url /hello/{userName}
def hello(userName: String): String = {
  if (authenticatedUsers.contains(userName)) s"world $userName"
  else "Unauthorized access"
}

// example url /foo/{userName}
def foo(userName: String): String = {
  if (authenticatedUsers.contains(userName)) s"bar $userName"
  else "Unauthorized access"
}

println(s"request to /hello/Alex: ${hello("Alex")}")
println(s"request to /hello/David: ${hello("David")}")
println(s"request to /foo/Alex: ${foo("Alex")}")

The server-side has a list of authenticated users, so when the client-side requests hello() action, the server checks first if the user is authenticated and then returns the string generated by the business logic. So, hello() and foo() methods are nearly identical, it's just the domain logic that is a bit different. Let's extract a higher order function that will handle user authorization in one place:

def userAwareAction(userName: String,
                    authUsers: List[String],
                    f: String => String): String = {
  if (authUsers.contains(userName))  f(userName)
  else "Unauthorized access"
}

and the refactored methods hello() and foo():

val authenticatedUsers = List("Alex", "Sam")

def hello(userName: String): String = userAwareAction(userName, authenticatedUsers,
  userName => s"world $userName")

def foo(userName: String): String = userAwareAction(userName, authenticatedUsers,
  userName => s"bar $userName")

In addition to the removed repetitions, you can create the method userAwareAction()without side-effects, so it has all of the advantages we've discussed, such as the testability without needing to mock anything. Aside from that, this method may also be moved to the parent controller so that any child controller can benefit from it. In this case, if you need to change the way a user is authenticated, such as fetch this information from a database instead of a list, or modify the reply in case authentication failed with a “403” page instead of just a String, you can do it in one place from now on.

To summarize, if you are dealing with an application that has a lot of code repetitions, look for a higher order function that will contain the common code. The next part of the chapter covers higher order functions that help you with writing more meaningful and declarative code applied to collections.

As for the part where a method returns a function, that is covered in the section about currying.

CORE COLLECTION METHODS

Collections, including sequences, sets, and maps, may be considered one of the most used data structure in programming languages. In fact, some languages are entirely constructed from lists and primitives; just look at Lisp! Functional programming languages contain quite a number of ­collection methods, so it is important to know them. In imperative languages you are used to ­dealing with collections using loops. For example, to transform a collection of User objects into a collection of users' names, you can do the following:

case class User(name: String, age: Int)

val users = List(User("Alex", 26), User("Sam", 24))

var names = List[String]()

for (user <- users){
  names = names :+ user.name
}

println(users)

A common pattern is to take an element from one array, transform it, and insert it into another. Let's put aside the version with while, because we already know why it is considered bad. But look at the previous code: doesn't it have the same problems? It has a mutable state coupled with side effects, and poor method extraction. The fact that this is called micro management is another reason and looks like what is shown in Figure 2.3.

You need to transform the pile in Figure 2.3 on the left into the pile on the right. When doing for loops, you are telling each one of the workers to go to the first pile, take the box, change its color, go to the second pile, and put it there. This is for every loop and iteration. This is quite tiresome for real life management. Why not just tell them to transform this pile of white boxes into the dark ones and leave them to do their job? Luckily, in functional programming there is a function that does such a thing. It is called map(), and here is how you can refactor the previous iterative code:

val users = List(User("Alex", 26), User("Sam", 24))

var names = users.map(user => user.name)

Or its shortened and, arguably, more concise version:

var names = users.map(_.name)

The function that is passed to the method is called lambda expression. If at first the map() doesn't sound like a “transform” to you, don't worry, because after time in functional programing, you will find it normal because the map() method is everywhere. But there is more on this in Chapter 10 on monaïdic types.

The second most common operation on a collection is ridding it of unwanted elements:

val users = List(User("Alex", 12), User("Sam", 22))

var adults = List[User]()

for (user <- users){
  if (user.age >= 18) {
    adults = adults :+ user
  }
}

println(users)

Here you are like club security checking everyone personally, instead of just filtering the flow:

val adults = users.filter(_.age >= 18)

By now the benefit of declarative programming should be obvious, and the iterative version of the method is no longer needed to prove it. You may say that this is less debuggable, but in a modern IDE, you already have a possibility to directly debug lambda expressions.

The Collection API is divided into two categories: methods that return other collections (including maps) and methods that return values. When doing an operation on an iterable that sounds like it may be already implemented in the core library, you should have a reflex of going to the corresponding API section (like this one: http://www.scala-lang.org/api/current/index.html#scala.collection.Seq for Seq) and to look up if there is already an implementation.

The following is a list of selected methods returning a collection. They are not sorted in any particular order, but it is useful to know that they exist as they are implemented in many functional programming languages. More often than not, the purpose of the method may be guessed from its name, but if it's not obvious, the description and the example will better describe the function's goal.

val shakespeare = List(
  "Words are easy",
  "like the wind",
  "faithful friends",
  "are hard to find"
)

val words: List[String] = shakespeare.flatMap(line => line.split(" "))

val wordsNotFlattened: List[Array[String]] =  shakespeare.map(_.split(" "))
val words: List[String] = wordsNotFlattened.flatten

val listWithRepetitions = List("Alex", "Alex", "Sam")

println(listWithRepetitions.distinct) // prints List(Alex, Sam)
println(listWithRepetitions.toSet) // prints Set(Alex, Sam)

case class User(identifier: String, likes: String)

val users = List(User("Alex", "Kiwi"),
                 User("Sam", "Banana"),
                 User("Alex", "Apple"))
val likesByUser = users.groupBy(_.identifier)
/* likesByUser contains:
Map(
  Alex -> List(User2(Alex,Kiwi), User2(Alex,Apple)),
  Sam -> List(User2(Sam,Banana))
)*/

val (moreThanTwo, lessOrEqualThanTwo) = List(1, 2, 3, 4).partition(_ > 2)

List(3, 4, 1, 2).sorted

case class User(name: String, age: Int)

val users = List(User("Alex", 26), User("Sam", 24), User("David", 25))

users.sortBy(_.age)
users.sortWith((a, b) => a.age < b.age)  // same as sortBy example,
  but more flexible

List(3, 4, 1, 2).sorted.reverse // returns List(4, 3, 2, 1)

List(1, 2, 3).foreach(println)

List(1, 2, 3).contains(1) // returns true
List(1, 3, 5).exists(_ % 2 == 0) // returns false
List(2, 4, 6).forall(_ % 2 == 0) // returns true

List(1, 2, 4, 6).find(_ % 2 == 0) // returns Some(2) 

List(1, 2, 4, 6).count(_ % 2 == 0) // returns 3
List(1, 2, 4, 6).mkString("|") // returns "1|2|4|6"

fold(0)((acc, value) => acc + value)

reduce((acc, value) => acc + value)

List(1, 2, 3).reduce(_ / _)
List(1, 2, 3).fold(1)(_ / _)

val listX = List(1, 2, 3)
val listY = List(1, 2, 3)

for {
  x <- listX
  y <- listY
} yield (x, y)

listX.flatMap(x => listY.map(y => (x, y)))

CURRYING AND PARTIALLY APPLIED FUNCTIONS

Partially applied functions or currying (also known as Schönfinkelization, thanks to the work of mathematician Moses Schönfinkel) are, one of the most difficult concepts to put into practice among those mentioned in this chapter. It is often used for code factorization, local dependency injection, or simply to improve code readability.

As an example, let's take this add method:

def add(a: Int, b: Int) = a + b

It may seem to be quite useless, but if you have add5 and add7 in your code, you can simplify their definition with a more generalized one using partially applied functions:

def add5 = add(5, _: Int)
def add7 = add(7, _: Int)

add5(3) // returns 8
add7(2) // returns 9

Here you have the adding logic in one place: inside the add() function. If someday it changes, you can take care of it by changing the add() method, similarly to the technique you already saw in the section on higher order functions. We can also write the add() function using currying:

def add(a: Int)(b: Int) = a + b

def add5 = add(5) _
def add7 = add(7) _

So, what's the difference? These two are essentially the same, but the curry version is used more because it is visually better when the second parameter is a function, because you can use curly braces instead of parentheses:

def using[T](fileName: String)(f: List[String] => T): T = {
       val file = scala.io.Source.fromFile("file.txt").getLines().toList
       f(file)
}

val words = using("Hello.txt") { lines =>
    lines.flatMap(_.split(" "))
}

Let's see another example where currying is useful. Imagine you have a huge list of users and you need to modify each user into some other value. This transformation is handled by a transform() method that takes an instance of Config and a user to output the same user with a slightly changed name:

def transform(config: Config, user: User): User = {
       user.copy(name = s"${user.name} ${config.userNameSuffix}")
}

val transformedUsers = users.map(user => transform(config, user))

And here is a version with currying:

def transform(config: Config)(user: User): User = {
       user.copy(name = user.name + config.userNameSuffix)
}

val transformedUsers = users.map(transform(config))

As you can see, other than the added parentheses, there were no changes to the function. This is cleaner, but a bit harder to read if you don't know about the syntax.

It is possible that you are already using partially applied functions, when dealing with Maps. Using pattern matching, you can use this syntax:

val usersWithId = Map(1 -> User("Alex", 27), 2 -> User("Sam", 23))

val users = usersWithId.map{ tup => s"${tup._1}: ${tup._2.name}" }
  // extracting user's name prepended with her id

But this is ugly. You can use pattern matching instead of accessing tuple's cryptic properties (_1, _2 and so on):

val usersNames = usersWithId.map{ user => user match {
    case (id, value) => s"$id ${value.name}" // using "user" for the value would
  be bad because we already have a "user" as a parameter
  }
}

And here is a version with a partially applied function (with Scala compiler's help):

val usersNames = usersWithId.map {
  case (id, user) => user.name
}

To conclude, use currying when you need to “prepare” functions with some dependencies. Also use it when it makes a more readable code thanks to the curly braces for functional parameters. Partially applied functions may be useful in conjunction with pattern matching to avoid useless repetitions.

NULL HANDLING (OPTION)

Scala has a very neat way to avoid nulls. In fact, if there was no need to keep the compatibility with Java, it's certain that Scala would remove null from the language altogether. Pure functional programming languages don't have null at all, but instead they have a Maybe type that is represented in Scala by Option. Consider this method signature:

def findUserById(id: Int): User

Let's say you don't have access to its internals, and here is an example of this usage:

val user = findUserById(10)

Would you check the resulting user value for being null? If not, have you considered what happens if the user doesn't exist and you try to access the name with user.name, which throws the famous NullPointerException. Instead let's define the method the other way around:

def findUserById(id: Int): Option[User]

Just by reading the signature you may guess what this method would do if it does not find the user in the database: it would return None; otherwise it would be Some(user). So, every time there is a possibility of an absence of the result, use Option. There are different ways to handle this type, and you may do it immediately:

findUserById(10) match {
  case Some(user) => user.name
  case None => "anonymous"
}

// or
val user = findUserById(10).getOrElse(User("anonymous", 0))

Or you may just modify the value inside the Option and leave the handling of None to the layer above.

val name: Option[String] = findUserById(10).map(_.name)

The map() here will transform the result from Option[User] to Option[String] containing the user's name. If there is no user, the final result is still None. It is easier to understand if you imagine Option as a sort of list that is either empty or containing one element.

If you are dealing with something that may return null, just wrap it into an Option, and it will transform the returned null into None. Also, never use the Option's get() method, because it will throw an exception if the element is None, effectively replacing NullPointerException problem with a similar one.

STRICT VERSUS NON-STRICT INITIALIZATION

It's worth writing a few words on a non-strict initialization. There are two kinds of it: lazy values and call-by-name parameters. There is a “code smell” when you know you need the former:

class foo{
  var database = _

  def bar() = {
    database = initDb()
    // … the rest of the code that uses database here …
  }
}

Here we didn't initialize the configuration directly because it would consume a connection from a connection pool, or it would simply throw an exception (cannot be initialized in the constructor). You may also notice a var and a null that are bad, and were discussed previously in this chapter. All of these problems can be solved using lazy:

class foo{
  lazy val database = initDb()

  def bar() = {
    // … the rest of the code that uses configuration here …
  }
}

Other than a benefit, which is an immutable code without null instances, the initialization of the database value becomes thread-safe (it uses double-checked locking internally), so different workers accessing the value won't see an uninitialized database.

Call-by-name parameters are useful when there is a need to wrap your code with some initializations. For instance, if you want to measure code execution with System.nanoTime, the solution is to do the following:

def measure[T](code: => T): String = {
  val start = System.nanoTime
  code
  val end = System.nanoTime
  s"It took ${end - start} nanoseconds to execute the code"
}

measure(1 + 1)

In the measure() function, the code passed in parameters will be executed only when called in the body of the function. This provides better flexibility of what code to test. As a note, in Chapter 9 you will see that there is a better tool for micro-benchmarking than nanoTime.

SUMMARY

This chapter covered the basic building blocks of functional programming. Functional code was compared to its imperative counterpart, and new techniques were detailed that help you write more readable and maintainable code.

Immutability makes it possible to create a thread-safe, stateless code that insists on writing more intermediate values with meaningful names, making the code more readable. Free functions were also covered that don't change the external scope, providing referential transparency that makes unit testing a breeze. Details about why while is bad, and how the recursion may be used for something more than just traversing recursive structures, were also covered.

Higher order functions help you write more concise, declarative code, as you saw in the section about the collection API. Higher order functions also provide you with tools to factorize your code more effectively.

We examined two useful features of functional programming: the Option type and lazy execution. The former helps you avoid NullPointerExceptions, and provides a way to describe a function that may have an absent value just by its return type, while the latter is a nice way to bypass value initialization until later when it's needed.