11.1 Identifiers

The names of variables, functions, classes, and so on are collectively called identifiers. In Scala, you have more choices for forming identifiers than in most other programming languages. Of course, you can follow the time-honored pattern: sequences of alphanumeric characters, starting with an alphabetic character or an underscore, such as input1 or next_token.

Unicode characters are allowed. For example, quantité or ποσό are valid identifiers.

In addition, you can use operator characters in identifiers:

• The ASCII characters ! # % & * + - / : < = > ? @ ^ | ~ that are not letters, digits, underscore, the . , ; punctuation marks, parentheses () [] {}, or quotation marks ’ ` ".

• Unicode mathematical symbols or other symbols from the Unicode categories Sm and So.

For example, ** and √ are valid identifiers. With the definition

val √ = scala.math.sqrt

you can write √(2) to compute a square root. This may be a good idea, provided one’s programming environment makes it easy to type the symbol.

You can also form identifiers from alphanumerical characters, followed by an underscore, and then a sequence of operator characters, such as

Click here to view code image

val happy_birthday_!!! = "Bonne anniversaire!!!"

This is probably not a good idea.

Finally, you can include just about any sequence of characters in backquotes. For example,

val `val` = 42

That example is silly, but backquotes can sometimes be an “escape hatch.” For example, in Scala, yield is a reserved word, but you may need to access a Java method of the same name. Backquotes to the rescue: Thread.`yield`().

11.2 Infix Operators

You can write

a identifier b

where identifier denotes a method with two parameters (one implicit, one explicit). For example, the expression

1 to 10

is actually a method call

1.to(10)

This is called an infix expression because the operator is between the arguments. The operator can contain letters, as in to, or it can contain operator characters—for example,

1 -> 10

is a method call

1.->(10)

To define an operator in your own class, simply define a method whose name is that of the desired operator. For example, here is a Fraction class that multiplies two fractions according to the law

(n₁ / d₁) × (n₂ / d₂) = (n₁n₂ / d₁d₂)

Click here to view code image

class Fraction(n: Int, d: Int) :
  private val num = ...
  private val den = ...
  ...
  def *(other: Fraction) = Fraction(num * other.num, den * other.den)

If you want to call a symbolic operator from Java, use the @targetName annotation to give it an alphanumeric name:

Click here to view code image

@targetName("multiply") def *(other: Fraction) =
  Fraction(num * other.num, den * other.den)

In Scala code, you use f * g, but in Java, you use f.multiply(g).

In order to use a method with an alphanumeric name as an infix operator, use the infix modifier:

Click here to view code image

infix def times(other: Fraction) = Fraction(num * other.num, den * other.den)

Now you can call f times g in Scala.

f repeat { "Hello" }

val smallPrimes = ArrayBuffer[Int]()
smallPrimes += 2 // Binary infix operator, invokes +=(Int)
smallPrimes += (3, 5) // Multiple arguments, invokes +=(Int, Int, Int*)

val twinPrimes = ArrayBuffer[(Int, Int)]()
twinPrimes += (11, 13) // Error

twinPrimes += ((11, 13))

11.3 Unary Operators

Infix operators are binary operators—they have two parameters. An operator with one parameter is called a unary operator.

The four operators +, -, !, ~ are allowed as prefix operators, appearing before their arguments. They are converted into calls to methods with the name unary_operator. For example,

-a

means the same as a.unary_-.

If a unary operator follows its argument, it is a postfix operator. For example, the expression 42 toString is the same as 42.toString.

However, postfix operators can lead to parsing errors. For example, the code

val result = 42 toString
println(result)

yields the error message “Recursive value result needs type.” Since parsing precedes type inference and overload resolution, the compiler does not yet know that toString is a unary method. Instead, the code is parsed as val result = 42.toString(println(result)).

For that reason, Scala now discourages the use of postfix operators. If you really want to use them, you must use the compiler option -language:postfixOps or add the clause

Click here to view code image

import scala.language.postfixOps

11.4 Assignment Operators

An assignment operator has the form operator=, and the expression

a operator= b

means the same as

a = a operator b

For example, a += b is equivalent to a = a + b.

There are a few technical details.

• <=, >=, and != are not assignment operators.

• An operator starting with an = is never an assignment operator (==, ===, =/=, and so on).

• If a has a method called operator=, then that method is called directly.

11.5 Precedence

When you have two or more operators in a row without parentheses, the ones with higher precedence are executed first. For example, in the expression

1 + 2 * 3

the * operator is evaluated first.

In most languages, there is a fixed set of operators, and the language standard decrees which have precedence over which. Scala can have arbitrary operators, so it uses a scheme that works for all operators, while also giving the familiar precedence order to the standard ones.

Except for assignment operators, the precedence is determined by the first character of the operator (see Table 11–1).

Characters in the same row yield operators with the same precedence. For example, + and -> have the same precedence.

Postfix operators have lower precedence than infix operators:

a infixOp bpostfixOp

is the same as

(a infixOp b)postfixOp

11.6 Associativity

When you have a sequence of operators of the same precedence, the associativity determines whether they are evaluated left-to-right or right-to-left. For example, in the expression 17 − 2 − 9, one computes (17 − 2) − 9. The − operator is left-associative.

In Scala, all operators are left-associative except for

• operators that end in a colon (:)

• assignment operators

In particular, the :: operator for constructing lists is right-associative. For example,

1 :: 2 :: Nil

means

1 :: (2 :: Nil)

This is as it should be—we first need to form the list containing 2, and that list becomes the tail of the list whose head is 1.

A right-associative binary operator is a method of its second argument. For example,

2 :: Nil

means

Nil.::(2)

11.7 The apply and update Methods

Scala lets you extend the function call syntax

f(arg1, arg2, ...)

to values other than functions. If f is not a function or method, then this expression is equivalent to the call

f.apply(arg1, arg2, ...)

unless it occurs to the left of an assignment. The expression

f(arg1, arg2, ...) = value

corresponds to the call

Click here to view code image

f.update(arg1, arg2, ..., value)

This mechanism is used in arrays and maps. For example,

Click here to view code image

val scores = scala.collection.mutable.HashMap[String, Int]()
scores("Bob") = 100 // Calls scores.update("Bob", 100)
val bobsScore = scores("Bob") // Calls scores.apply("Bob")

11.8 The unapply Method L2

An apply method takes construction parameters and turns them into an object. An unapply method does the opposite. It takes an object and extracts values from it—usually the values from which the object was, or could be, constructed.

Consider the Fraction class from Section 11.2, “Infix Operators,” on page 151. A call such as Fraction(3, 4) calls the Fraction.apply method which makes a fraction from a numerator and denominator. An unapply method does the opposite and extracts the numerator and denominator from a fraction.

One way to invoke an extractor is in a variable declaration. Here is an example:

Click here to view code image

val Fraction(n, d) = Fraction(3, 4) * Fraction(2, 5)
  // n, d are initialized with the numerator and denominator of the result

This statement declares two variables n and d, both of type Int, and not a Fraction. The variables are initialized with the values that are extracted from the right-hand side.

More commonly, extractors are invoked in pattern matches such as the following:

Click here to view code image

val value = f match
  case Fraction(a, b) => a.toDouble / b
    // a, b are bound to the numerator and denominator
  case _ => Double.NaN

The details of implementing unapply are somewhat tedious. You may want to skip them until after reading Chapter 14.

Since a pattern match can fail. the unapply method returns an Option. Upon success, the Option contains a tuple holding the extracted values. In our case, we return an Option[(Int, Int)].

Click here to view code image

object Fraction :
  def unapply(input: Fraction) =
    if input.den == 0 then None else Some((input.num, input.den))

This method returns None when the fraction is malformed (with a zero denominator), indicating no match.

A statement

val Fraction(a, b) = f

leads to the method call

Fraction.unapply(f)

If the method returns None, a MatchError is thrown. Otherwise, the variables a and b are set to the components of the returned tuple.

Note that neither the Fraction.apply method nor the Fraction constructor are called. However, the intent is to initialize a and b so that they would yield f if they were passed to Fraction.apply. This is sometimes called destructuring. In that sense, unapply is the inverse of apply.

It is not a requirement for the apply and unapply methods to be inverses of one another. You can use extractors to extract information from an object of any type.

For example, suppose you want to extract first and last names from a string:

Click here to view code image

val author = "Cay Horstmann"
val Name(first, last) = author // Calls Name.unapply(author)

Provide an object Name with an unapply method that returns an Option[(String, String)]. If the match succeeds, return a pair with the first and last name. Otherwise, return None.

Click here to view code image

object Name :
  def unapply(input: String) =
    val pos = input.indexOf(" ")
    if pos >= 0 then Some((input.substring(0, pos), input.substring(pos + 1)))
    else None

11.9 The unapplySeq Method L2

The unapply method extracts a fixed number of values. To extract an arbitrary number of values, the method needs to be called unapplySeq. In the simplest case, the method returns an Option[Seq[T]], where T is the type of the extracted values. For example, a Name extractor can produce a sequence of the name’s components:

Click here to view code image

object Name :
  def unapplySeq(input: String): Option[Seq[String]] =
    if input.strip == "" then None else Some(input.strip.split(",?\s+").toSeq)

Now you can extract any number name components:

Click here to view code image

val Name(first, middle, last, rest*) = "John D. Rockefeller IV, B.A."

The rest variable is set to a Seq[String].

11.10 Alternative Forms of the unapply and unapplySeq Methods L3

In Section 11.8, “The unapplyMethod,” on page 155, you saw how to implement an unapply method that returns an Option of a tuple:

Click here to view code image

object Fraction :
  def unapply(input: Fraction) =
    if input.den == 0 then None else Some((input.num, input.den))

But the return type of unapply is quite a bit more flexible.

• You don’t need an Option if the match never fails.

• Instead of an Option, you can use any type with methods isEmpty and get.

• Instead of a tuple, you can use any subtype of Product with methods _1, _2, ..., _n.

• To extract a single value, you don’t need a tuple or Product.

• Return a Boolean to have the match succeed or fail without extracting a value.

In the preceding example, the Fraction.unapply method has return type Option[(Int, Int)]. Fractions with zero denominators return None. To have the match succeed in all cases, simply return a tuple without wrapping it into an Option:

Click here to view code image

object Fraction :
  def unapply(input: Fraction) = (input.num, input.den)

Here is an example of an extractor that produces a single value:

Click here to view code image

object Number :
  def unapply(input: String): Option[Int] =
    try
      Some(input.strip.toInt)
    catch
      case ex: NumberFormatException => None

With this extractor, you can extract a number from a string:

val Number(n) = "1729"

An extractor returning Boolean tests its input without extracting any value. Here is such a test extactor:

Click here to view code image

object IsCompound :
  def unapply(input: String) = input.contains(" ")

You can use this extractor to add a test to a pattern:

Click here to view code image

author match
  case Name(first, last @ IsCompound()) => ...
    // Matches if the last name is compound, such as van der Linden
  case Name(first, last) => ...

Finally, the return type of unapplySeq can be more general than an Option[Seq[T]]:

• An Option isn’t needed if the match never fails.

• Any type with methods isEmpty and get can be used instead of Option.

• Any type with methods apply, drop, toSeq, and either length or lengthCompare can be used instead of Seq.

11.11 Dynamic Invocation L2

Scala is a strongly typed language that reports type errors at compile time rather than at runtime. If you have an expression x.f(args), and your program compiles, then you know for sure that x has a method f that can accept the given arguments. However, there are situations where it is desirable to define methods in a running program. This is common with object-relational mappers in dynamic languages such as Ruby or JavaScript. Objects that represent database tables have methods findByName, findById, and so on, with the method names matching the table columns. For database entities, the column names can be used to get and set fields, such as person.lastName = "Doe".

In Scala, you can do this too. If a type extends the trait scala.Dynamic, then method calls, getters, and setters are rewritten as calls to special methods that can inspect the name of the original call and the parameters, and then take arbitrary actions.

import scala.language.dynamics

Here are the details of the rewriting. Consider obj.name, where obj belongs to a class that’s a subtype of Dynamic. Here is what the Scala compiler does with it.

1. If name is a known method or field of obj, it is processed in the usual way.

2. If obj.name is followed by (arg1, arg2, ...),

   1. If none of the arguments are named (of the form name=arg), pass the arguments on to applyDynamic:

      Click here to view code image

      obj.applyDynamic("name")(arg1, arg2, ...)

   2. If at least one of the arguments is named, pass the name/value pairs on to applyDynamicNamed:

      Click here to view code image

      obj.applyDynamicNamed("name")((name1, arg1), (name2, arg2), ...)

      Here, name1, name2, and so on are strings with the argument names, or "" for unnamed arguments.

3. If obj.name is to the left of an =, call

   Click here to view code image

   obj.updateDynamic("name")(rightHandSide)

4. Otherwise call

   obj.selectDynamic("sel")

Let’s look at a few examples. Suppose person is an instance of a type extending Dynamic. A statement

person.lastName = "Doe"

is replaced with a call

Click here to view code image

person.updateDynamic("lastName")("Doe")

The Person class must have such a method:

Click here to view code image

class Person :
  ...
  def updateDynamic(field: String)(newValue: String) = ...

It is then up to you to implement the updateDynamic method. For example, if you are implementing an object-relational mapper, you might update the cached entity and mark it as changed, so that it can be persisted in the database.

Conversely, a statement

val name = person.lastName

turns into

Click here to view code image

val name = name.selectDynamic("lastName")

The selectDynamic method would simply look up the field value.

Method calls are translated to calls of the applyDynamic or applyDynamicNamed method. The latter is used for calls with named parameters. For example,

Click here to view code image

val does = people.findByLastName("Doe")

becomes

Click here to view code image

val does = people.applyDynamic("findByLastName")("Doe")

and

Click here to view code image

val johnDoes = people.find(lastName = "Doe", firstName = "John")

becomes

Click here to view code image

val johnDoes =
  people.applyDynamicNamed("find")(("lastName", "Doe"), ("firstName", "John"))

It is then up to you to implement applyDynamic and applyDynamicNamed as calls that retrieve the matching objects.

Here is a concrete example. Suppose we want to be able to dynamically look up and set elements of a java.util.Properties instance, using the dot notation:

Click here to view code image

val sysProps = DynamicProps(System.getProperties)
sysProps.username = "Fred" // Sets the "username" property to "Fred"
val home = sysProps.java_home // Gets the "java.home" property

For simplicity, we replace periods in the property name with underscores. (Exercise 13 on page 165 shows how to keep the periods.)

The DynamicProps class extends the Dynamic trait and implements the updateDynamic and selectDynamic methods:

Click here to view code image

class DynamicProps(val props: java.util.Properties) extends Dynamic :
  def updateDynamic(name: String)(value: String) =
    props.setProperty(name.replaceAll("_", "."), value)
  def selectDynamic(name: String) =
    props.getProperty(name.replaceAll("_", "."))

As an additional enhancement, let us use the add method to add key/value pairs in bulk, using named arguments:

Click here to view code image

sysProps.add(username="Fred", password="Secret")

Then we need to supply the applyDynamicNamed method in the DynamicProps class. Note that the name of the method is fixed. We are only interested in arbitrary parameter names.

Click here to view code image

def applyDynamicNamed(name: String)(args: (String, String)*) =
  if name != "add" then throw IllegalArgumentException()
  for (k, v) <- args do
    props.setProperty(k.replaceAll("_", "."), v)

These examples are only meant to illustrate the mechanism. Is it really that useful to use the dot notation for map access? Like operator overloading, dynamic invocation is a feature that is best used with restraint.

11.12 Typesafe Selection and Application L2

In the preceding section, you saw how to resolve selections obj.selector and method calls obj.method(args) dynamically. However, that approach is not typesafe. If selector or method are not appropriate, a runtime error occurs. In this section, you will learn how to detect invalid selections and invocations at compile time.

Instead of the Dynamic trait, you use the Selectable trait. It has methods

Click here to view code image

def selectDynamic(name: String): Any
def applyDynamic(name: String)(args: Any*): Any

The key difference is that you specify the selectors and methods that can be applied.

Let us first look at selection. Suppose we have objects with properties from a cache of database records, or JSON values, or, to keep the example simple, from a map. Then we can select a property with this class:

Click here to view code image

class Props(props: Map[String, Any]) extends Selectable :
  def selectDynamic(name: String) = props(name)

Now let’s move on to the typesafe part. Suppose we want to work with invoice items. Define a type with the valid properties, like this:

type Item = Props {
  val description: String
  val price: Double
}

This is an example of a structural type—see Chapter 18 for details.

Construct instances as follows:

Click here to view code image

val toaster = Props(
  Map("description" -> "Blackwell Toaster", "price" -> 29.95)).asInstanceOf[Item]

When you call

toaster.price

the selectDynamic method is invoked. This is no different from that in the preceding section.

However, a call toaster.brand is a compile-time error since brand is not one of the listed selectors in the Item type.

Next, let’s turn to methods. We want to write a library for making REST calls with Scala syntax. Calls such as

Click here to view code image

val buyer = myShoppingService.customer(id)
shoppingCart += myShoppingService.item(id)

should, behind the scenes, invoke REST requests http://myserver.com/customer/id and http://myserver.com/item/id, yielding the JSON responses.

And we want this to be typesafe. A call to a nonexistent REST service should fail at compile time.

First, make a generic class

Click here to view code image

class Request(baseURL: String) extends Selectable :
  def applyDynamic(name: String)(args: Any*): Any =
    val url = s"$baseURL/$name/${args(0)}"
    scala.io.Source.fromURL(url).mkString

Then constrain to the services that you know to be actually supported.

I don’t have access to a shopping service, but I have a service that produces random nouns and adjectives:

Click here to view code image

type RandomService = Request {
  def nouns(qty: Int) : String
  def adjectives(qty: Int) : String
}

Construct an instance:

Click here to view code image

val myRandomService =
  new Request("https://horstmann.com/random").asInstanceOf[RandomService]

Now a call

myRandomService.nouns(5)

is translated to a call

Click here to view code image

myRandomService.applyDynamic("nouns")(5)

However, if the method name is neither nouns nor adjectives, a compiler error occurs.

Exercises

1. According to the precedence rules, how are 3 + 4 -> 5 and 3 -> 4 + 5 evaluated?

2. The BigInt class has a pow method, not an operator. Why didn’t the Scala library designers choose ** (as in Fortran) or ^ (as in Pascal) for a power operator?

3. Implement the Fraction class with operations + - * /. Normalize fractions, for example, turning 15/−6 into −5/2. Divide by the greatest common divisor, like this:

Click here to view code image

class Fraction(n: Int, d: Int) :
  private val num: Int = if d == 0 then 1 else n * sign(d) / gcd(n, d);
  private val den: Int = if d == 0 then 0 else d * sign(d) / gcd(n, d);
  override def toString = s"$num/$den"
  def sign(a: Int) = if a > 0 then 1 else if a < 0 then -1 else 0
  def gcd(a: Int, b: Int): Int = if b == 0 then abs(a) else gcd(b, a % b)
  ...

4. Implement a class Money with fields for dollars and cents. Supply +, - operators as well as comparison operators == and <. For example, Money(1, 75) + Money(0, 50) == Money(2, 25) should be true. Should you also supply * and / operators? Why or why not?

5. Provide operators that construct an HTML table. For example,

Click here to view code image

Table() | "Java" | "Scala" || "Gosling" | "Odersky" || "JVM" | "JVM, .NET"

should produce

Click here to view code image

<table><tr><td>Java</td><td>Scala</td></tr><tr><td>Gosling...

6. Provide a class ASCIIArt whose objects contain figures such as

 /\_/
( ' ' )
(  -  )
 | | |
(__|__)

Supply operators for combining two ASCIIArt figures horizontally

Click here to view code image

 /\_/     -----
( ' ' )  / Hello 
(  -  ) <  Scala |
 | | |    Coder /
(__|__)    -----

or vertically. Choose operators with appropriate precedence.

7. Implement a class BitSequence that stores a sequence of 64 bits packed in a Long value. Supply apply and update operators to get and set an individual bit.

8. Provide a class Matrix. Choose whether you want to implement 2 × 2 matrices, square matrices of any size, or m × n matrices. Supply operations + and *. The latter should also work with scalars, for example, mat * 2. A single element should be accessible as mat(row, col).

9. Define an object PathComponents with an unapply operation class that extracts the directory path and file name from an java.nio.file.Path. For example, the file /home/cay/readme.txt has directory path /home/cay and file name readme.txt.

10. Modify the PathComponents object of the preceding exercise to instead define an unapplySeq operation that extracts all path segments. For example, for the file /home/cay/readme.txt, you should produce a sequence of three segments: home, cay, and readme.txt.

11. Show that the return type of unapply can be an arbitrary subtype of Product. Make your own concrete type MyProduct that extends Product and defines methods _1, _2. Then define an unapply method returning a MyProduct instance. What happens if you don’t define _1?

12. Provide an extractor for strings where the first component is a title from an appropriate enumeration, and the remainder is a sequence of name components. For example, when called with "Dr. Peter van der Linden", the result would be Some(Title.DR, Seq("Peter", "van", "der", "Linden"). Show how the values can be obtained through destructuring or in a match expression.

13. Improve the dynamic property selector in Section 11.11, “Dynamic Invocation,” on page 159 so that one doesn’t have to use underscores. For example, sysProps.java.home should select the property with key "java.home". Use a helper class, also extending Dynamic, that contains partially completed paths.

14. Define a class XMLElement that models an XML element with a name, attributes, and child elements. Using dynamic selection and method calls, make it possible to select paths such as rootElement.html.body.ul(id="42").li, which should return all li elements inside ul with id attribute 42 inside body inside html.

15. Provide an XMLBuilder class for dynamically building XML elements, as builder.ul(id="42", style="list-style: lower-alpha;"), where the method name becomes the element name and the named arguments become the attributes. Come up with a convenient way of building nested elements.

16. Section 11.12, “Typesafe Selection and Application,” on page 162 describes a Request class that makes requests to https://$server/$name/$arg. However, the URLs of real REST APIs aren’t always that regular. Consider https://www.thecocktaildb.com/api.php. Change the Request class so that it receives a map from method names to template strings, where the $ character is replaced with the argument value.

For this service, the following should work:

Click here to view code image

val cocktailServiceTemplate = Map(
  "cocktail" -> "https://www.thecocktaildb.com/api/json/v1/1/search.php?s=$",
  "ingredient" -> "https://www.thecocktaildb.com/api/json/v1/1/search.php?i=$")
val cocktailService =
  Request(cocktailServiceTemplate).asInstanceOf[CocktailService]

Now the call

Click here to view code image

cocktailService.cocktail("Negroni")

should invoke

Click here to view code image

https://www.thecocktaildb.com/api/json/v1/1/search.php?s=Negroni