Puzzler 32

Set the Record Straight

The Scala collections library provides convenience methods, such as toMap, toList, etc., that allow easy conversion between different collection types. The following program shows one of these in action. What does it do?

  val numbers = List("1", "2").toSet() + "3"
  println(numbers)

Possibilities

1. Prints:

     Set(1, 2, 3)
   

2. Fails to compile.
3. Prints:

     false3
   

4. Prints:

     123
   

Explanation

The candidate answer that looks most probable is perhaps the first one:

  Set(1, 2, 3)

And that would indeed be the correct answer if the first line were:

  val numbers = List("1", "2").toSet + "3"

However, the original program produces an entirely different result.

  scala> val numbers = List("1", "2").toSet() + "3"
  numbers: String = false3

The correct answer is number 3. The extra parentheses after toSet are the only difference between the two statements. How could this cause the results be so different?

Let's examine closely how the compiler parses the first statement. The method toSet on a List[A] clearly converts the list to a set, but a careful look at the Scaladoc reveals two important facts:

  def toSet[B >: A]: Set[B]

First, the method itself takes no parameters. Recall that in Scala, if a method is defined without parentheses, it can only be invoked without parentheses:

  scala> def noParens = "foo"
  noParens: String
  
  scala> noParens
  res1: String = foo
  
  scala> noParens()
  <console>:1: error: not enough arguments for method apply: 
    (index: Int)Char in class StringOps.
  Unspecified value parameter index.
                noParens()
                        ^

Second, the element type of the resulting set can be different than that of the original list. More specifically, the type of the set elements can be a supertype of the list element type.

You could provide the desired element type explicitly:

  scala> def withParens() = "bar"
  withParens: ()String
  
  scala> withParens
  res0: String = bar
  
  scala> withParens()
  res1: String = bar

  scala> List("1", "2").toSet[AnyRef]
  res1: scala.collection.immutable.Set[AnyRef] = Set(1, 2)

Needless to say, if the type parameter is not provided, the compiler needs to infer it. Usually, it is fairly obvious—the element type of the resulting collection is the same as the element type of the source collection:

  scala> List("1", "2").toSet
  res2: scala.collection.immutable.Set[String] = Set(1, 2)

In this case, however, the additional set of parentheses forces the compiler to reason differently. Because toSet cannot be invoked with parentheses, the compiler can only interpret the parentheses as an attempt to invoke apply on the result of calling toSet. Coincidentally, trait Set[A] happens to have an apply method, which is equivalent to method contains:

  def apply(elem: A): Boolean

Tests if some element is contained in this set.

Hence, the compiler treats the invocation of toSet with parentheses as:

  List("1", "2").toSet.apply() // apply() == contains()

But what is the element being tested for in this case? Method apply definitely expects an argument (its parameter elem does not have a default value) and it looks as if none is being provided. Here a compiler feature called argument adaptation comes into play—the compiler adapts the argument list, inserting the Unit value, (), to match the single parameter specified by the method declaration. You can verify this by compiling the same line of code with the -Ywarn-adapted-args option:[1]

  scala> List("1", "2").toSet.apply()
  <console>:8: warning: Adapting argument list by inserting (): 
    this is unlikely to be what you want.
          signature: GenSetLike.apply(elem: A): Boolean
    given arguments: <none>
   after adaptation: GenSetLike((): Unit)
                List("1", "2").toSet.apply()

  def tell(o: Any): Unit

  tell(a, b, c)

  tell((a, b, c))

  tell()

  tell(())

As a result of argument adaptation, the final expression is actually:

  List("1", "2").toSet.apply(())

At this point, the compiler must determine whether the set returned by toSet contains the Unit value. How does that even compile, since the Unit value is clearly not a String? Recall that the compiler can infer the element type of the resulting set to be any supertype of String.[3] As illustrated in the Scala type hierarchy,[4] the common supertype of Unit and String is Any. The resulting expression is:

  List("1", "2").toSet[Any].apply(())

This, unsurprisingly, evaluates to false, since the Unit value is not in the List. Thus the actual expression being evaluated is:

  false + "3"

Since Scala, like Java, allows Strings to be concatenated with any other type, this compiles and results in the string "false3".[5]

Discussion

You have already seen that omitting the parentheses after the call to toSet produces the expected result. Note, though, that if you wanted to walk through how the expression is evaluated step-by-step, and what types are inferred along the way, you would find this hard to achieve in the REPL:

  scala> val list = List("1", "2")
  list: List[String] = List(1, 2)
  
  scala> val set = list.toSet
  set: scala.collection.immutable.Set[String] = Set(1, 2)
  
  scala> val result = set()
  <console>:9: error: not enough arguments for method apply:
    (elem:String)Boolean in trait GenSetLike.
  Unspecified value parameter elem.
         val result = set()
                         ^

The last statement here fails to compile because the type widening that turns Set[String] into Set[Any] does not take place. That is, the element type of the set has already been inferred as String, so it is no longer possible to check whether the Unit value, which is clearly not a String, is contained in the set.

In the first line of the example program, on the other hand, the type inferencer is able to infer Any as the element type of the resulting set as it always examines entire expressions.

If you want to ensure the original collection type is preserved, you can use the more general conversion method to, defined on List[A] (or any Traversable[A], for that matter), with the following signature:

  def to[Col[_]]: Col[A]

Calling this method instead of toSet results in a compiler error:

  scala> List("1", "2").to[Set]() + "3"
  <console>:7: error: not enough arguments for method to: 
    (implicit cbf: scala.collection.generic.CanBuildFrom[
    Nothing,String,Set[String]])Set[String].
  Unspecified value parameter cbf.
                List("1", "2").to[Set]() + "3"
                                      ^

Invoking to correctly (i.e., without parentheses) works as expected:

  scala> List("1", "2").to[Set] + "3"
  res0: scala.collection.immutable.Set[String] = Set(1, 2, 3)

Include empty parentheses in method invocations only for side-effecting methods. Beware of unintended type widening caused by methods on collections that allow the element type of a returned collection to be wider than the original element type.

Footnotes for Chapter 32:

[1] Starting with Scala 2.11, this behavior has been deprecated and the compiler warning is issued by default. See Puzzler 26 for a more detailed discussion of -Ywarn-adapted-args and related compiler options.

[2] See Puzzler 26 for a related discussion.

[3] See Puzzler 25 for a related discussion.

[4] See Figure 27.0 here.

[5] See Puzzler 36 for a related discussion.