Puzzler 30

Quite the Outspoken Type

Scala's powerful type inference allows you to omit type declarations in many places, leaving the compiler to figure things out. For non-trivial expressions, though, explicitly specifying types is regarded as good practice.

The following example defines two versions of an implicit conversion function from numeric strings to integers: first without, then with, an explicit type declaration. A println statement that relies on the two implicit variants follows. What is the result of executing the following code?

  class QuietType {
    implicit val stringToInt = (_: String).toInt
    println("4" - 2)
  }
  class OutspokenType {
    implicit val stringToInt: String => Int = _.toInt
    println("4" - 2)
  }
  
  new QuietType()
  new OutspokenType()

Possibilities

1. Prints:

     2
     2
   

2. The first statement fails to compile and the second prints:

     2
   

3. Both statements fail to compile.
4. The first statement prints:

     2
   

   and the second throws a runtime exception.

Explanation

You may question whether the type inferred for QuietType.stringToInt and the type explicitly declared for OutspokenType.stringToInt are really the same, but that is indeed the case:

  // in QuietType
  scala> implicit val stringToInt = (_: String).toInt
  stringToInt: String => Int = <function1>
  
  // in OutspokenType
  scala> implicit val stringToInt: String => Int = _.toInt
  stringToInt: String => Int = <function1>

Having verified this, you may suspect instead that the implicits are not applicable to the println statements, resulting in compiler errors. Surely, though, explicitly specifying the return type that the compiler otherwise infers will not affect the behavior of the code?

Actually, it does—the correct answer is number 4:

  scala> new QuietType()
  2
  
  scala> new OutspokenType()
  java.lang.StackOverflowError
    at OutspokenType$$anonfun$1.apply(<console>:8)

A stack overflow error? How does that come about? In order to understand what happens when you construct an instance of OutspokenType, it is useful to start by looking at what is going on in QuietType.

Specifically, zoom in on the moment when the compiler has parsed the beginning of the declaration of QuietType.stringToInt and is trying to make sense of the expression (_: String).toInt. At this point, the type of stringToInt is as yet undetermined, since the compiler is still in the process of trying to figure it out.

Unfortunately, the compiler quickly runs up against a problem: there is, in fact, no toInt method on String. As per the language specification,[1] the compiler starts searching among all implicits in scope[2] and, conveniently, finds a suitable option: Predef.augmentString, which converts the string to a StringOps object on which toInt can be called.

The implicit stringToInt method can then be compiled successfully. It is assigned the type String => Int and is successfully found and applied when the subsequent println("4" - 2) statement is processed.

So far, so good. What, then, causes OutspokenType to behave so differently? Again, focus on the moment the compiler is about to process the expression _.toInt in the body of OutspokenType.stringToInt. As before, the compiler needs to invoke an implicit search, since String does not have a toInt method. The crucial difference lies here: because the type of OutspokenType.stringToInt is already known, it is included in the list of implicits to consider. As luck would have it, it is not only included in the list, it is even applicable, since Int also happens to have a toInt method.

Instead of the expected conversion to StringOps, what you actually end up with, therefore, is an implicit that immediately calls itself. When invoked, this results in the endless loop observed.

Discussion

An important detail not examined so far is why exactly the compiler chooses stringToInt over augmentString in OutspokenType, when both implicits are in scope. Surprisingly, it is not the result of static overloading resolution,[3] which is used to determine which implicit to apply. In fact, if you make a small change and declare stringToInt as an implicit def rather than a val, the compiler indeed complains about ambiguous implicits:

  scala> class OutspokenType2 {
           implicit def stringToInt(s: String): Int = s.toInt
           println("4" - 2)
         }
  <console>:8: error: type mismatch;
   found   : s.type (with underlying type String)
   required: ?{def toInt: ?}
  Note that implicit conversions are not applicable because
    they are ambiguous:
   both method augmentString in object Predef of type 
     (x: String)scala.collection.immutable.StringOps
   and method stringToInt in class OutspokenType2 of type
     (s: String)Int
   are possible conversion functions from s.type 
     to ?{def toInt: ?}
           implicit def stringToInt(s: String): Int = s.toInt
                                                      ^

The short answer is that function vals take precedence over defs during implicit search. The point is that this behavior is not justified by the language specification, which specifies static overloading resolution as the algorithm for determining implicit precedence. Yet overloading resolution treats "regular" defs and function vals equivalently and does not exhibit the preference for function vals that is observed with implicits:

  scala> object DefAndFunVal extends App {
           def method(s: String): Int = ???
           val method: String => Int = ???
           println(method("hello"))
         }
  <console>:11: error: ambiguous reference to overloaded
    definition,
  both value method in object DefAndFunVal of type 
    => String => Int
  and  method method in object DefAndFunVal of type
    (s: String)Int
  match argument types (String) and expected result type Any
           println(method("hello"))
                   ^

It appears that Scala's behavior when distinguishing between def foo: T and val foo: () => T during static overloading resolution is undefined: neither the fact that the compiler treats them equivalently for explicit methods, nor that the compiler prefers the val for implicit methods, are covered by the language specification. As explained by Jason Zaugg, a member of the Scala compiler team:

Both implicit and explicit use of static overload resolution between methods and nullary methods (i.e., methods without parameters) returning function types is implementation-specific and not covered by the [language] specification.

Prefer defs over vals when defining implicit conversions to ensure potential "ambiguous implicit" errors are not hidden by implementation-specific compiler behavior. Improve readability and safety by explicitly specifying the return type of implicit defs.

Footnotes for Chapter 30:

[1] Odersky, The Scala Language Specification, Section 6.26. [Ode14]

[2] See Puzzler 29 for a more detailed discussion of implicit resolution.

[3] Odersky, The Scala Language Specification, Section 7.2. [Ode14]