Puzzler 18

Information Overload

Overloaded methods are a common way of providing functions that can be applied to various combinations of arguments. Some care is required here, since new versions of an overloaded method can mean that the compiler needs to distinguish between multiple applicable versions where the method is called. If the compiler cannot identify a single, most applicable method, compilation will fail:

  def foo(n: Int, a: Any) { 
    println(s"n: ${n}, a: ${a}") }
  
  scala> foo(1, 2)
  n: 1, a: 2
  
  object A {
    def foo(n: Int, a: Any) { 
      println(s"n: ${n}, a: ${a}") }
    def foo(a: Any, n: Int) { 
      println(s"a: ${a}, n: ${n}") }
  }
  
  scala> A.foo(1, 2)
  <console>:10: error: ambiguous reference to 
    overloaded definition,
  both method foo in object A of type (a: Any, n: Int)Unit
  and  method foo in object A of type (n: Int, a: Any)Unit
  match argument types (Int,Int)
                A.foo(1, 2)
                  ^

In the following example, the method signatures appear to be designed to prevent such ambiguous method calls. What is the result of executing the following code?

  object Oh {
    def overloadA(u: Unit) = "I accept a Unit"
    def overloadA(u: Unit, n: Nothing) = 
        "I accept a Unit and Nothing"
    def overloadB(n: Unit) = "I accept a Unit"
    def overloadB(n: Nothing) = "I accept Nothing"
  }
  
  println(Oh overloadA 99)
  println(Oh overloadB 99)

Possibilities

1. The first statement fails to compile and the second prints:

     I accept Nothing
   

2. The first statements prints:

     I accept a Unit
   

   and the second fails to compile.

3. Both statements fail to compile.
4. Prints:

     I accept a Unit
     I accept a Unit
   

Explanation

Since Int does not inherit from Unit, you may suspect that either the first or both statements fail to compile. Or you may assume that the compiler is able to treat the argument 99 as a Unit, and deduce that "I accept a Unit" would be printed in both cases. But surely the compiler cannot accept Int as a Unit in the first case, but reject in the second?

Oh yes, it can. The correct answer is indeed number 2:

  scala> println(Oh overloadA 99)
  <console>:9: warning: a pure expression does nothing 
    in statement position; you may be omitting necessary 
    parentheses
                println(Oh overloadA 99)
                                     ^
  I accept a Unit
  
  scala> println(Oh overloadB 99)
  <console>:9: error: overloaded method value overloadB
      with alternatives:
    (n: Nothing)String <and>
    (n: Unit)String
   cannot be applied to (Int)
                println(Oh overloadB 99)
                           ^

To be clear: the compiler is not failing here because an overloaded method that is also applicable has been added, resulting in an ambiguous reference. The overloadB(n: Nothing) method does not accept an Int argument. What is instead happening here is that a method that was applicable now no longer applies because of an overloaded method that also does not apply at the point where the method is called.

For an explanation of this seemingly extraordinary behavior, you must dive into the details of Scala's overloading resolution.[1] If an identifier, such as overloadA, refers to multiple members of a class, Scala applies a two-stage algorithm to determine, if possible, the appropriate member to invoke.

In the first stage, the compiler compares the "shapes" of the member declarations with the invocation to see which of the possible alternatives "looks right." Roughly speaking, the compiler looks at the number of parameters of each overloaded alternative and compares these with the invocation.[2] If precisely one alternative is feasible, it is chosen.

This convenient shortcut, which allows the compiler to avoid potentially expensive type-based resolution, is sufficient in the first case: only one of the alternatives for overloadA has the "right shape" (i.e., takes one argument), so this option is chosen by the compiler. One hurdle remains: how to get the Int argument 99 to match with the expected Unit type?

Here, one of Scala's value conversions[3] comes into play. These implicit conversions can be applied by the compiler to convert a value type to a different expected type, if needed. In this case, value discarding is applied: the compiler converts the Int to the expected Unit type by embedding it in the term { 99; () }. This, incidentally, causes the observed compiler warning.

In the case of overloadB, however, shape-based resolution is not sufficient to determine a unique alternative, since both options take one argument. The compiler therefore proceeds to the second stage of overloading resolution, which attempts to find precisely one option with the most specific parameter type.[4]

Herein lies the explanation for the observed behavior, because the first step of this type-based resolution stage is to determine the types of all of the supplied arguments without considering the expected types of the possible methods to be called. In this case, the process is equivalent to evaluating val arg = 99 (i.e., as opposed to val arg: Unit = 99) and looking at the resulting type. This seems reasonable enough, since at this point the compiler does not yet know which of the methods will be invoked, so it also does not know what the expected types will be.

The argument 99 is thus determined to be an Int, and now the compiler proceeds to try to identify a most specific overloaded method to which the argument can be applied. But Int inherits neither from Unit nor from Nothing, so none of the alternatives are applicable!

Furthermore, the value conversion, which in the case of overloadA turns 99 into a Unit, is also not available, since it only applies if the expected type of the value is known. But since the compiler has not been able to identify a most specific method to call, it also does not know the expected type that would be required. Ergo: neither of the two alternatives are applicable, so the code fails to compile.

Discussion

A variation of this problem that is likely to occur in production code involves methods with empty parameter lists. These can be converted to () => T function types via eta expansion where such a type is expected, but this conversion cannot be taken into account by the compiler during a type-based overloading resolution:

  object Oh2 {
    def nonOverloadedA(f: () => Any) =
      "I accept a no-arg function"
    def overloadedA(f: () => Any) =
      "I accept a no-arg function"
    def overloadedA(n: Nothing) = "I accept Nothing"
  }
  
  def emptyParamList() = 99
  
  scala> Oh2 nonOverloadedA emptyParamList
  res0: String = I accept a no-arg function
  
  scala> Oh2 overloadedA emptyParamList
  <console>:10: error: overloaded method value overloadedA
      with alternatives:
    (n: Nothing)String <and>
    (f: () => Any)String
   cannot be applied to (Int)
                Oh2 overloadedA emptyParamList
                    ^

What to do about this potential problem? Since the error occurs only where the method is invoked, you as the code's author have few options beyond avoiding overloaded methods of the same shape entirely. As a user, there is also little you can do other than to keep a keen eye out for implicit type conversions: unfortunately, there are currently no compiler options that flag applications of these automatically.

Watch out for type conversions inserted by the compiler that depend on the expected type, such as value discarding, view application (i.e., applying implicit conversions), and eta expansion. These conversions may no longer be applicable if overloaded versions of the invoked method are added. Be aware that defining an overloaded method with the same shape as an existing alternative can cause calling code to fail to compile regardless of the parameter types of the new method.

---

Footnotes for Chapter 18:

[1] Odersky, The Scala Language Specification, Section 6.26.3. [Ode14]

[2] Odersky, The Scala Language Specification, Section 6.26.3. [Ode14]

[3] Odersky, The Scala Language Specification, Section 6.26.1. [Ode14]

[4] Odersky, The Scala Language Specification, Section 6.26.3. [Ode14]

[5] Zaugg, "Why `avoid method overloading'?". [Zau10b]

[6] Mathias, "The Magnet Pattern." [Mat12]