Scala code compiles to the Java bytecode and runs on the JVM (Java Virtual Machine). As the name suggests, the JVM was not built specifically for Scala. Because of this, there is a mismatch between what is expressible with the Scala language and what the JVM supports. The consequences are twofold:

• The compiler converts Scala features that are not supported by the JVM into the proper bytecode, mostly by creating wrapper classes. As a result, the compilation of a simple Scala program might lead to the creation of dozens or hundreds of classes, which in turn leads to decreased performance and a higher garbage footprint. These negative consequences, in essence, are just an implementation detail. As the JVM improves, it is possible to optimize the bytecode produced by the compiler for the newer versions of Java without any efforts from the application developer.
• Looking in the opposite direction, there are some features of the platform that are not especially consistent with Scala. Support for them is required, though, partly for Scala-Java interoperability reasons, and partly because if something happens in the underlying runtime, it needs to be expressible in the language. 

For us, as users of the language, the first class of differences is more of a theoretical interest as it is comfortable to assume that the compiler developers are doing their best to generate the bytecode which is of the best standards for the current version of the JVM, so we can just rely on them. Consequences of the second kind concern us more directly because they might influence the way we structure our code, and they definitely affect the code we write to interact with existing libraries, especially Java libraries.

Here, we're talking about features that might negatively impact on our possibility to reason about the code, especially those which break their referential transparency. 

To illustrate the last point, let's take a look at the following code:

scala> import java.util
import java.util
scala> val map = new util.HashMap[String, Int] { put("zero", 0) }
map: java.util.HashMap[String,Int] = {zero=0}
scala> val list = new util.ArrayList[String] { add("zero") }
list: java.util.ArrayList[String] = [zero]
scala> println(map.get("zero"))
0
scala> println(list.get(0))
zero

We've defined a Java HashMap, an ArrayList, put some items in them, and got these back as expected. So far, so good.

Let's push this a bit further:

scala> println(map.get("one"))
null
scala> :type null
Null

For an element that can't be found in the map, we got a null: Null back, which is a bit unexpected. Is it really so bad? Well, it probably is:

scala> println(map.get("one").toString)
java.lang.NullPointerException
  ... 40 elided

We've got a NullPointerException which would crash our program at runtime if not caught! 

OK, but we can check if the returned element is null. We just need to remember to do this each time we call a function that can potentially return null. Let's do this with the list:

scala> list.get(1) match {
     | case null => println("Not found")
     | case notNull => println(notNull)
     | }
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
  at java.util.ArrayList.rangeCheck(ArrayList.java:653)
  at java.util.ArrayList.get(ArrayList.java:429)
  ... 49 elided

Oh, the list does not return null for absent elements, it just throws the IndexOutOfBoundsException straight away! Looks like we need to add a catch clause to our call so that we can make it safe...

At this moment, our point is already clear – it is hard or impossible to reason about what the possible result of execution of some code written in this style is without looking at the JavaDocs, and eventually at the implementations' source code. The reason for this is that the functions we're calling can return the result in a way that's not encoded in their types. In the first example, the function returns a special null value for the case where there's no element in the collection. But this special value could also be something different! Another example is -1 for the indexOf method that's defined on the ArrayList. Yet another case is the indication of the impossibility to perform an operation, which is done by throwing an exception.

In a way, the functions we've called altered the environment they were executed in. In the case of an exception, the execution path of the program changed to propagate the exception, and in the case of null being returned, the caller's expectations had changed, unfortunately not at compile time but at runtime.

In functional programming, we call such a behavior an effect, and strive to express this effect at the type level. Effects in functional programming (FP) overlap with side-effects, but represent a wider concept. For example, an effect of optionality (returning null) of the result is not a side-effect. 

Effects have the advantage that they don't need to be implemented on the language level. Because of this, the same effect can be designed in different ways, depending on the goals and architectural considerations of the library author. Most importantly, they can be extended and combined, which allows us to represent complex sets of effects in a structured and type-safe way.

In further sections of this chapter, we will take a look at four different kinds of effects that are available in the standard library, starting with Option.