The Grain of a Programming Language

Every programming language has a grain — if you work with the grain, things go smoothly; if you go against the grain, things are more difficult.

For example, it has always been possible to write Java code in a functional style, but few programmers did before Java 8 — for good reasons. Look what was required in Java 1 to do something as simple as calculating the sum of a list of numbers by reducing them with the addition operator.

Java 1 did not have first-class functions, so we would have had to define our own interfaces for function types:

public interface Function2 {
    Object apply(Object arg1, Object arg2);
}

Then we would have had to write the reduce higher-order function, hiding the iteration and mutation required by the Vector class. (The Java standard library didn’t include the collections framework yet).

public class Vectors {
    public static Object reduce(Vector l, Object initial, Function2 f) {
        Object result = initial;
        for (int i = 0; i < l.size(); i++) {
            result = f.apply(result, l.get(i));
        }
        return result;
    }

    // ... and other operations on vectors
}

We would have had to define a separate class for every function you wanted to pass to our reduce function. The addition operator couldn’t be passed around as a value, and the language had no method references, lambdas or closures back then, not even inner classes. Nor did Java 1 have generics or autoboxing — we would have had to cast the arguments to the expected type and write the boxing between reference types and primitives.

public class AddIntegers implements Function2 {
    public Object apply(Object arg1, Object arg2) {
        int i1 = ((Integer) arg1).intValue();
        int i2 = ((Integer) arg2).intValue();
        return new Integer(i1 + i2);
    }
}

And finally we could use all that to calculate the sum:

int sum = ((Integer) Vectors.reduce(counts, new Integer(0), new AddIntegers()))
    .intValue();

That’s a lot of effort for what can be achieved with a single expression in a modern language, but that’s not the end of it. Because Java had no standard function types, we couldn’t easily combine different libraries written in a functional style. We had to write adapter classes to map between the function types defined in different libraries. And because the virtual machine had no JIT and a simple garbage collector, functional code would have worse performance than the imperative alternative.

There was just not enough pay-off. Java programmers found it easier to write imperative code that iterated over collections and mutated state.

int sum = 0;
for (int i = 0; i < counts.size(); i++) {
    sum += ((Integer)counts.get(i)).intValue();
}

Writing functional code went against the grain of Java.

Every programming language has a grain. It’s easier to work with the grain of a language. Going against the grain involves constant effort with an uncertain pay-off. A language’s grain forms over time as its designers and users build a common understanding of how language features interact, and encode their understanding and preferences in libraries that others build upon. The grain influences the way that programmers write code in the language, which influences the evolution of the language, its libraries and programming tools, changing the grain, altering the way that programmers write code in the language, on and on in a continual cycle of mutual feedback and evolution.

For example, Java 1.1 added anonymous inner classes to the language, and Java 2 added the collections framework to the standard library. Anonymous inner classes meant that we wouldn’t need to write a named class for each function we wanted to pass to our reduce function, but the resulting code would arguably be harder to read.

int sum = ((Integer) Lists.reduce(counts, new Integer(0),
    new Function2() {
        public Object apply(Object arg1, Object arg2) {
            int i1 = ((Integer) arg1).intValue();
            int i2 = ((Integer) arg2).intValue();
            return new Integer(i1 + i2);
        }
    })).intValue();

Functional idioms still went against the grain of Java 2.

Java 5 was the next release that significantly changed the language. It added generics and autoboxing, which improved type safety and reduced boilerplate code:

public interface Function2<A,B,R> {
    R apply(A arg1, B arg2);
}

int sum = Lists.reduce(counts, 0,
    new Function2<Integer, Integer, Integer>() {
        @Override
        public Integer apply(Integer arg1, Integer arg2) {
            return arg1 + arg2;
        }
    });

Google’s Guava library included some common higher-order functions over collections, although reduce was not among them. Even the authors of Guava recommended writing imperative code where possible, because it had better performance and was usually easier to read.

Functional programming still went largely against the grain of Java 5, but we could see the start of a trend.

Java 8 added anonymous functions (aka “lambda expressions”) and method references to the language, and the Streams API to the standard library. The compiler and virtual machine optimised lambdas to avoid the performance overhead of anonymous inner classes. The Streams API fully embraced functional idioms.

int sum = counts.stream().reduce(0, Integer::sum);

Still, it wasn’t entirely plain sailing. We still could not pass the addition operator as a parameter to our reduce function, but we now had have the standard library function Integer::sum that did the same thing. Java’s type system still created awkward edge cases because of its distinction between reference and primitive types. The Streams API was missing some common higher-order functions that we would expect to find if coming from a functional language (or even Ruby). Checked exceptions didn’t play well with the Streams API and functional programming in general. And making immutable classes with value semantics still involved a lot of boilerplate code. But with Java 8, Java had fundamentally changed to make a functional style work, if not completely with the grain, at least not against it.

In the case of Java, the grain of the language, and the way programmers adapted to it, evolved through several distinct programming styles. Kotlin has features designed specifically to simplify interoperability with code written in these styles.

An Opinionated History of Java Programming Style

Like ancient poets, we divide the development of Java programming style into four distinct ages: the Primeval Age, the Beans Age, the Enterprise Age, and the Modern Age. We’ll use these names as a convenient shorthand when discussing Kotlin’s interop features.

The Grain of Kotlin

Kotlin is a young language, but it clearly has a different grain to Java.

The “Why Kotlin” section of kotlinlang.org lists four design goals: “Concise”, “Safe”, “Interoperable”, and “Tool-friendly”. The designers of the language and its standard library also encoded implicit preferences that contribute to these design goals.

Kotlin prefers the transformation of immutable data to mutation of state.

Data classes make it easy to define new types with value semantics. The standard library makes it easier and more concise to transform collections of immutable data than iterate and mutate data in place.

Kotlin prefers behaviour to be explicit.

For example, there is no implicit coercion between types, even from smaller to larger range. Java implicitly converts int values to long values, because there is no loss of precision. In Kotlin you have to explicitly call Int.toLong(). The preference for explicitness is especially strong when it comes to control flow. Although you can overload arithmetic and comparison operators for your own types, you cannot overload the shortcut logical operators (&& and ||), because that would allow you to define different control flow. Polymorphism is opt-in, not opt-out.

Kotlin prefers static over dynamic binding.

Kotlin encourages a type-safe, compositional coding style. Extension functions are bound statically. By default, classes are not extensible and methods are not polymorphic. You must explicitly opt in to polymorphism and inheritance. If you want to use reflection, you have to add a platform-specific library dependency. Kotlin is designed from the outset to be used with a language aware editor that statically analyses the code to guide the programmer automate navigation and automate program transformation.

Kotlin doesn’t like special cases.

Compared to Java, Kotlin has fewer special cases that interact in unpredictable ways. There is no distinction between primitive and reference types. There is no void type for functions that return but do not return a value — functions in Kotlin either return a value, or never return at all. Extension functions allow you to add new operations to existing types that look the same at the call point. You can write new control structures as inline functions and the break, continue and return statements act the same as they do in built-in control structures.

Kotlin breaks its own rules to make migration easier.

The Kotlin language has features to allow idiomatic Java and Kotlin code to coexist in the same codebase. Some of those features remove guarantees provided by type checker, and should only be used to interop with legacy Java. For example, lateinit opens a hole in the type system so that Java dependency injection frameworks that initialise objects by reflection can inject values through the encapsulation boundaries that are normally enforced by the compiler. If you declare a property as lateinit var, it’s up to you to ensure the code initialises the property before reading it — the compiler will not catch your mistakes.

Adapting to Kotlin’s Grain

When we look back on the earliest code we wrote in Kotlin, it tends to look like Java dressed in Kotlin syntax. We came to Kotlin after years writing a lot of Java, and had ingrained habits that affected how we wrote Kotlin code. We wrote unnecessary boilerplate, didn’t make good use of the standard library, and avoided using null because we weren’t yet used to the type checker enforcing null safety. The Scala programmers on our team went too far the other way — their code looked like Kotlin cosplaying as Scala cosplaying as Haskell. None of us had yet found the sweet spot that comes with working with the grain of Kotlin.

The path to idiomatic Kotlin is complicated by the Java code we have to keep working along the way. In practice, it is not enough to just learn Kotlin. We have to work with the different grains of Java and Kotlin, being sympathetic to both as we gradually transition from one to the other.

Refactoring to Kotlin

When we started our journey to Kotlin we were responsible for maintaining and enhancing business-critical systems. So we were never able to focus only on converting our Java codebase to Kotlin. We always had to migrate code to Kotlin at the same time as changing the system to meet new business needs, maintaining a mixed Java/Kotlin codebase as we did so. We managed the risk by working in small changes, making each easy to understand, and cheap to discard if we found out it broke something. Our process was to first convert Java code to Kotlin, giving us a Java-esque design in Kotlin syntax. We then incrementally applied Kotlin language features to make the code increasingly easier to understand, more type safe, more concise, and with a more compositional structure that is easier to change without unpleasant surprises.

Small, safe, reversible changes that improved the design: we refactored from idiomatic Java to idiomatic Kotlin.

These days most projects are multilingual. A “Java” web app, for example, will involve code written in Java, HTML, CSS, JavaScript, at least one templating language, maybe more if you have templating on the server and in the browser, JSX if you’re using React, and one or more query languages (SQL, etc). It’s common to refactor logic between these languages: persistence logic from Java into a query language or vice versa, browser-side presentation between Javascript code and CSS rules or HTML elements as browsers evolve, or between JavaScript code manipulating the DOM and browser-side templating; server-side presentation between the template language and functions or objects in the host language; and so on.

Refactoring between languages is usually harder than refactoring within a single language because refactoring tools do not work well across the boundaries between the languages, if they work at all. Porting logic from one language to another must be done manually, which takes longer and introduces more risk. Once multiple languages are in use, the language boundary impedes refactoring because when you refactor code in one language, the IDE does not update dependent code written in other languages to be compatible.

What makes the combination of Java and Kotlin unique is how seamless is the boundary between the two languages. Thanks to the design of the Kotlin language, the way it is mapped to the JVM platform, and JetBrains’ investment in developer tooling, refactoring Java to Kotlin and refactoring a combined Java/Kotlin codebase is almost as easy as refactoring in a single codebase.

Our experience has been that we can refactor Java to Kotlin without impacting productivity, and that as more of the codebase is converted to Kotlin, productivity accelerates.

What Are We Working On?

In the chapters that follow we demonstrate refactorings from Java to Kotlin in the codebase of Travelator, a (fictional) application for planning and booking international, overland travel. Our users plan routes by sea, rail, and road; search for places to stay and sights to see; compare their options by price, time and spectacle; book their trips; and share photos and writings from their travels.

Travelator has web and mobile front-ends that invoke web services via HTTP.

Each chapter pulls an informative example from a different part of the Travelator system, but they share common domain concepts: money, currency conversion, journeys, itineraries, bookings, and so on.

We hope that, like our Travelator application, this book will help you plan your journey from Java to Kotlin.

How This Book is Organised

This book is about how to transition from Java to Kotlin, mainly focused on code, but touching on projects and organisations. The chapters each address an aspect of this transition, looking at some aspect of typical Java projects that can be improved on the journey. They are named in the pattern Java Way to Kotlin Way, where we recommend that you prefer the latter over the former. Maybe Kotlin makes easier an approach that was difficult in Java, or Kotlin discourages an approach that is common in Java, to guide design in a direction that is less error prone, more concise, and more tool-friendly.

We don’t just recommend you adopt the Kotlin way though; the chapters also show how to make the transformation. Not by just re-writing the Java, but by gradually refactoring it to Kotlin it in a way that is safe and allows us to maintain a mixed language codebase.

How did we choose the topics? We looked for places where the grain of Kotlin and Java differ significantly. The topics are not exhaustive: we cannot cover every way in which the languages differ. We cover the areas that we found most notable as we adopted Kotlin. We also favour transformations that result in code that is less complex.

Complexity

How should we judge the internal quality of our software? Assuming that it does what our customers want or need it to do, how can we compare two potential implementations, or decide whether a change makes one better or worse? The answer that your authors choose is complexity. Other things being equal, we favour simple designs that yield predictable behaviour.

Of course to some extent simplicity and complexity are in the eye of the beholder. Even your authors sometimes disagree over whether one implementation or another is better. Where we disagree, we sometimes discuss it in the relevant chapter. We do though have a shared belief in the power of functional programming to reduce the complexity of our systems, especially when combined with object-oriented message passing.

Java has been moving in this direction over the years. Scala ran full-tilt towards functional programming but away from OO. We have found that the grain of Kotlin allows the mixing of functional and object programming in a way that reduces complexity and allows the writing of high-quality software by mere mortal developers.

Code Conventions

Our code follows (our interpretation of) the standard coding conventions of Java and Kotlin where possible.

We have though had to split more lines than we might usually to make the code fit horizontally in the pages of the book. Where our production Kotlin would usually have up to 4 or 5 parameters to a function definition, or arguments to a function call, on one line; to fit in the book we will often move to one item per line after only 3. That said, we find that Kotlin feels natural taking more vertical space than Java, and scrolling sideways is almost as inconvenient in an IDE as a book - so maybe our production style will move to be more like that you see here.

We will sometimes hide code that isn’t relevant to the discussion. A line comment that starts with an ellipsis of three dots indicates that we have omitted some code for clarity or brevity. For example, in the following code, the comment is not part of the example, but indicates that we have omitted overloads of the Money function that are not relevant to the text.

fun Money(amount: String, currency: Currency) =
    Money(BigDecimal(amount), currency)

// ... and other convenience overloads

Directions in Code

There are lots of places in this book where we show how to move code, or some aspect of code, from one place to another. For example in Chapter 3, we talk about moving the mutability in an interaction from a mutable class to a mutable property. We’d like to have a shorthand for the direction of these changes, but there are many ways to think about the orientation of code: base or super classes, layer diagrams, onion architecture, hexagaonal architecture etc. In some cases our execution paths are top-down, some bottom up, some outside in, some outside to inside to outside to inside to outside.

Unless stated otherwise, we will refer to direction in terms of dependencies, with the entry points (main and event handlers) at the top of the system. This top-level code should be at the highest level of abstraction, and will be built with types and functions provided by lower-level code, which will eventually depend on the core Kotlin and Java runtimes, until at the bottom we reach code provided by the operating system and chipset.

Unless we are specifically writing a library to be depended on by other systems, we are usually writing the top-level code in a system, and some of the layers beneath it. We are only able to refactor within the layers that we are able to change. If we say that we are moving some aspect of code (functionality or mutability or IO or error handling) up, it means towards the top of the call-stack and hence the entry point - in the direction of the code with the most dependencies.

The reality of software is that this metaphor will sometimes fail us. In these cases we hope to make ourselves clear in context.