3
Defining modules and their properties

This chapter covers

• What modules are, and how module declarations define them
• Discerning different types of modules
• Module readability and accessibility
• Understanding the module path
• Building module graphs with module resolution

We’ve talked a lot about modules already. They’re the core building blocks not only of modular applications, but also of a true comprehension of the module system. As such, it’s important to build a deeper understanding of what they are and how their properties shape a program’s behavior.

Of the three essential steps of defining, building, and running modules, this chapter explores the first (for the other two, see chapters 4 and 5). This chapter explains in detail what a module is and how a module’s declaration defines its name, dependencies, and API (section 3.1). Some examples from the JDK give you a first look at the module landscape we’re going to explore from Java 9 on and categorize the kinds of modules to help you navigate.

We also discuss how the module system—and, by extension, the compiler and runtime—interact with modules (sections 3.2 and 3.3). Last but not least, we examine the module path and how the module system resolves dependencies and builds a graph from them (section 3.4).

If you want to code along, check out ServiceMonitor’s master branch. It contains most of the module declarations shown in this chapter. By the end of the chapter, you’ll know how to define a module’s name, dependencies, and API and how the module system behaves based on that information. You’ll understand the error messages the module system may throw at you and be able to analyze and fix their root cause.

3.1 Modules: The building blocks of modular applications

After all that lofty talk about modules, it’s time to get your hands dirty. We’ll first look at the two file formats in which you may encounter modules (JMODs and modular JARs) before turning to how you declare a module’s properties. Laying some groundwork for easier discussions in the rest of this book, we’ll categorize the different kinds of modules.

3.1.1 Java modules (JMODs), shipped with the JDK

During the work on Project Jigsaw, the Java code base was split into about 100 modules, which are delivered in a new format called JMOD. It’s deliberately unspecified to allow a more aggressive evolution than was possible with the JAR format (which is essentially a ZIP file). It’s reserved for use by the JDK, which is why we won’t discuss it in depth.

Although we aren’t supposed to create JMODs, we can examine them. To see the modules contained in a JRE or JDK, call java --list-modules. The information comes from an optimized module storage, the modules file in the runtime install’s libs folder. JDKs (not JREs) also contain the raw modules in a jmods folder; and the new jmod tool, which you can find in the bin folder next to jmods, can be used to output their properties with the describe operation.

The following snippet shows an example of examining a JMOD file. Here, jmod is used to describe java.sql on a Linux machine, where JDK 9 is installed in /opt/jdk-9. Like most Java modules, java.sql uses several of the module system’s advanced features, so not all details will be clear by the end of the chapter:

$ jmod describe /opt/jdk-9/jmods/java.sql.jmod

> java.sql@9.0.4    ①  

> exports java.sql    ②  
> exports javax.sql
> exports javax.transaction.xa

> requires java.base mandated    ③  
> requires java.logging transitive    ④  
> requires java.xml transitive

> uses java.sql.Driver    ⑤  

> platform linux-amd64    ⑥  

3.1.2 Modular JARs: Home-grown modules

We aren’t supposed to create JMODs, so how do we deliver the modules we create? This is where modular JARs come in.

The module descriptor holds all the information needed by the module system to create a run-time representation of the module. All properties of an individual module are represented in this file; consequently, many of the features discussed throughout this book have their counterpart in it, too. Creating such a descriptor from a source file, as covered in the next section, and including it in a JAR allow developers and tools to create modules.

Although the module descriptor allows a modular JAR to be more than a mere class file archive, using it that way isn’t mandatory. Clients can choose to use it as a simple JAR, ignoring all module-related properties, by placing it on the class path. This is indispensable for incremental modularizations of existing projects. (Section 8.2 introduces the unnamed module.)

3.1.3 Module declarations: Defining a module’s properties

So a module descriptor, module-info.class, is all you need to turn any old JAR into a module. That begs the question, though, of how you create a descriptor. As the file extension .class suggests, it’s the result of compiling a source file.

The module declaration determines a module’s identity and behavior in the module system. Many of the features the following chapters introduce have corresponding directives in the module declaration, presented in due time. For now, let’s look at the three basic properties lacking in JARs: a name, explicit dependencies, and encapsulated internals.

module ${module-name} {
 requires ${module-name};
 exports ${package-name};
}

Of course, ${module-name} and ${package-name} need to be replaced with actual module and package names.

Take the descriptor of ServiceMonitor’s monitor.statistics module as an example:

module monitor.statistics {
 requires monitor.observer;
 exports monitor.statistics;
}

You can easily recognize the structure I just described: the module keyword is followed by the module’s name, and the body contains requires and exports directives. The following sections look at the details of declaring these three properties.

Naming modules

The most basic property that JARs are missing is a name that the compiler and JVM can use to identify them. So, this is the most prominent characteristic of a module. You’ll have the opportunity and even the obligation to give every module you create a name.

Naming a module will be fairly natural, because most tools you use on a daily basis already have you name your projects. But although it makes sense to take the project name as a springboard on the search for a module name, it’s important to choose wisely!

As you’ll see in section 3.2, the module system leans heavily on a module’s name. Conflicting or evolving names in particular cause trouble, so it’s important that the name is

• Globally unique
• Stable

The best way to achieve this is to use the reverse-domain naming scheme that’s commonly used for packages. Together with the limitations for identifiers, this often leads to a module’s name being a prefix of the packages it contains. That isn’t mandatory, but it’s a good sign that both were chosen deliberately.

Keeping the module name and package name prefix in sync emphasizes that a module name change (which would imply a package name change) is one of the most severe breaking changes possible. In the interest of stability, it should be an exceedingly rare event.

For example, the following descriptor names the module monitor.statistics (to keep names succinct, the modules making up the ServiceMonitor application don’t follow the reverse-domain naming scheme):

module monitor.statistics {
 // requires and exports truncated
}

All other properties are defined within the curly braces following the module’s name. No particular order is enforced, but it’s common to start off with dependencies before coming to exports.

Requiring modules to express dependencies

Another thing we miss in JARs is the ability to declare dependencies. With JARs, we never know what other artifacts they need to run properly, and we depend on build tools or documentation to determine that. With the module system, dependencies have to be made explicit. (See figure 3.1 for how this plays out.)

The monitor.statistics module has a compile-time and run-time dependency on monitor.observer, which is declared with a requires directive:

module monitor.statistics {
 requires monitor.observer;
 // exports truncated
}

If a dependency is declared with a requires directive, the module system will throw an error if it can’t find a module with that exact name. Compiling as well as launching an application will fail if modules are missing (see section 3.2).

Exporting packages to define a module’s API

Last up are exports, which define a module’s public API. Here you can pick and choose which packages contain types that should be available outside the module and which packages are only meant for internal use.

The module monitor.statistics exports a package of the same name:

module monitor.statistics {
 requires monitor.observer;
 exports monitor.statistics;
}

Note that even though we like to think they are, packages aren’t hierarchical! The package java.util doesn’t contain java.util.concurrent; accordingly, exporting the former doesn’t expose any types in the latter. This is in line with imports, where import java.util.* will import types all from java.util but none from java.util.concurrent (see figure 3.2).

This implies that if a module wants to export two packages, it always needs two exports directives. The module system also offers no wildcards like exports java.util.* to make that easier—exposing an API should be a deliberate act.

Example module declarations

To get your feet wet, let’s look at some real-life module declarations. The most fundamental module is java.base, because it contains java.lang.Object, a class without which no Java program could function. It’s the dependency to end all dependencies: all other modules require it, but it requires nothing else. The dependency on java.base is so fundamental that modules don’t even have to declare it as the module system fills it in automatically (the following section goes into more detail). Although it depends on nothing, it exports a whopping 116 packages, so I’ll only show a heavily truncated version of it:

module java.base {
 exports java.lang;
 exports java.math;
 exports java.nio;
 exports java.util;
 // many, many more exports
 // use of fancy features is truncated
}

A much simpler module is java.logging, which exposes the java.util.logging package:

module java.logging {
 exports java.util.logging;
}

To see a module that requires another, let’s turn to java.rmi. It creates log messages and accordingly depends on java.logging for that. The API it exposes can be found in java.rmi and other packages with that prefix:

module java.rmi {
 requires java.logging;
 exports java.rmi;
 // exports of other `java.rmi.*` packages
 // use of fancy features is truncated
}

For more examples, flip back to section 2.2.3, particularly the code that declares the modules of the ServiceMonitor applications.

3.1.4 The many types of modules

Think of an application you’re working on at the moment. There’s a good chance it consists of a number of JARs, which, at some point in the future, will likely all be modules. They aren’t the only ones making up the application, though. The JDK was also split into modules, and they will become part of your consideration, as well. But wait, there’s more! In this set of modules, some have characteristics that make it necessary to call them out specifically.

• Application modules —A non-JDK module; the modules Java developers create for their own projects, be they libraries, frameworks, or applications. These are found on the module path. For the time being, they will be modular JARs (see section 3.1.2).
• Initial module —Application module where compilation starts (for javac) or containing the main method (for java). Section 5.1.1 shows how to specify it when launching the application with the java command. The compiler also has a use for the concept: as explained in section 4.3.5, it defines which module the compilation starts with.
• Root modules —Where the JPMS starts resolving dependencies (a process explained in detail in section 3.4.1). In addition to containing the main class or the code to compile, the initial module is also a root module. In tricky situations you’ll encounter further into the book, it can become necessary to define root modules beyond the initial one (as explained in section 3.4.3).
• Platform modules —Modules that make up the JDK. These are defined by the Java SE Platform Specification (prefixed with java.) as well as JDK-specific modules (prefixed with jdk.). As discussed in section 3.1.1, they’re stored in optimized form in a modules file in the runtime’s libs directory.
• Incubator modules —Nonstandard platform modules whose names always start with jdk.incubator. They contain experimental APIs that could benefit from being tested by adventurous developers before being set in stone.
• System modules —In addition to creating a run-time image from a subset of platform modules, jlink can also include application modules. The platform and application modules found in such an image are collectively called its system modules. To list them, use the java command in the image’s bin directory and call java --list-modules.
• Observable modules —All platform modules in the current runtime as well as all application modules specified on the command line; modules that the JPMS can use to fulfill dependencies. Taken together, these modules make up the universe of observable modules.
• Base module —The distinction between application and platform modules exists only to make communication easier. To the module system, all modules are the same, except one: the platform module java.base, the so-called base module, plays a particular role.

Platform modules and most application modules have module descriptors that are given to them by the module’s creator. Do other modules exist? Yes:

• Explicit modules —Platform modules and most application modules that have module descriptors given to them by the module’s creator.
• Automatic modules —Named modules without a module description (spoiler: plain JARs on the module path). These are application modules created by the runtime, not a developer.
• Named modules —The set of explicit modules and automatic modules. These modules have a name, either defined by a descriptor or inferred by the JPMS.
• Unnamed modules —Modules that aren’t named (spoiler: class path content) and hence aren’t explicit.

Both automatic and unnamed modules become relevant in the context of migrating an application to the module system—a topic discussed in depth in chapter 8. To get a better sense of how these types of modules relate to one another, see figure 3.3.

To apply these terms to an example, let’s turn to the ServiceMonitor application we explored in chapter 2. It consists of seven modules (monitor, monitor.observer, monitor.rest, and so forth) plus the external dependencies Spark and Hibernate and their transitive dependencies.

When it’s launched, the folders containing its seven modules and its dependencies are specified on the command line. Together with the platform modules in the JRE or JDK that’s running the application, they form the universe of observable modules. This is the pool of modules from which the module system will try to fulfill all dependencies.

ServiceMonitor’s modules as well as those making up its dependencies, Hibernate and Spark, are the application modules. Because it contains the main method, monitor is the initial module—no other root modules are required. The only platform module the program depends on directly is the base module java.base, but Hibernate and Spark pull in further modules like java.sql and java.xml. Because this is a brand-new application and all dependencies are assumed to be modularized, this isn’t a migration scenario; hence, no automatic or unnamed modules are involved.

Now that you know what types of modules exist and how to declare them, it’s time to explore how Java processes this information.

3.2 Readability: Connecting the pieces

Modules are the atomic building blocks: the nodes in a graph of interacting artifacts. But there can be no graph without edges connecting the nodes! This is where readability comes in, based on which the module system will create connections between nodes.

Whereas phrases like “customer requires bar” and “customer depends on bar” mirror a static, compile-time relationship between customer and bar, readability is its more dynamic, run-time counterpart. Why is it more dynamic? The requires directive is the primal originator of reads edges, but it’s by no means the only one. Others are command-line options (see --add-reads in section 3.4.4) and the reflection API (see section 12.3.4), both of which can be used to add more; in the end, it’s irrelevant. Regardless of how reads edges come to be, their effects are always the same: they’re the basis for reliable configuration and accessibility (see section 3.3).

3.2.1 Achieving reliable configuration

As described in section 1.5.1, reliable configuration aims to ensure that the particular configuration of artifacts a Java program is compiled against or launched with can sustain the program without spurious run-time errors. To this end, it performs a couple of checks (during module resolution, a process explained in section 3.4.1).

This verification of course isn’t airtight, and it’s possible for problems to hide long enough to crash a running application. If, for example, the wrong version of a module ends up in the right place, the application will launch (all required modules are present) but will crash later, when, for example, a class or method is missing.

Because the module system is developed to exhibit consistent behavior across compile time and run time, these errors can be further minimized by basing compilation and launch on the same artifacts. (In the example, the compilation against the module with the wrong version would have failed.)

3.2.2 Experimenting with unreliable configurations

Let’s try to break things! What are some unreliable configurations the module system detects? To investigate, we’ll turn to the ServiceMonitor application introduced in chapter 2.

Missing dependencies

Consider monitor.observer.alpha and its declaration:

module monitor.observer.alpha {
 requires monitor.observer;
 exports monitor.observer.alpha;
}

This is what it looks like to try to compile it with monitor.observer missing:

> monitor.observer.alpha/src/main/java/module-info.java:2:
>     error: module not found: monitor.observer
>         requires monitor.observer
>                         ^
> 1 error

If the module is present at compile time but gets lost on the way to the launch pad, the JVM will quit with the following error:

> Error occurred during initialization of boot layer
> java.lang.module.FindException:
>     Module monitor.observer not found,
>     required by monitor.observer.alpha

Although it makes sense to enforce the presence of all transitively required modules at launch time, the same can’t be said for the compiler. Accordingly, if an indirect dependency is missing, the compiler emits neither a warning nor an error, as you can see in the following example.

These are the module declarations of monitor.persistence and monitor.statistics:

module monitor.persistence {
 requires monitor.statistics;
 exports monitor.persistence;
}

module monitor.statistics {
 requires monitor.observer;
 exports monitor.statistics;
}

It’s clear that monitor.persistence doesn’t require monitor.observer directly, so compilation of monitor.persistence succeeds even if monitor.observer isn’t on the module path.

Launching an application with a missing transitive dependency won’t work. Even if the initial module doesn’t directly depend on it, some other module does, so it will be reported as missing. The branch break-missing-transitive-dependency in the ServiceMonitor repository creates a configuration where a missing module leads to an error message.

Duplicate modules

Because modules reference one another by name, any situation where two modules claim to have the same name is ambiguous. Which one is correct to pick is highly dependent on the context and not something the module system can generally decide. So instead of making a potentially bad decision, it makes none at all, and instead produces an error. Failing fast like this allows the developer to notice the problem and fix it before it causes any more issues.

This is the compile error the module system produces when trying to compile a module with two variants of monitor.observer.beta on the module path:

> error: duplicate module on application module path
>     module in monitor.observer.beta
> 1 error

Note that the compiler can’t link the error to one of the files under compilation because they aren’t the reason for the problem. Instead, the artifacts on the module path are causing the error.

When the error goes undetected until the JVM is launched, it gives a more precise message that lists the JAR filenames as well:

> Error occurred during initialization of boot layer
> java.lang.module.FindException:
>     Two versions of module monitor.observer.beta found in mods
>     (monitor.observer.beta.jar and monitor.observer.gamma.jar)

As we discussed in section 1.5.6 and further explore in section 13.1, the module system has no concept of versions, so in this case the same error will occur. I’d say it’s a good guess that the vast majority of duplicate-module errors will be caused by having the same module in several versions on the module path.

The module system also throws the duplicate module error if the module isn’t actually required. It suffices that the module path contains it! Two of the reasons for that are services and optional dependencies, which are presented in chapter 10 and section 11.2. The ServiceMonitor branch break-duplicate-modules-even-if-unrequired creates an error message due to a duplicate module even though it isn’t required.

Dependency cycles

Accidentally creating cyclic dependencies isn’t hard, but getting them past the compiler is. It isn’t even straightforward to present them to the compiler. In order to do that, you’d have to solve the chicken-and-egg problem that if two projects depend on each other, it isn’t possible to compile one without the other. If you tried, you’d have missing dependencies and get the corresponding errors.

One way to get past this is to compile both modules at once, starting with both the chicken and the egg at the same time, so to speak; section 4.3 explains how. Suffice it to say, if there’s a cyclic dependency between the modules being compiled, the module system recognizes that and causes a compile error. This is how it looks if monitor.persistence and monitor.statistics depend on each other:

> monitor.statistics/src/main/java/module-info.java:3:
>     error: cyclic dependence involving monitor.persistence
>         requires monitor.persistence;
>                         ^
> 1 error

Another way to go about this is to establish the cyclic dependency not at once but over time, after a valid configuration is already built. Let’s once more turn to monitor.persistence and monitor.statistics:

module monitor.persistence {
 requires monitor.statistics;
 exports monitor.persistence;
}

module monitor.statistics {
 requires monitor.observer;
 exports monitor.statistics;
}

This configuration is fine and compiles without problems. Now the trickery begins: compile the modules and keep the JARs around. Then change the module declaration of monitor.statistics to require monitor.persistence, which creates a cyclic dependency (the change doesn’t make much sense in this example, but in more-complex applications it often does):

module monitor.statistics {
 requires monitor.observer;
 requires monitor.persistence;
 exports monitor.statistics;
}

The next step is to compile just the changed monitor.statistics with the already-compiled modules on the module path. This must include monitor.persistence, because the statistics module now depends on it. In turn, the persistence module still declares its dependency on monitor.statistics, which is the second half of the dependency cycle. Unfortunately, for this round of hacking, the module system recognizes the cycle and causes the same compile error as before.

Taking the shell game to the next level finally tricks the compiler. In this scenario, two completely unrelated modules—let’s pick monitor.persistence and monitor.rest —are compiled into modular JARs. Then comes the sleight of hand:

One dependency is added, say from persistence to rest, and the changed persistence is compiled against the original set of modules. This works because the original rest doesn’t depend on persistence.

The second dependency, rest to persistence, is added, but rest is also compiled against the original set of modules, including the version of persistence that doesn’t yet depend on it. As a consequence, it can be compiled as well.

Confused? Look at figure 3.5 to get another perspective.

Now there are versions of monitor.persistence and monitor.rest that depend on each other. For this to happen in real life, the compilation process—maybe managed by a build tool—must be in serious disarray (but that isn’t unheard of). Luckily, the module system has your back and reports the error when the JVM is launched with such a configuration:

> Error occurred during initialization of boot layer
> java.lang.module.FindException:
>     Cycle detected:
>         monitor.persistence
>           -> monitor.rest
>           -> monitor.persistence

All the examples show a cyclic dependency between two artifacts, but the module system detects cycles of all lengths. It’s good that it does! Changing code always risks breaking upstream functionality, meaning other code that uses the code that’s being changed—either directly or transitively.

If dependencies go in one direction, there’s only so much code a change can impact. On the other hand, if dependencies can form cycles, then all code in that cycle and all that depends on it can be affected. Particularly if cycles are large, this can quickly turn into all the code being affected, and I’m sure you agree you want to avoid that. And the module system isn’t alone in helping you here—so is your build tool, which also bristles at dependency cycles.

Split packages

A split package occurs when two modules contain types in the same package. For example, recall that the monitor.statistics module contains a class Statistician in the monitor.statistics package. Now let’s assume the monitor module contained a simple fallback implementation, SimpleStatistician, and to promote uniformity, it’s in monitor’s own monitor.statistics package.

When trying to compile monitor, you get the following error:

> monitor/src/main/java/monitor/statistics/SimpleStatistician.java:1:
>     error: package exists in another module: monitor.statistics
>         package monitor.statistics;
>         ^
> 1 error

To try this, let’s go a different route: SimpleStatistician is gone, and this time it’s monitor.statistics that creates the split package. In an attempt to reuse some utility methods, it creates a Utils class in the monitor package. It has no desire to share that class with other modules, so it continues to only export the monitor.statistics package.

Compiling monitor.statistics is error-free, which makes sense because it doesn’t require monitor and is hence unaware of the split package. It gets interesting when the time comes to compile monitor. It depends on monitor.statistics, and both contain types in the package monitor. But, as I just mentioned, because monitor.statistics doesn’t export the package, compilation works.

Great! Now it’s time to launch:

> Error occurred during initialization of boot layer
> java.lang.reflect.LayerInstantiationException:
>     Package monitor in both module monitor.statistics and module monitor

That didn’t go well. The module system checks for split packages on launch, and here it doesn’t matter whether they’re exported or not: no two modules can contain types in the same package. As you’ll see in section 7.2, this can turn into a problem when migrating code to Java 9.

The ServiceMonitor repository demonstrates the split-package problem at compile and at run time in the branches break-split-package-compilation and break-split-package-launch.

The modular diamond of death

A particularly devious mixture of split packages and missing dependencies is the modular diamond of death (see figure 3.6). Assume a module changed its name between two releases: one of your dependencies requires it by its old name, and another dependency requires it by its new name. Now you need the same code to appear under two different module names, but the JPMS isn’t going to let that happen.

You’ll have one of the following situations:

• One modular JAR, which can only appear as one module with one name and will thus trigger an error because one dependency couldn’t be fulfilled
• Two modular JARs with different names but the same packages, which will cause the split-package error you just observed

3.3 Accessibility: Defining public APIs

With modules and the read edges in place, you know how the module system constructs the graph you have in mind. To keep that graph from behaving like the ball of mud you wanted to escape, there’s one more requirement: the ability to hide a module’s internals so no outside code can access it. This is where accessibility comes in.

For an accessible type’s members (meaning its fields, methods, and nested classes), the usual visibility rules hold: public members are fully accessible, and protected members only to inheriting classes. Technically, package-private members are accessible in the same package, but as you saw in the previous section, that isn’t helpful due to the rule against split packages across modules.

To understand how accessibility shapes a module’s public API, it’s important to understand that term first: what is the public API?

In case you find it weird that I’m suddenly talking about names, think about what you can change in a type while keeping dependent code from outside the package compiling. Private and package-visible fields? Definitely! Private and package-visible methods? Sure. Bodies of public methods? Yes. What needs to stay untouched are the names of everything that other code may be compiled against: the type’s name, the signature of public methods, and so forth.

Looking over the definition of what makes a public API, it becomes clear that the module system changes things from before Java 9 on the level of packages (must be exported) and types (must be public). Within a type, on the other hand, nothing changed, and a type’s public API is the same in Java 8 as in Java 9 and later.

3.3.1 Achieving strong encapsulation

As an example, consider a high-performance library superfast with custom implementations of the known Java collections. Let’s focus on a hypothetical SuperfastHashMap class, which implements Java’s Map interface and is not accessible (maybe it is package visible in an exported package, maybe the package is not exported at all).

If code outside the superfast module gets a SuperfastHashMap instance (maybe from a factory), then it’s limited to using it as a Map. It can’t assign it to a variable of type SuperfastHashMap and can’t call superfastGet on it (even if that method is public) but everything that’s defined on accessible supertypes like Map and Object is no problem. (See figure 3.8.)

The accessibility rules make it possible to expose carefully selected features while strongly encapsulating a module’s internals, making sure no outside code can depend on implementation details. Interestingly, this includes reflection, which can’t bypass the rules either if used across module boundaries! (We’re going to talk about reflection throughout the rest of the chapter—if you need to catch up on the basics, see appendix B.)

Maybe you’re wondering how reflection-based libraries like Spring, Guice, Hibernate, and others will work in the future, or how code will be able to break into a module if it absolutely has to. There are a few ways to give or gain access:

• Regular exports (see section 3.1)
• Qualified export (see section 11.3)
• Open modules and open packages (see section 12.2)
• Command-line options (summarized in section 7.1)

Chapter 12 explores reflection in more depth.

But let’s go back to the three conditions that are the premise for accessibility (public type, exported package, reading module). They have interesting consequences.

The three conditions also imply that you can also no longer accidentally rely on transitive dependencies. Let’s see why.

3.3.2 Encapsulating transitive dependencies

Without the module system, it’s possible to use types from a JAR that a dependency draws in but that isn’t declared as a dependency. Once a project uses types this way, the build configuration no longer reflects the true set of dependencies, which can lead to anything from uninformed architectural decisions to run-time errors.

As an example, let’s say a project is using Spring, which depends on OkHttp. Writing code that uses types from OkHttp is as easy as letting the IDE add the import statements it helpfully suggests. The code will compile and run because the build tool will make sure Spring and all its dependencies, including OkHttp, are present at all times. This makes it unnecessary to declare the dependency on OkHttp, so it’s easily forgotten. (See figure 3.9.)

As a consequence, a dependency analysis of the project would deliver misleading results, based on which problematic decisions could be made. The OkHttp version also isn’t fixed and depends entirely on what Spring uses. If that version is updated, the code depending on OkHttp is silently running on a different version, creating the real risk that the program will misbehave or crash at run time.

Due to the module system’s requirement that the accessing module must read the accessed module, this can no longer happen. Unless the project declares its dependency on OkHttp by using a requires directive, the module system won’t allow it to access OkHttp’s classes. This way it forces you to keep your configuration up to date.

Note that modules have the ability to pass their own dependencies on to modules that depend on them with a feature called implied readability. Check section 11.11 for details.

3.3.3 Encapsulation skirmishes

As we did with readability, let’s break things! But before we do so, I want to show a scenario that follows all the rules and works. Once again, it’s based on the ServiceMonitor application introduced in chapter 2.

For the sake of these examples, assume that the module monitor.observer contained in its package monitor.observer a class DisconnectedServiceObserver. What it does is irrelevant: what counts is that it implements ServiceObserver, that it has a constructor that doesn’t require any arguments, and that the monitor module uses it.

The module monitor.observer exports monitor.observer and DisconnectedServiceObserver is public. This fulfills the first two accessibility requirements, so other modules can access it if they read monitor.observer. The module monitor fulfills that precondition, too, because it requires module.observer in its module declaration. Taken together (figure 3.10 and listing 3.1), all requirements are fulfilled, and code in monitor can access DisconnectedServiceObserver. Accordingly, compilation and execution are error-free. Let’s fiddle with the details and watch how the module system reacts.

Listing 3.1 DisconnectedServiceObserver, accessible by monitor

// --- TYPE DisconnectedServiceObserver ---
package monitor.observer;

public class DisconnectedServiceObserver  //    ①  
  implements ServiceObserver {
 // class body truncated
}

// --- MODULE DECLARATION monitor.observer ---
module monitor.observer {
 exports monitor.observer;  //  ②  
}

// --- MODULE DECLARATION monitor ---
module monitor {
 requires monitor.observer;  //  ③  
 // other requires directives truncated
}

Type not public

If DisconnectedServiceObserver is made package-visible, compilation of monitor fails. More precisely, the import causes the first error:

> monitor/src/main/java/monitor/Monitor.java:4: error:
>     DisconnectedServiceObserver is not public in monitor.observer;
>     cannot be accessed from outside package
> import monitor.observer.DisconnectedServiceObserver;
>                        ^

Accessing package-visible types from another package wasn’t possible before Java 9, either, and for that reason the error message is nothing new—you’d get the same one without the module system in play.

Similarly, if you bypass the compiler checks by recompiling just the monitor.observer module after DisconnectedServiceObserver is made package-visible and then launching the entire application, the error is the same as without the module system:

> Exception in thread "main" java.lang.IllegalAccessError:
>     failed to access class monitor.observer.DisconnectedServiceObserver
>     from class monitor.Monitor

Before Java 9, it was possible to use the reflection API to access the type at run time, and this is something strong encapsulation prevents. Consider the following code:

Constructor<?> constructor = Class
 .forName("monitor.observer.DisconnectedServiceObserver")
 .getDeclaredConstructor();
constructor.setAccessible(true);
ServiceObserver observer = (ServiceObserver) constructor.newInstance();

In Java 8 and before, this works regardless of whether DisconnectedServiceObserver is public or package-visible. In Java 9 and later, the module system prevents access if DisconnectedServiceObserver is package-visible, and the call to setAccessible causes an exception:

> Exception in thread "main" java.lang.reflect.InaccessibleObjectException:
>     Unable to make monitor.observer.DisconnectedServiceObserver()
>     accessible: module monitor.observer does not "opens monitor.observer"
>     to module monitor

The ServiceMonitor repository’s branch break-reflection-over-internals demonstrates the behavior shown here. The complaint that monitor.observer doesn’t open monitor.observer points toward one solution for this problem—something section 12.2 explores.

Package not exported

Next on the list of requirements is that the package containing the accessed type must be exported. To toy with that, let’s make DisconnectedServiceObserver public again but move it into another package monitor.observer.dis, which monitor.observer doesn’t export. The imports in monitor are updated to the new package:

> monitor/src/main/java/monitor/Monitor.java:4: error:
>     package monitor.observer.dis does not exist
> import monitor.observer.dis.DisconnectedServiceObserver;
>                            ^
> (package monitor.observer.dis is declared in module
>  monitor.observer, which does not export it)

That’s pretty straightforward.

To see how the runtime fares in this case, you need to bypass the compiler checks. To that end, edit monitor.observer to export monitor.observer.dis, compile all modules, and then compile monitor.observer once again without that export. You can launch the application as before and provoke a run-time error:

> Exception in thread "main" java.lang.IllegalAccessError:
>     class monitor.Monitor (in module monitor) cannot access class
>     monitor.observer.dis.DisconnectedServiceObserver (in module
>     monitor.observer) because module monitor.observer does not export
>     monitor.observer.dis to module monitor

Like the compiler, the runtime is pretty talkative and explains what the problem is. The same is true for the reflection API when you try to make the constructor accessible, so you can create an instance of DisconnectedServiceObserver:

> Exception in thread "main" java.lang.reflect.InaccessibleObjectException:
>     Unable to make public
>     monitor.observer.dis.DisconnectedServiceObserver() accessible:
>     module monitor.observer does not "exports monitor.observer.dis"
>     to module monitor

If you look closely, you’ll see that both the runtime and the reflection API talk about exporting a package to a module. That’s called a qualified export (explained in section 11.3).

Module not read

The last requirement on the list is that the exporting module must be read by the one accessing the type. Removing the requires monitor.observer directive from monitor’s module declaration leads to the expected compile-time error:

> monitor/src/main/java/monitor/Monitor.java:3: error:
>     package monitor.observer is not visible
> import monitor.observer.DiagnosticDataPoint;
>               ^
> (package monitor.observer is declared in module
>  monitor.observer, but module monitor does not read it)

To see how the runtime reacts to a missing requires directive, first compile the entire application with a working configuration, meaning monitor reads monitor.observer. Then remove the requires directive from monitor’s module-info.java, and recompile just that file. This way, the module’s code is compiled with a module declaration that still requires monitor.observer, but the runtime will see a module description that claims nothing of the kind. As expected, the result is a run-time error:

> Exception in thread "main" java.lang.IllegalAccessError:
>     class monitor.Monitor (in module monitor) cannot access class
>     monitor.observer.DisconnectedServiceObserver (in module
>     monitor.observer) because module monitor does not read module
>     monitor.observer

Again, the error message is pretty clear.

Finally, let’s turn to reflection. You can use the same compilation trick to create a monitor module that doesn’t read monitor.observer. And reuse the reflection code from earlier when DisconnectedServiceObserver wasn’t public but you wanted to create an instance anyway.

Surely running these modules together fails as well, right? Yes, it does, but not the way you may have expected:

> Exception in thread "main" java.lang.IllegalAccessError:
>     class monitor.Monitor (in module monitor) cannot access class
>     monitor.observer.ServiceObserver (in module monitor.observer)
>     because module monitor does not read module monitor.observer

Why is the error message complaining about ServiceObserver? Because that type is also in monitor.observer, which monitor no longer reads. Let’s change the reflection code to only use Object:

Constructor<?> constructor = Class
 .forName("monitor.observer.DisconnectedServiceObserver")
 .getDeclaredConstructor();
constructor.setAccessible(true);
Object observer = constructor.newInstance();

Run this—it works! But what about the missing read edge, you may ask? The answer is simple but a little surprising at first: the reflection API fills it in automatically. Section 12.3.1 explores the reasons behind that.

3.4 The module path: Letting Java know about modules

You now know how to define modules and their essential properties. What’s still a little unclear is how you tell the compiler and runtime about them. Chapter 4 looks into building modules from source to JAR, and you’ll quickly run into the situation where you need to reference existing modules the code under compilation depends on. The situation is the same in chapter 5, where the runtime needs to know about the application modules, so you can launch one of them.

Before Java 9, you would have used the class path, which had plain JARs on it (see appendix A for a quick class-path recap), to inform compiler and runtime where to find artifacts. They search it when they’re looking for individual types required during compilation or execution.

The module system, on the other hand, promises not to operate on types, but to go one level above them and manage modules instead. One way this approach is expressed is a new concept that parallels the class path but expects modules instead of bare types or plain JARs.

Listing 3.2 shows how the ServiceMonitor application’s monitor module could be compiled, packaged, and launched. It uses --module-path to point to the directory mods, which you assume contains all required dependencies as modular JARs. For details on compilation, packaging, and launching, see sections 4.2, 4.5, and 5.1.

Listing 3.2 Compiling, packaging, and launching monitor

$ javac
 --module-path mods   ①  
 -d monitor/target/classes
 ${source-files}    ②  
$ jar --create
 --file mods/monitor.jar    ③  
 --main-class monitor.Main
 -C monitor/target/classes .
$ java
 --module-path mods:libs    ④  
 --module monitor

The phrase “modules specified on the command line” is a little vague. Now you know that they’re artifacts that can be found on the module path.

Note that I said artifacts, not modules! Not only modular JARs but also plain ones will, when placed on the module path, be turned into modules and become part of the universe of observable modules. This somewhat surprising behavior is part of the migration story, and discussing it here would derail our exploration of the module path, so let me defer explaining it to section 8.3. What I want to mention now is that symmetrical to the module path interpreting every artifact as a module, the class path treats all artifacts as plain JARs, regardless of whether they contain a module descriptor.

Because the module path is used by several tools, most notably the compiler and the virtual machine, it makes sense to look at the concept in general. Unless otherwise noted, the described mechanisms work the same in all environments.

3.4.1 Module resolution: Analyzing and verifying an application’s structure

What happens after calling javac or java with a bunch of modules on the module path? This is when the module system starts checking the launch configuration, meaning the modules and their declared dependencies, for reliability.

The process has to start somewhere, so the module system’s first order of business is to decide on the set of root modules. There are several ways to make a module a root, and we’ll discuss them all in due time, but the most prominent is specifying the initial module. For the compiler, that’s either the module under compilation (if a module declaration is among the source files) or the one specified with --module (if the module source path is used). In the case of launching the virtual machine, only the --module option remains.

Next, the module system resolves dependencies. It checks the root modules’ declarations to see which other modules they depend on and tries to satisfy each dependency with an observable module. It then goes on to do the same with those modules and so forth. This continues until either all transitive dependencies of the initial module are fulfilled or the configuration is identified as unreliable.

The easiest way to demonstrate that modules can be duplicated across folders is to pick a project that’s ready to be launched and has all its modules in a folder (say, mods). Then create a copy of the entire folder (say, mods-copy) and place both on the module path:

$ java
 --module-path mods:mods-copy:libs
 --module monitor

All modules appear once in each folder, but the application starts nonetheless.

Now consider that build tools usually create a module path that lists each dependency individually. That means that as long as the build tool is in control, for example during compilation and testing, ambiguity checks aren’t applied across all dependencies.

I find this unfortunate, because it voids a part of the promise of reliable configuration. On the other hand, it does have the upside that you can purposely shadow modules with versions you like better as long as you put yours first. Just remember that unlike in class-path times, different JARs are never “mixed.” If the module system picks one module as a package’s origin, it will look up all classes from that package in that JAR and never look in other JARs (this is related to split packages, discussed in sections 3.2.2 and 7.2).

Next, let’s assume all modules were resolved. If no errors were found, the module system guarantees that each required module is present. Or rather, that modules with the right names are present.

There are no additional checks during this phase, so if a module depends on, for example, com.google.common (the module name for Google’s Guava library) and an empty module with that name was found, the module system is content. But the missing types will still cause trouble down the road, in the form of compile-time or run-time errors. While empty modules are unlikely, a module with a different version than expected, missing a couple of types, isn’t implausible. Still, a reliable configuration will greatly reduce the number of NoClassDefFoundErrors that crop up during execution.

3.4.2 Module graph: Representation of an application’s structure

One of this book’s first headings is “Visualizing software as graphs” (section 1.1.1). The ensuing paragraphs explain how developers and tools tend to see code in general but particularly dependencies between artifacts as graphs. The rest of chapter 1 illustrates that Java instead sees them as mere containers for types it consequently rolls into a ball of mud and how that mismatch is the root of a few hard problems plaguing the ecosystem.

The module system promises to solve many of those issues by aligning Java’s perception with yours. All of this builds up to one revelation: the module system also sees a graph of artifacts. So here it is: the module graph!

Figure 3.11 shows how module resolution creates the module graph for a simplified ServiceMonitor application. You don’t have to leave everything up to the JPMS, though. With the right command-line options, you can add more modules and reads edges to the graph; we’ll explore that next.

3.4.3 Adding modules to the graph

It’s important to note that modules that didn’t make it into the module graph during resolution aren’t available later during compilation or execution, either. For cases where all application code is in modules, this is often irrelevant. After all, following the rules for readability and accessibility, even if such modules were available, their types would be inaccessible because nobody reads the modules. But there are scenarios using more advanced features where this may pop up as a compile-time or run-time error or even as an application that doesn’t behave the way it’s supposed to.

Various use cases can lead to the scenario of modules not making it into the graph. One of them is reflection. It can be used to have code in one module call code in another without explicitly depending on it. But without that dependency, the depended-on module may not make it into the graph.

Assume there was some alternative statistics module, monitor.statistics.fancy, that couldn’t be present on the module path for each deployment of the service. (The reason is irrelevant, but let’s go with a license that prevents the fancy code from being used “for evil.” Evil masterminds that we are, we occasionally want to do just that.) So the module may sometimes be present and sometimes not, and hence no other module can require it because then the application couldn’t launch if the module was missing.

How could the application handle that? The code depending on the fancy statistics library could use reflection to check whether the library is present and only call it if it is. But according to what you just learned, that will never be the case! By necessity, the fancy statistics module isn’t required by any other module and hence won’t end up in the module graph, meaning it can never be called. For these and other scenarios that pop up throughout the book, the module system offers a solution.

In the case of the ServiceMonitor application having an optional dependency on monitor.statistics.fancy, you have to make sure the module shows up in the module graph for those deployments that ship with it. In such cases, you’d use --add-modules monitor.statistics.fancy to make it a root module, causing the module system to add it and its dependencies to the module graph:

$ java
 --module-path mods:libs
 --add-modules monitor.statistics.fancy
 --module monitor

You can see the resulting module graph in figure 3.12.

A particularly important use case for --add-modules are JEE modules, which, as section 6.1 explains, aren’t resolved by default when running an application from the class path. Because you can add modules to the graph, it’s only natural to wonder whether you can also remove them. The answer is yes, kind of: the option --limit-modules goes in that direction, and section 5.3.4 shows how it works.

Unfortunately, it isn’t possible to let the module system know a specific dependency won’t be fulfilled and you’re OK with that. That would allow you to exclude (transitive) dependencies you don’t need. Judging by the number of exclusions I see in typical Maven POMs, this is common, but, alas, the module system’s strictness doesn’t allow it.

3.4.4 Adding edges to the graph

When a module is added explicitly, it’s on its own in the module graph, without any incoming reads edges. If access to it is purely reflective, that’s fine, because the reflection API implicitly adds a reads edge. But for regular access, such as when importing a type from it, accessibility rules require readability.

Back to monitor.statistics.fancy, you can use add-reads to allow monitor.statistics to read it:

$ java
 --module-path mods:libs
 --add-modules monitor.statistics.fancy
 --add-reads monitor.statistics=monitor.statistics.fancy
 --module monitor

The resulting module graph is the same as in figure 3.12, except the dashed line is now replaced by a proper reads edge. Toward the end of section 8.3.2 is a case where --add-reads … =ALL-UNNAMED saves the day.

3.4.5 Accessibility is an ongoing effort

Once the module system has resolved all dependencies, built the module graph, and established readability between modules, it stays active by checking the accessibility rules section 3.3 defines. If these rules are broken, compile-time or run-time errors ensue, as shown in section 3.3.3. If you encounter a problem with the module system and can’t tell from the error message what went wrong, see section 5.3 for advice on how to debug the situation.

If you’re interested in learning more about building and running modular applications, such as your own green-field projects, chapters 4 and 5 go deeper into that. Alternatively, you can check out the module system’s effects on your existing project in chapters 6 and 7. You’re also well-prepared to go deeper and check out the advanced features, particularly chapters 10 and 11.

Summary

• Modules come in two forms:

• The ones shipped with the Java runtime are platform modules. They’re merged into a modules file in the runtime’s libs directory. A JDK also holds them in raw form as JMOD files in the jmods directory. Only java.base, the base module, is explicitly known to the module system.
• Library, framework, and application developers create modular JARs, which are plain JARs containing a module descriptormodule-info.class. These are called application modules, with the one containing the main method being the initial module.

• The module descriptor is compiled from a module declarationmodule-info.java that developers (and tools) can edit. It lies at the heart of the work with the module system and defines a module’s properties:

• Its name, which should be globally unique due to the reverse-domain naming scheme
• Its dependencies, which are stated with requires directives that refer to other modules by name
• Its API, which is defined by exporting selected packages with exports directives

• Dependency declarations and the readability edges the module system is creating from them are the basis for reliable configuration. It’s achieved by making sure, among other things, that all modules are present exactly once and no dependency cycles exist between them. This allows you to catch application-corrupting or crashing problems earlier.
• Readability edges and package exports together are the basis for strong encapsulation. Here the module system ensures that only public types in exported packages are accessible and only to modules that read the exporting one. This prevents accidental dependencies on transitive dependencies and enables you to make sure outside code can’t easily depend on types you designed as being internal to a module.
• Accessibility limitations apply to reflection as well! This requires a little more work to interact with reflection-based frameworks like Spring, Guice, or Hibernate.

The module path (option --module-path or -p) consists of files or directories and makes JARs available to the module system, which will represent them as modules. Use it instead of the class path to make the compiler or JVM aware of your project’s artifacts.

The application modules, specified on the module path, and the platform modules, contained in the runtime, make up the universe of observable modules. During resolution, the universe is searched for modules, starting with root modules, so all required modules must either be on the module path or in the runtime.

Module resolution verifies that the configuration is reliable (all dependencies present, no ambiguities, and so on, as introduced in section 3.2) and results in the module graph—a close representation within the module system of how you see artifact dependencies. Only modules that make it into the module graph are available at run time.