4
Building modules from source to JAR

This chapter covers

• Project directory structures
• Compiling sources from a single module to class files
• Compiling multiple modules at the same time
• Packaging class files into a modular JAR

Being able to define modules as described in chapter 3 is a good skill to have, but what is it good for without knowing how to turn those source files into modular artifacts (JARs) that can be shipped and executed? This chapter looks into building modules, all the way from organizing sources, to compiling them to class files, and eventually packaging those into modular JARs that can be distributed and executed. Chapter 5 focuses on running and debugging modular applications.

At times we’ll look at the javac and jar commands available on the command line. You may be wondering about that—aren’t IDEs and other tools going to use them for you? Likely, yes, but even putting aside the argument that it’s always good to know how those tools work their magic, there is a more important reason to get to know these commands: they’re the most direct path into the module system’s heart. We’ll use them to explore its features inside and out, and when we’re done, you can use any tool that gives access to these features.

The first thing we’ll look at in this chapter is how a project’s files should be organized on disk (section 4.1). This may seem trivial, but a new recommendation is making the rounds and it’s worth looking into. With the sources laid out and the modules declared as described in chapter 3, we’ll turn to compiling them. This can happen one module at a time (section 4.2) or for multiple modules at once (section 4.3). The final section discusses how to package class files into modular JARs. To see some real-life build scripts, take a look at ServiceMonitor’s master branch.

By the end of this chapter, you’ll be able to organize, compile, and package your source code and module declarations. The resulting modular JARs are ready to be deployed or shipped to anyone who uses Java 9 or later and is ready to take full advantage of modules.

4.1 Organizing your project in a directory structure

A real-life project consists of myriad files of many different types. Obviously, source files are the most important, but are nonetheless only one kind of many—others are test sources, resources, build scripts or project descriptions, documentation, source control information, and many others. Any project has to choose a directory structure to organize those files, and it’s important to make sure it doesn’t clash with the module system’s characteristics.

If you’ve been following the module system’s development under Project Jigsaw and studied the official quick-start guide or early tutorials, you may have noticed that they use a particular directory structure. Let’s look at the recommendation to check whether it should become a new convention and juxtapose it with the established default that’s implicitly understood by tools like Maven and Gradle.

4.1.1 New proposal—new convention?

In early publications covering the module system, the project directory often contains a src directory in which each module that belongs to the project has its own subdirectory containing the project’s source files. If the project needs more than just sources, the proposal suggests organizing these concerns in parallel trees with folders like test and build next to src. This results in a hierarchy concern/module, as shown in figure 4.1.

It’s important to recognize this single-src structure for what it is: the structure of a particular project (the JDK) and a proposal used in introductory material. Due to its tendency to split a single module’s files across parallel trees, I wouldn’t advise following it for anything but the smallest projects or ones where a meticulous examination concludes that this structure is preferable. Otherwise, I recommend using the established default, which we’ll discuss next.

4.1.2 Established directory structure

Most projects that consist of several subprojects (what we now call modules) prefer separate root directories, where each contains a single module’s sources, tests, resources, and everything else mentioned earlier. They use a hierarchy module/concern, and this is what established project structures provide.

The default directory structure, implicitly understood by tools like Maven and Gradle, implements that hierarchy (see figure 4.2). First and foremost, the default structure gives each module its own directory tree. In that tree, the src directory contains production code and resources (in main/java and main/resources, respectively) as well as test code and resources (in test/java and test/resources, respectively).

It’s no requirement to structure projects this way. Putting aside the added work of configuring build tools for deviating directories and the specific case of multimodule compilation (covered in section 4.3.), all structures are equally valid and should be chosen based on their merits for the project at hand.

All of that being said, the examples in this book use this default structure with one exception: using the command line is less cumbersome if all modular JARs end up in the same directory, so the ServiceMonitor application’s tree has a top-level mods folder containing the created modules.

4.1.3 The place for module declarations

However the source files are structured, module declarations have to be named module-info.java. Otherwise, the compiler produces an error like this one, which tries to compile monitor-observer-info.java:

> monitor.observer/src/main/java/monitor-observer-info.java:1:
>     error: module declarations should be in a file named module-info.java
> module monitor.observer {
> ^
> 1 error

Although not strictly necessary, the declaration should be located in the root source directory. Otherwise, using the module source path as described in section 4.3.2 doesn’t work properly because the module system can’t locate the descriptor. As a consequence, it doesn’t recognize the module, leading to “module not found” errors.

To try that out, move the descriptor of monitor.observer into a different directory and compile monitor. As you can see, this results in an error that the module monitor.observer, which is required by monitor, can’t be found:

> ./monitor/src/main/java/module-info.java:2:
>     error: module not found: monitor.observer
>         requires monitor.observer;
>                         ^
> 1 error

4.2 Compiling a single module

Once the project files are laid out in a directory structure, some code has been written, and the module declarations are created, it’s time to compile the source files. But what will it be—a collection of types or a shiny module? Because the former didn’t change, we’ll focus on the latter before exploring how the compiler discerns the two cases.

4.2.1 Compiling modular code

This section focuses on the compilation of a single module in a world where all dependencies are already modularized. You can only compile a module if a declaration module-info.java is among the source files, so let’s assume this is the case.

In addition to operating on the module path and checking readability and accessibility, another addition to the compiler is its ability to process module declarations. The result of compiling a module declaration is a module descriptor, a file module-info.class. Like other .class files, it contains bytecode and can be analyzed and manipulated by tools like ASM and Apache’s Byte Code Engineering Library (BCEL).

Other than using the module path instead of the class path, compilation works exactly as it did before Java 9. The compiler will compile all given files and produce a directory structure that matches the package hierarchy in the output directory specified with -d.

Figure 4.3 shows how the monitor.observer module, which uses the default directory structure, is laid out. To compile it, you create a javac call that’s similar to what you would have done before Java 9:

• The --module-path option points the compiler to the directory that contains required application modules.
• The -d options determines the target directory for the compilation; it works the same as before Java 9.
• List or find all source files in monitor.observer/src/main/java/, including module-info.java (represented by ${source-files}).

Put together, you issue the following command in the ServiceMonitor application’s root directory (i.e. the one containingmonitor.observer):

$ javac
    --module-path mods
    -d monitor.observer/target/classes
    ${source-files}

Collapsing src and looking into target/classes, figure 4.4 shows the expected result.

4.2.2 Modular or non-modular?

The Java Platform Module System is built with the intention to create and eventually run modules, but this is by no means mandatory. It’s still possible to build plain JARs, and this begs the question of how these two cases are distinguished. How does the compiler know whether to create a module or a bunch of types?

What’s the difference between compiling a module and compiling just types? It comes down do readability, as explained in section 3.2. If code that includes a module declaration is compiled

• It must require its dependencies to be able to access the types these dependencies export
• The required dependencies have to be present

If, on the other hand, non-modular code is compiled, no dependencies are expressed, due to the lack of a module declaration. In that case, the module system lets the code under compilation read all modules and everything it finds on the class path. Section 8.2 goes into detail on that class-path mode.

In contrast to readability, the accessibility rules described in section 3.3 apply to both cases. Regardless of whether the code is compiled as a module or as a bunch of sources, it’s bound to the rules when accessing types in other modules. This is particularly relevant regarding JDK-internal classes, be they public classes in non-exported packages or nonpublic classes, because they’re inaccessible regardless of how code is compiled. Figure 4.5 shows the difference between readability and accessibility.

Compiling a module obviously requires clearing more hurdles than compiling just types. So why do it? Again, I come back to the comparison to writing code in a statically typed language. As Java developers, we generally believe that static typing is worth the additional upfront costs because in exchange, we get fast and reliable consistency checks. They don’t prevent all errors, but they do prevent a lot of them.

The same applies here: using the module system to compile modules requires more effort than creating plain JARs, but in exchange we get checks that reduce the likelihood of runtime errors. We exchange compile-time effort for runtime safety—a deal I’ll make any day of the week.

4.3 Compiling multiple modules

Compiling a single module as just described is straightforward, and compiling all seven ServiceMonitor modules is more of the same. But is it necessary to compile modules one by one? Or, to look at it another way, is there any reason not to do it like that? The answer to the latter is yes, a few details may make it preferable to compile multiple modules at once:

• Effort —Although compiling a single module is simple, the effort required for multiple modules adds up quickly. And it surely feels redundant to more or less repeat the same command over and over with only slight variations. Chances are you’ll rarely do that by hand unless you’re experimenting with Java 9. But the developers working on your tools should be considered as well.
• Performance —Compiling a single module descriptor takes about half a second on my system, and compiling all modules of the ServiceMonitor application takes about four. That’s a little much, considering that there are less than 20 source files involved and full builds of much larger projects take less time. It stands to reason that I pay the price for launching the compiler seven times (for seven modules).
• Weak circular dependencies—Although the module system forbids circular dependencies with requires directives, there are other ways to have modules reference one another (trust me for now) that are deemed acceptable. Although the dependencies are circular, they can be considered weak because if the right one is missing, you only get a warning. Still, warning-free compilation is worth some effort, and to get there, both modules must be compiled together.

4.3.1 The naive approach

How does compiling multiple modules at once work? Can you list source files from several modules and have the compiler figure it out? Nope:

$ javac
    --module-path mods:libs
    -d classes
    monitor/src/main/java/module-info.java
    monitor.rest/src/main/java/module-info.java

> monitor.rest/src/main/java/module-info.java:1:
>     error: too many module declarations found
> module monitor.rest {
> ^
> 1 error

Clearly, the compiler prefers to work on a single module at a time. This makes sense, too, because as discussed previously, it enforces readability and accessibility based on clearly defined module boundaries. Where would they come from, with sources from many different modules mixed up in the list of files to compile? Somehow the compiler needs to know where one module ends and the next begins.

4.3.2 The module source path: Informing the compiler about the project structure

The way out of that default single-module mode is a command-line option that informs the compiler about the project’s directory structure. The compiler supports multimodule compilation, where it can build multiple modules at once. The command-line option --module-source-path ${path} is used to enable this mode and to point out the directory structure containing the modules. All other compiler options work as usual.

That sounds pretty easy, but there are important details to consider. Before doing that, though, let’s get a simple example to work.

Let’s assume for a moment the ServiceMonitor application used the single-src structure defined in section 4.1.1 with all module source directories below src (see figure 4.6). Then you could use --module-source-path src to point the compiler toward the src folder, which contains all the modules’ sources, and tell it to compile everything it finds at once.

As with a single-module build, the module path is used to point the compiler to the directory that contains required application modules—in this case, these are external dependencies because all ServiceMonitor modules are currently being compiled. The -d option works the same way as with a single-module build, and you still list all source files in src, including all module declarations.

Put together, this is the command:

$ javac
    --module-path mods:libs
    --module-source-path src
    -d classes
    ${source-files}

A look into classes shows a directory per module, each containing that module’s class files, including the module descriptor. Neat.

But it’s not always that easy. How would this apply to a project that doesn’t use the single-src structure? This is where a nifty detail of the module source path comes in.

4.3.3 The asterisk as a token for the module name

The module source path can contain an asterisk (*). Although it’s commonly interpreted as a wildcard, which in paths usually means “anything in the directory up to the asterisk,” this isn’t the case here. Instead, the asterisk functions as a token that indicates where on the path the module names appear. The rest of the path after the asterisk must point to the directory containing the modules’ packages.

This way, the compiler can match source file paths to the module source path and deduce which module a source file belongs to. For that to work, each source file must match the module source path.

This may seem complicated, but an example will clarify. Let’s return to the ServiceMonitor application as structured in section 4.1.2, where each module has the common src/main/java directories that contain the source files. Starting in the project’s top-level directory, these are the relative paths to some of the sources:

• monitor/src/main/java/monitor/Monitor.java
• monitor/src/main/java/monitor/Main.java
• monitor/src/main/java/module-info.java
• monitor.rest/src/main/java/monitor/rest/MonitorServer.java
• monitor.rest/src/main/java/module-info.java
• monitor.persistence/src/main/java/monitor/persistence/StatisticsRepository.java
• monitor.persistence/src/main/java/module-info.java

This makes the shared structure pretty obvious: all paths follow the schema ${modules}/src/main/java/${packages}/${sources}.

Looking back at how the module source path is to be used, you can see that ${modules} must be replaced with * and that you have to omit the package directories, leaving */src/main/java. Unfortunately, it doesn’t work yet, because the compiler doesn’t accept the asterisk as the first character—you have to pad it with ./. Now, multimodule compilation works like a charm:

$ javac
    --module-path mods:libs
    --module-source-path "./*/src/main/java"
    -d classes
    ${source-files}

As before, all class files end up in module-specific subdirectories of classes. With what you know about the asterisk being a token for the module name, you could summarize those paths as -d classes/*. Unfortunately, the -d option doesn’t understand the token, and you can’t use it to build output paths like ./*/target/classes. What a shame.

You may wonder how the asterisk relates to the use of --module-source-path src in the first example. After all, there you didn’t specify where the module names would appear, and the compiler was able to deduce them. What may look like an inconsistency at first glance is an effort to make the simple case simple to use.

If the module source path contains no asterisk, the compiler will silently add it as the final path element. So you’ve effectively been specifying src/* as the module source path, which matches the directory structure in that example.

Being able to compile multiple modules if all use the same directory structure should cover most cases. For those with more complicated setups, we need another technique.

4.3.4 Multiple module source path entries

It’s possible a single module source path doesn’t suffice. Maybe different modules have different directory structures or some modules have sources in more than one directory. In such cases, you can specify several module source path entries to make sure every source file matches a path.

The JDK, being a complex project, has a nontrivial directory structure. Figure 4.7 shows just a tiny snippet of it—there are many more directories on all levels.

Assuming you’re in the directory jdk and want to build for UNIX, what would a module source path look like that spans all modules and the correct source folders? The path to the UNIX sources is src/java.desktop/unix/classes or, more generally, src/${module}/unix/classes. Similarly, for the shared sources, it’s src/${module}/share/classes. Putting these two together, you get

--module-source-path "src/*/unix/classes":"src/*/share/classes"

To reduce redundancy, the module source path lets you define alternative paths with {dir1,dir2}. You can unify various paths if they only differ in the name of single path elements. With alternatives, you can unify the paths to source in share and unix as follows:

--module-source-path "src/*/{share,unix}/classes"

4.3.5 Setting the initial module

With everything set up for multimodule compilation, another possibility opens up: compiling a single module and its dependencies just by naming it. Why would you want to do that? Because it no longer requires you to explicitly list the source files to compile!

If the module source path is set, the option --module lets you compile a single module and its transitive dependencies without explicitly listing the source files. The module source path is used to determine which source files belong to the specified module, and dependencies are resolved based on its declaration.

Compiling monitor.rest and its dependencies is now easy. As before, you use --module-path mods:libs to specify where to find dependencies and -d classes to define the output folder. With --module-source-path "./*/src/main/java", you inform the compiler of your project’s directory structure; and with --module monitor.rest, you command it to start with compiling monitor.rest:

$ javac
    --module-path mods:libs
    --module-source-path "./*/src/main/java"
    -d classes
    --module monitor.rest

If classes was empty before, it now contains class files for monitor.rest (specified module), monitor.statistics (direct dependency), and monitor.observer (transitive dependency).

Listings 2.3, 2.4, and 2.5 showed how to compile the ServiceMonitor application step by step. Armed with the knowledge of how to use multimodule compilation, it could instead be done as easily as the following:

$ javac
    --module-path mods:libs
    --module-source-path "./*/src/main/java"
    -d classes
    --module monitor

Because the initial module monitor depends on all other modules, all of them are built. Unlike with the step-by-step approach, the class files don’t go in */target/classes, but in classes/* (using * as a token for the module name).

In addition to making the command easier to read, the combination of --module-source-path and --module also operates on a higher level of abstraction. As opposed to listing individual source files, it clearly states the intent of compiling a specific module. I like that.

There are two downsides, though:

• The compiled class files can’t be redistributed into deeper directory structures and instead all end up below the same directory (in the recent examples, classes). If following stages of the build process depend on a precise location of those files, additional preparatory steps would have to be taken, which may void the advantages of using the module source path in the first place.
• If compilation is kicked off with --module (as opposed to listing all module’s source files), the compiler will apply optimizations that can lead to unexpected results. One of them is unused code detection: classes that aren’t transitively referenced from the initial module aren’t compiled, and even entire modules can be missing from the output if they were decoupled via services (see chapter 10).

4.3.6 Is it worth it?

Does multimodule compilation pay off? I listed three reasons to motivate its use, so it makes sense to return to them:

• Effort —Once you grasp how the module source path has to be constructed, it’s considerably less effort to compile multiple modules. This expressly includes building a particular module and its dependencies, which becomes easier as well. At the same time, build tools usually compile projects one by one, and configuring them to do so all at once may add complexity, particularly if further steps have to be taken to distribute the class files into module-specific directories.
• Performance —With multimodule compilation, the ServiceMonitor application builds in less than a second, which is four times faster than building seven modules step by step. But this is a pretty extreme case, because each module contains only two or three classes. Relatively speaking, there’s a lot of overhead in launching the compiler seven times; but in absolute terms, it comes down to only three seconds. Given the build time of any decently sized project, shaving off a couple of seconds is hardly worth making the build more complex.
• Weak circular dependencies—In this case, there’s no way around multimodule compilation if the build should be free of warnings.

Multimodule compilation is optional, and its benefits aren’t substantial enough to recommend it as the default practice. Particularly if your tools don’t support it seamlessly, setting it up may not be worth the effort. This is a classic “it depends” situation. I have to say, though, I like it for operating on a higher level of abstraction: modules instead of just types.

4.4 Compiler options

With the module system comes a host of new command-line options that are explained throughout this book. To make sure you can easily find them, table 4.1 lists all of those pertaining to the compiler. Have a look at https://docs.oracle.com/javase/9/tools/javac.htm for the official compiler documentation.

Option	Description	Ref.
--add-exports	Lets a module export additional packages	11.3.4
--add-modules	Defines root modules in addition to the initial module	3.4.3
--add-reads	Adds read edges between modules	3.4.4
--limit-modules	Limits the universe of observable modules	5.3.5
--module, -m	Sets the initial module	4.3.5
--module-path, -p	Specifies where to find application modules	3.4
--module-source-path	Conveys a project’s directory structure	4.3.2
--module-version	Specifies the version of the modules under compilation	13.2.1
--patch-module	Extends an existing module with classes during the course of compilation	7.2.4
--processor-module-path	Specifies where to find annotation processor modules	4.2.1
--system	Overrides the location of system modules
--upgrade-module-path	Defines the location of upgradeable modules	6.1.3

4.5 Packaging a modular JAR

On the way from idea to running code, the next step after coding and compiling is to take the class files and package them as a module. As section 3.1.2 explains, this should result in a modular JAR, which is just like a plain JAR but contains the module’s descriptor module-info.class. Consequently, you expect the trusted jar tool to be in charge of packaging. This is how simple it is to create a modular JAR (in this case for monitor.observer):

$ jar --create
    --file mods/monitor.observer.jar
    -C monitor.observer/target/classes .

Putting the new command-line aliases aside, this call works exactly the same as before Java 9. The interesting and implicit detail is that because monitor.observer/target/classes contains a module-info.class, so will the resulting monitor.observer.jar, making it a modular JAR.

Although the jar tool works much like before, there are a couple of module-related details and additions, like defining a module’s entry point, that we should look at.

4.5.1 Quick recap of jar

To make sure we’re all on the same page, let’s take a quick look at how jar is used to package archives. As I just pointed out, the result is a modular JAR if the list of included files contains a module descriptor module-info.class.

Let’s take the command that packages monitor.observer as an example. The result is a module.observer.jar in mods that contains all class files from monitor.observer/target/classes and its subdirectories. Because classes contains a module descriptor, the JAR will also contain it and thus be a modular JAR without any additional effort:

$ jar --create    ①  

    --file mods/monitor.observer.jar    ②  

    -C monitor.observer/target/classes .    ③  

You should consider recording a module’s version with --module-version when packaging it. Section 13.2.1 explains how to do that.

4.5.2 Analyzing a JAR

When working with JARs, it helps to know ways to analyze what you’ve created. Particularly important are the files a JAR contains and what its module descriptor has to say. Fortunately, jar has options for both.

Listing a JAR’s contents

The most obvious thing to do is to look at a JAR’s contents, which is possible with --list. The following snippet shows the content of the monitor.observer.jar created in the previous section. It contains a META-INF folder, which we don’t go into because it’s been around for years and doesn’t pertain to the module system. There’s also a module descriptor, and DiagnosticDataPoint and ServiceObserver classes in the package monitor.observer. Nothing spectacular or unexpected:

$ jar --list --file mods/monitor.observer.jar

> META-INF/
> META-INF/MANIFEST.MF
> module-info.class
> monitor/
> monitor/observer/
> monitor/observer/DiagnosticDataPoint.class
> monitor/observer/ServiceObserver.class

This is not a new command—it just looks different due to new aliases: --list is long for –t, and --file is long for -f. Before Java 9, jar -t -f some.jar would have done the same thing.

Examining module descriptor

A module descriptor is a class file and thus consists of bytecode. This makes it necessary to use tools to look at its content. Fortunately, jar can do that with --describe-module (alternatively -d). Examining monitor.observer.jar, you see that it’s a module named monitor.observer that exports a package of the same name and requires the base module:

$ jar --describe-module --file mods/monitor.observer.jar

> monitor.observer jar:.../monitor.observer.jar/!module-info.class
> exports monitor.observer
> requires java.base mandated

(If you wonder what mandated means, remember from section 3.1.4 that every module implicitly requires the base module, meaning the presence of java.base is mandated.)

4.5.3 Defining an entry point

To launch a Java application, it’s necessary to know the entry point, which is one of the classes containing a public static void main(String[]) method. A class containing that method can either be specified on the command line when the application launches or be recorded in the manifest file that ships with the JAR. Don’t worry if you don’t know exactly how one or even both of these options work, because Java 9 adds a third one that’s the way to go with modules.

When jar is used to package class files into an archive, you can define a main class with --main-class ${class}, where ${class} is the fully qualified name (meaning the package name appended with a dot and the class name) of the class with the main method. It will be recorded in the module descriptor and used by default as the main class when the module is the initial module for launching an application (see section 5.1 for details).

The ServiceMonitor application has a single entry point in monitor.Main. You can use --main-class monitor.Main to record that during packaging:

$ jar --create
    --file mods/monitor.jar
    --main-class monitor.Main
    -C monitor/target/classes .

Using --describe-module, you can see that the main class was recorded in the descriptor:

$ jar --describe-module
    --file mods/monitor.jar

> monitor jar:.../monitor.jar/!module-info.class
# requires and contains truncated
> main-class monitor.Main

It’s interesting that the jar tool has neither the capabilities nor the responsibility to verify your claim that there is such a class. There’s no check of whether it exists or whether it contains a suitable main method. If things go wrong, no error will occur now, but launching the module will fail.

4.5.4 Archiver options

We just explored only the most important options jar has to offer. A couple of others become interesting in different contexts and are explained in the relevant chapters. To make sure you can find them easily, table 4.2 lists the options that have to do with the module system. Visit https://docs.oracle.com/javase/9/tools/jar.htm for the official jar documentation.

Option	Description	Ref.
--hash-modules	Records hashes of dependent modules
--describe-module, -d	Shows the module’s name, dependencies, exports, packages, and more	4.5.2
--main-class	Application entry point	4.5.3
--module-path, -p	Specifies where to find application modules for recording hashes	3.4
--module-version	Specifies the version of the modules under compilation	13.2.1
--release	Creates a multi-release JAR containing bytecode for different Java versions	Appendix E
--update	Updates an existing archive, for example by adding more class files	9.3.3

Summary

• Make sure to pick a directory structure that fulfills your project’s requirements. If in doubt, stick to your build system’s default structure.
• The javac command to compile all of a module’s sources, including the declaration, is the same as before Java 9, except that it uses the module path instead of the class path.
• The module source path (--module-source-path) informs the compiler of how the project is structured. This lifts the compiler operation from processing types to processing modules, allowing you to compile a selected module and all its dependencies with a simple option (--module or -m) instead of listing source files.
• Modular JARs are just JARs with a module descriptor module-info.class. The jar tool processes them just as well as other class files, so packaging all of them into a JAR requires no new options.
• Optionally, jar allows the specification of a module’s entry point (with --main-class), which is the class with the main method. This makes launching the module simpler.