Chapter 9. Packaging

Even though our Toast system is fully functioning with a client and server part, you still can’t send it to your friends because it lives in your workspace. In this chapter we show you how to package that configuration and export it in various forms. The goal is to take Toast from a laboratory prototype to a complete and ready-to-install OSGi system.

As part of this process we also look at increasing the rigor of the Toast component specifications by better managing package imports and exports and through the use of version numbers, version ranges, and privacy mechanisms supported by Equinox.

By the end of this chapter you will have learned about

• Creating product configurations

• Exporting Toast and running it outside the workspace

• Branding the Toast executable

• Exporting Toast for other platforms

• Version numbers, version ranges, and managing package imports and exports

9.1 Defining a Toast Product

So far we have been running Toast directly from the workspace using launch configurations. To get Toast out of the workspace, you have to define some product configurations. A product configuration gathers together all of the information needed to create a complete system in one convenient place. This includes the list of bundles and features as well as the information about splash screens, executable icons, and so on.

Product configurations themselves are not defined by OSGi, nor are they part of your deployed application. They are development-time artifacts used by the tooling to help you describe the system you want to create. They are somewhat like launch configurations on steroids. Here we will take the two launch configurations used to run the client and the back end and convert them to product definitions. Let’s start with the back end.

9.1.1 Creating a Product Configuration

First we will create the product configuration using the following steps. Then the subsequent sections take you on a guided tour of the product editor and highlight points of interest.

• Use File > New > Project > General > Project to create a new project called ToastBackEnd. It is a good idea to create a separate project to hold these higher-level artifacts. Since products are really configurations of function, they stand apart from the function—bundles and features—and have a home of their own.

• Select the new org.equinoxosgi.toast.product.backend project, and if you are in the Plug-in Development perspective, use File > New > Product Configuration to start the New Product Configuration wizard, as shown in Figure 9-1. Otherwise, use File > New > Other... > Plug-in Development > Product Configuration to get the wizard.

Figure 9-1 New Product Configuration wizard

• Fill in the wizard fields as shown in the figure by setting the product definition file name to backend.product.

• Next, choose a technique for initializing the configuration. The wizard can extract information from an existing product or launch configuration or simply create a basic product. If you have been following along, you already have a launch configuration called backend. The currently defined configurations are listed in the Use a launch configuration drop-down. Enable this option and pick a configuration you have already used and which you know launches the back end.

• Click Finish.

The wizard reads the launch configuration and uses it to build a product definition. In particular, it gets the list of bundles and any configuration information and command-line arguments used. The new product configuration is opened in an editor, similar to that shown in Figure 9-2.

Figure 9-2 Back end product overview

9.1.2 The Overview Page

As with the bundle editor, the product configuration editor gathers together information from many different files and presents it all in one place. The configuration information is grouped onto several tabs within the editor. The Overview page in Figure 9-2 shows the General Information section at the top.

This gives access to the ID, Version, and Name of the product. None of these fields is mandatory for running the product, but when exporting or building, the ID and Version will be particularly useful.

• Set the ID to org.equinoxosgi.toast.backend. The wizard filled in the version. For completeness, fill in the Name, Toast Back End.

The Product Definition section is next. It exposes the Product and Application values. These identify runtime branding and system entry point information, respectively. Leave these fields blank for now. We will use them in a later chapter.

Notice the two radio buttons beside The product configuration is based on at the bottom of the section. This allows you to say how you want to list the contents of the product—as either a list of bundles (plug-ins) or a list of features. For now we will use bundles, so ensure the plug-ins button is selected.

9.1.3 The Dependencies Page

Next take a look at the Dependencies page of the product editor, as shown in Figure 9-3. Here there is a simple list of bundles, since we decided to have a bundle-based product definition. You can add and remove bundles from the list using workflows similar to those seen in the launch configuration dialog.

Figure 9-3 Back end product dependencies

The Add Required Plug-ins button on the Dependencies page looks very attractive. The faith that you can click this one little button and magically all your requirements will be met, however, is sadly misplaced. In general, this button does more harm than good. It can be useful for getting an initial start or “just getting something running,” but it can also wreak havoc on your bundle list and add all manner of unnecessary entries.

A better, though sometimes laborious, approach is to use the validate button in the top right of the editor to check for missing dependencies. This enables the same workflow as in the launch configuration dialog by showing you a list of all bundles with missing dependencies along with what those dependencies are. You can then add the needed bundles manually. Following this workflow, you get a good understanding of the requirements of your system and avoid lots of extra stuff being added just because it is available.

If you decide to use a feature-based product, as we will in later chapters, the list on the Dependencies page will include just features. This can be very convenient for readily seeing the structure and content of your product as it gets more complex. Whole collections of functions can be manipulated simply by adding or removing features. Validation continues to function, but you can no longer add individual bundles to satisfy missing dependencies. Rather, you have to find and add appropriate features that contain bundles that address the problem.

9.1.4 The Configuration Page

Flip over to the Configuration page. Here you see a list of bundles that are included or implied by the content of the Dependencies page. For each bundle listed, you can set whether or not it should be Auto-Started and at which Start Level. These are the same concepts we have seen in the launch configuration dialog. Auto-Start means that the associated bundle is both installed and started when the system is run. The start level helps define the order of starting. Start levels are discussed in more detail in Chapter 22, “Dynamic Best Practices.”

In previous chapters we set up the launch configurations to ensure that the Default Auto-Start setting on the Bundles page was set to true. This ensured that all bundles listed in the launch configuration would be installed and started. While somewhat brute-force, it also avoids the tedious and error-prone listing of individual bundles to be started.

Unfortunately the current product editor Configuration page does not expose similar capability. Users are forced to explicitly list the bundles to start. To address this shortcoming, we’ve implemented an “auto-starter” bundle, org.equinoxosgi.core.autostart. A project containing the bundle is included in the Samples Manager for this chapter. The bundle simply starts and waits for other bundles to become resolved. On discovering a resolved bundle, the auto-starter starts the bundle. Using this you can get the effect of the launch configuration setup as follows:

• Use the Samples Manager to load the auto-starter bundle from the Chapter 9 solution.

• On the Dependencies tab, add org.equinoxosgi.core.autostart to the list of bundles in the product configuration.

• Flip over to the Configuration tab and mark it as Auto-Start = true. Having done that, there is no need to list any other bundles in the configuration—all bundles will be started upon becoming resolved.

For those familiar with the Equinox config.ini file, the Configuration page allows you to essentially manipulate the contents of the osgi.bundles entry in that file. In fact, given this capability and the various Arguments entry fields on the Launching page, the config.ini file is largely obsolete at development time—it can now be fully generated from the information in the product file. As such, you can ignore the contents of the Configuration File section on the Configuration page.

9.1.5 The Launching Page

The executable is the program that end users run when they want to start Toast, such as toast.exe on Windows. Having an executable more tightly integrates with the underlying system. For example, the application shows up correctly in the process lists, the Windows Taskbar, the Mac Dock, and so on.

You could just use the executable that comes with Eclipse, but of course you don’t want to tell users to “double-click on eclipse.exe” to run Toast—you want a toast.exe for them to run. Furthermore, the Eclipse executable has Eclipse icons associated with it. It makes more sense for these icons to be specific to your system.

The Program Launcher section shown in Figure 9-4 allows you to set this up.

Figure 9-4 Executable branding

The Launcher Name box allows you to enter the simple name of the executable. You should not append .exe. That information is platform dependent, and PDE takes care of it when the product is exported. Here we entered backend for the launcher name.

Below the Launcher Name is a series of entry fields for identifying the icons associated with Toast’s executable. It turns out that each OS requires different image sizes and formats, so the product editor has a section for the supported OSs. You need to fill in the image names only for the OSs in which you are interested. The Toast images are in the sample files for this chapter in the ToastBackEnd/branding folder. These images are used in the process of exporting Toast. During export, PDE creates an executable program that behaves exactly like the standard Eclipse executable but is renamed and branded with the icons you specified. You can add these to the product definition if desired.

Also on the Launching page are areas for specifying Program Arguments and VM Arguments as we have seen with the launch configurations in previous chapters.

9.1.6 Running the Product

Finally, launch the product using the links in the Testing section of the Overview or the run and debug buttons in the top right corner of the editor. The Toast Back End should start up as before. You can launch the client as before to verify that the back end is working correctly.

You may not have noticed, but launching the product caused a new launch configuration to be created. Open up the launch configuration dialog (Run > Run...) and take a look at the list. There is backend, the one you used as a base for the product configuration, and a new one called backend.product (assuming that is the name you used for your product configuration file). This new configuration is used by the product editor to launch your product. Since it is a normal launch configuration, you can run or debug it directly and use keyboard shortcuts such as F11 to debug the last launched configuration.

9.1.7 Productizing the Client

Converting the client to be defined as a product is exactly the same as converting the back end. Here is a capsule summary of the steps required:

• Create a ToastClient project and client.product product configuration. Base the product configuration on the client launch configuration you have been using to date.

• Add the org.equinoxosgi.toast.core.autostart bundle to the product dependencies and ensure it is marked to auto-start.

• Export the product and run it against the exported back end.

9.2 Exporting Toast

To get Toast out of the workspace, you have to export it. To do this, you identify what parts of the various projects should get packaged up into the corresponding binary bundles. For example, the org.equinoxosgi.toast.backend.emergency project has several development-time artifacts that should not go into the final bundle. Since we have been running from the workspace up until now, this has not been an issue. To export, you need to make this explicit.

The Build page in the bundle editor for the back end bundle helps with this. The Binary Build section shown in Figure 9-5 lists the set of development-time files and folders that are also part of the bundle’s structure. Notice that several files are already selected. PDE takes care of adding things like the compiled Java classes, the META-INF directory, and other elements that are known to be required at runtime. Your bundle may require additional files such as icons, messages, or web content.

Figure 9-5 Binary Build specification

Take the time now to go through each of the bundles and ensure that the OSGI-INF directory is selected in the Binary Build section of the bundle editor’s Build page.

You can export the product by running the export wizard as outlined here:

• Find the backend.product file in the Package Explorer or Navigator. Right-click and choose Export... > Plug-in Development > Eclipse product. Alternatively, open the product editor, select the Overview page, and click on the Product Export wizard link. There is also a convenient export button at the top right of all product editor pages. Either way, you should see the Product Export wizard, as shown in Figure 9-6.

Figure 9-6 Product Export wizard

• First, ensure that backend.product is selected in the Configuration drop-down.

• Fill in the Root directory, the top-level directory that is embedded in the export output. For example, it is useful to set this to be the name of your product with the version number. This way, people can extract the product and it gets laid out on disk in a descriptive directory structure. For now, use BackEnd-1.0 in this field.

• In the Synchronization section, uncheck the Synchronize before exporting box. We did not define a product extension for this product, so we don’t need to synchronize. PDE will complain if you leave the box checked.

• Next, pick the export Destination and set the shape of the export Archive file or Directory. This setting does not affect the content of the output, just the layout. For now choose Directory so you can easily test what you are exporting. Later you can export as an archive to make Toast easier to distribute. Put the output in any convenient location, but remember that the root directory entered earlier is appended to the location specified. We use c: in the example here.

After you click Finish, PDE starts the export. It should run in the background, so you can continue using Eclipse while the export completes. First, it compiles the code from the workspace according to the configuration you described. The export wizard then gathers the compiled code and required parts of the target platform and outputs them to the specified location.

When the export is done, c:BackEnd-1.0 contains a fully functional Toast Back End that runs outside the workspace. Navigate to c:BackEnd-1.0ackend.exe. Run the executable and enjoy your stand-alone Equinox-based server!

Undoubtedly you will want to share this with your friends and coworkers. Go back and export the product again. This time, specify an Archive file output and mail them the archive.

9.3 Packaging for Other Platforms

But what about your friends who use different operating or window systems or run on different hardware? They can’t run the Toast elements you just exported because they contain platform-specific code—at least the executable. To package for other platforms, either you need to run the IDE on that platform, which is hard to manage, or you must have the code for those platforms on your machine and cross-deploy. This is much easier.

First you need to acquire all the platform-specific code needed for these other platforms. Fortunately, Eclipse supplies a delta pack for every build. The delta pack contains all parts of the Eclipse SDK, including Equinox, that are platform-specific. This includes the launchers as well as graphics libraries and other support for various platforms. So, for example, if you are on Windows and want to export the Toast Client for Linux/GTK, the delta pack has everything you need.

We included the delta pack in the target platform setup in Chapter 3, “Tutorial Introduction,” and described how to get it if needed.

For now the Toast Client is relatively simple, and all we need from the delta pack is the executables. In later chapters we will add a GUI to the client and will need various SWT bundles.

With the delta pack in the target you can export for multiple platforms as follows:

• Open the Export wizard and complete the first page of the wizard as before. This time check the Export for multiple platforms option, and then click Next to get to the Cross-platform export page shown in Figure 9-7.

Figure 9-7 Cross-platform export

• Select the set of platforms for which you want to export and click Finish.

The export output goes to archives or directories specific to the related platform. For example, the Windows output appears in the directory named c:win32.win32.x86Client-1.0.

9.4 Getting Serious about Component Definition

Now that you have Toast exported and running independently of your workspace, others can start to build on your fleet management platform. To facilitate this, we need to get more serious about the rigor of our component definitions. Many cultures have a saying akin to “Good fences make good neighbors.” It is the same with components. Having a clear definition of what is in and out of bounds helps set expectations and smooths collaboration. Defining these boundaries in a formal way enables the tooling to help you write conformant code and the runtime to enforce the boundaries. In this section we talk about versions and version ranges and best practices for exporting packages as the key elements of component definition.

9.4.1 Versions and Version Ranges

In OSGi, bundles and packages are uniquely identified by a combination of their identifiers, Bundle-SymbolicName and package name respectively, and their version numbers. When you specify a dependency on a bundle or package, you can give a version range to narrow the set of content you are willing to accept. This is an extremely useful mechanism, but its value depends on the policy used around setting version numbers in the first place—if version numbers change randomly, ranges cannot help.

Since the IDs used for a given entity never change, the version number must change whenever the content of the entity changes. Version numbers in OSGi are made up of four parts: major.minor.service.qualifier:

Major—Differences in the major part indicate significant differences such that backward compatibility is not guaranteed.

Minor—Changes in the minor part indicate that the newer version of the entity is backward compatible with the older version, but it includes additional functionality and/or API.

Service—The service part indicates the presence of bug fixes and minor implementation (i.e., hidden) changes over previous versions.

Qualifier—The qualifier is not interpreted by the system. Qualifiers are compared using standard string comparison.

Following these numbering semantics is important. As parts of the system come to depend on one another, they need to know about changes in the compatibility contract as well as updating their requirements. For example, the following package import declaration claims that the back end bundle works with any version of the Toast core package. This is likely incorrect.

A more likely scenario is that the back end is buying into the Toast core API at a specific minimum level, for example:

In this case, the back end is happy with any core package from 1.0 to 2.0 (not including 2.0). Given the version number semantics described previously, the back end is saying that it likes the 1.0 level API and does not want to be broken. A future version of the bundle may use some new API added to core package version 1.1. In that case the back end would change its import version range to be [1.1.0,2.0.0).

Conservative teams who want only bug fixes might narrow this range to [1.1.0,1.2.0) with the goal of getting only bug fixes. Paranoid teams seek to lock down the exact version used and would narrow the range further to [1.1.0,1.1.0].

The challenge here is to specify the version dependencies loosely enough so that the dependents work in various settings but tightly enough that the required API contracts are guaranteed. Of course, this mechanism works only if producers update their version numbers according to the semantics and give their consumers a chance to get it right—it takes two to collaborate.

9.4.2 Exporting Packages and Friendship

OSGi has very strong boundary enforcement—you must state an Import-Package or Require-Bundle dependency before you can see the contents of another bundle, and you will only ever see contents that the bundle decides to expose. OSGi fences are quite high and robust.

This can be good as it supports implementation and information hiding. Hiding is great because it limits the assumptions you can make. With no knowledge to the contrary, you have to assume that anything (or nothing) can happen. Unfortunately, this black-and-white view of the world is not always optimal. For example, standard OSGi does have the notion of private collaboration between bundles. Exported packages are exported to everyone and registered services are available to all. Over the years we have seen many cases where this was just too rigid.

Returning to our fence analogy, even the most robust fences have gates. Whereas a fence stops people from traversing the boundary, a gate declares that there is a boundary but allows people to pass, understanding that there are new rights and obligations. The OSGi standard itself has the binary Export-Package gate—the package is available either to all or to no one. In Equinox we extended Export-Package with the x-friends and x-internal directives:

x-internal—Use this to mark an exported package as containing internal implementation details that are not considered API.

x-friends—Use this to mark an exported package as accessible only to a list of named bundles.

PDE includes tooling for these directives in the Package Visibility section of the Runtime page of the manifest editor, as shown in Figure 9-8. Here the package org.equinoxosgi.toast.internal.core is hidden from all other bundles—it is marked as x-internal:=true.

Figure 9-8 Package visibility manipulation

9.4.2.1 The Equinox x-internal Directive

Marking a package internal is like putting an “Enter at your own risk” sign on the gate. It tells people that they can use the package but there are no guarantees about the suitability, robustness, or durability of the code.

For example, it is common for Eclipse platform bundles to export all internal packages but mark them as x-internal. This supports early adopters and people with novel use cases as they seek to push the limits of your function. The markup tells them where the limit is. PDE supports developers by marking uses of internal code with discouraged access warnings. Under this model, developers wanting to stay in bounds have a clear indication that they are outside of their policies, and more aggressive developers can exercise informed consent. The discouraged access feedback can be set to ignore, warn, or error in the Java compiler preferences by searching for “discouraged.”

9.4.2.2 The Equinox x-friends Directive

Marking a package as internal is enough to enable access and denote API—if it is marked as internal, it is not API. Conversely, if it is not API, you should not use it. But what about closely related bundles that are collaborating at the implementation level? In this case we would like to list particular consumers as approved to use a package—we want to note them as friends.

The x-friends directive is a specialization of x-internal in that it marks the exported package as internal but allows undiscouraged use of the package by the listed bundles—they do not get errors or warnings when referencing the package. Adding friends to a package can be done by adding them to the Package Visibility list shown in Figure 9-8. This serves two purposes: It tells the consumer that it’s OK to use the package, and it reminds the producer that there are approved consumers of what would otherwise be non-API types. It essentially notes that there is a private contract between the two parties around the evolution of the code in the package.

9.4.2.3 Strict Mode

Some teams have a very strong API ethic that drives them to reject all use of internal code from other bundles. This is a powerful, though sometimes hard-to-maintain, position. Nonetheless, to support these teams, Equinox includes a strict mode. When the framework runs in strict mode, all internal package exports are ignored and friends-only access is enforced. The net effect is that no one in the system can violate the API boundaries—API lockdown. By default the framework does not use strict mode. You can enable strict mode using the following VM argument:

-Dosgi.resolverMode=strict

9.5 Summary

Exporting Toast as described here is the first step toward making Toast an engineered system offering. Without exporting, Toast is just a wad of code in your workspace. Exported, Toast becomes a full-fledged stand-alone system. PDE’s exporting facilities make it easy to create and brand these packages—even across platforms. Now you have something to send to others.

Rigorous component design is an integral part of collaboration. Strong API boundaries support and facilitate this. Key to the definition of API is the versioning and usage qualification. Here we showed that OSGi, Equinox, and PDE provide a number of powerful mechanisms and tools for defining and enforcing API boundaries.

Given a set of well-defined components that can be exported, further chapters describe how to automate component builds and how to compose and deploy components using Equinox’s p2 provisioning technology in various scenarios.