Chapter 16. Extensions

For the majority of this book we have used OSGi services to facilitate the interaction between bundles. Services are reasonably lightweight and, with the advent of the Declarative Services mechanism, relatively easy to work with. As mentioned in Chapter 2, “Concepts,” services are not the only collaboration game in town. The Equinox Extension Registry can also be used to hook bundles together.

In this chapter we look at how the Extension Registry works and compare and contrast it with the service registry. In particular we will talk about

• Extensions and extension points

• The dynamic behavior of the Extension Registry

• Providing data in extensions

• Scalability and startup performance (event storm, caching)

• Extension and extension point lifecycle

• Relationship with the OSGi service registry

16.1 The Extension Registry

OSGi provides a framework for defining and running separate components. As we have seen throughout this book, it also provides a service mechanism that enables inter-bundle collaboration. Equinox adds to that the Extension Registry as a means of facilitating collaboration. The Extension Registry works as follows:

• Bundles open themselves for extension or configuration by declaring an extension point and defining a contract. Such a bundle is essentially saying, “If you give me the following information, I will do. . . .”

• Other bundles contribute the prescribed information in the form of extensions. These provide data and/or identify classes to run and locations to access.

• The Extension Registry discovers both extensions and extension points and links them together according to their contributing bundles’ lifecycle.

• Extension point definers are free to access and use contributed extensions in support of their operation.

In this chapter we enhance the back end portal from Chapter 13, “Web Portal,” to allow for portal action contribution and discovery via the Extension Registry. In this approach the portal bundle declares an actions extension point and a contract that says,

“Bundles can contribute actions extensions that define portal actions with a path, a label, and a class that implements the interface IPortalAction. The portal will present the given label to the user organized according to the given path and such that when the user clicks on the label, a particular URL will be accessed. As a result of the URL request, the portal will instantiate the given action class, cast it to IPortalAction, and call its execute method.”

Figure 16-1 shows this relationship graphically.

Figure 16-1 Extension contribution and use

Extension-to-extension-point relationships are defined using XML in a file called plugin.xml. Each participating bundle has one of these files. In this scenario, org.equinoxosgi.toast.backend.portal’s plugin.xml includes the following markup:

org.equinoxosgi.toast.backend.portal/plugin.xml
<extension-point id="actions" name="Portal Actions"/>

A bundle containing an action then contributes an extension using the markup shown in the following plugin.xml snippet:

The actions extension point example here plays out as follows: When the portal composes the “browsing” page, it selects and presents the contributed extensions related to this page. Relationships are built using the value of the action attribute. In this case the “tracking” action is added to the generated HTML page as a link using the given label and accessing a related URL. When the user clicks on the link, the URL is accessed and the portal instantiates the TrackingAction class, casts it to IPortalAction, and calls its execute method.

This seemingly simple relationship is extremely powerful. The portal has effectively opened itself up as a pluggable web UI framework. It does not need to know about any of the contributions ahead of time, and no code is run to make the contributions—everything is declarative and lazy. These turn out to be key characteristics of the registry mechanism. Here are some other characteristics worth noting:

• The mechanism can be used to contribute code or data.

• The mechanism is declarative—bundles are connected without loading any of their code.

• The mechanism is lazy in that no code is loaded until it is needed. In our example, the TrackingAction class is loaded only when a user clicks on the link. If the user does not use the link, the class is not loaded.

• This approach scales well and enables various approaches for presenting, scoping, and filtering contributions.

• Extensions and extension points are well used. Eclipse-based systems use them for everything from contributing views and menu items to connecting Help documents and discovering builders that process resource changes.

The Eclipse IDE includes quite extensive tooling for the Extension Registry. To access the tooling, click on one of the links in the Extension/Extension Point Content section of the Overview page of a bundle’s manifest editor, as shown in Figure 16-2.

Figure 16-2 Enabling Extension Registry tooling

The next two sections give the details of how extensions and extension points work and are defined.

16.2 Extension Points

In Chapter 13, “Web Portal,” the portal was extended using the Whiteboard Pattern to detect adding IPortalAction services. In the example in the preceding section we proposed the use of the Extension Registry for this. Here we show you how to add Extension Registry support to the portal.

In the original design of the portal we purposely set out to isolate the use of OSGi and services from the domain logic of the portal itself. This was done by introducing the IActionLookup mechanism—the portal discovers available actions by consulting a service that manages actions. The original implementation of that service was based on OSGi services and called ServiceActionLookup. Here we’ll add ExtensionActionLookup, an IActionLookup service based on the Extension Registry.

The first step in making your system extensible is to define an extension point. This involves both defining the contract and writing the code to implement the contract.

Creating an extension point declaration is done using the Add... button on the Extension Points page of the relevant bundle editor. Figure 16-3 shows the actions extension point in the org.equinoxosgi.toast.backend.portal bundle.

Figure 16-3 New extension point declaration

The Extension Point Schema file location field is filled in automatically. Schemas are used at development time to describe the expected structure of extensions contributed to an extension point. They describe everything from the set of tags and attributes to the kinds of values placed in the attributes. See Plug-in Development Environment Guide > Reference > Wizards and Dialogs > New File Creation Wizard in the online Help for details on schemas and using the schema editor. Here you can just load the schema file using the Samples Manager.

Now that the extension point structure has been defined, various bundles can contribute extensions that conform to the schema. Section 16.3, “Extensions,” details the workflow for doing this.

The bundle contributing the extension point should also include some code that accesses any contributed extensions and implements its side of the contract. All extensions and extension points are maintained in an Extension Registry. The registry allows contributions to be navigated and looked up by name. The following snippet shows how the ExtensionActionLookup acquires actions:

Looking through the code, we see that the method first uses the extension point’s fully qualified ID to get a set of configuration elements. Configuration elements are object representations of the XML elements under the <extension> tag in the plugin.xml files; for example, the <action> markup in the previous example surfaces as a configuration element.

Accessing the registry as shown returns a list of the top-level configuration elements across all extensions contributed to the given extension point. This is convenient where the identity of the extensions and the number of configuration elements per extension are not important, as in this example.

The preceding snippet scans the set of configuration elements and contributed actions, and when it finds the one requested, it instantiates the class identified in the extension by calling createExecutableExtension. The result object is then used according to the contract of IPortalAction.

16.3 Extensions

Creating an extension is quite straightforward. As we’ve seen, each extension point has an associated schema that defines the structure of any contributed extension—the contract. The New Extension wizard and Extensions page in the bundle editor walk you through the process and validate your entries according to this schema.

To add the extension just described, edit the bundle that will contribute the extension. Here we will have the back-end-tracking UI bundle contribute one of its actions using an extension:

• Open the org.equinoxosgi.toast.backend.tracking.ui manifest editor. On the Extensions page, click Add... and create an extension of type org.equinoxosgi.toast.backend.portal.actions.

• Right-click on the extension and add an action using New > action from the context menu.

• When you click on the newly added action element, the Details pane at the right shows the default values for the action. Update the action and label fields to match those shown in Figure 16-4.

Figure 16-4 Adding an action extension to the portal

• Click on the Browse... button beside the class field and enter TrackingAction to identify the implementation for this action. Conveniently, since we wrote that action as a POJO, we can reuse it here.

If you need to create a class for the action, you could use the class link. This opens the New Java Class wizard with most of the fields, including the interface IPortalAction, already filled in. All you have to do is enter a class name.

16.4 Advanced Extension Topics

The Extension Registry has a number of advanced features put in place to handle specific situations. This section details some of these.

16.4.1 Contribution IDs

Extensions and extension points may have IDs. Traditionally developers needed to provide only a simple ID—just one word. The full ID of the contribution was then constructed by prepending the symbolic name of the bundle contributing the element. In this way the flat registry namespace could be managed and uniqueness guaranteed. This assumes, however, that the bundles contributing to the Extension Registry are singletons.

A singleton bundle is a bundle whose Bundle-SymbolicName header includes the directive singleton:=true. This indicates that the framework must resolve at most one bundle with this symbolic name even if multiple bundles with the same symbolic name are installed. All others are left unresolved and so do not participate in execution. As a result, there can be only one bundle of any give symbolic name, and the automatically created full contribution IDs cannot collide.

While beneficial, the automatic full qualification of IDs couples the registry contribution with the bundle making the contribution. This prevents replacement and refactoring. In practice today, developers are able to specify fully qualified extension and extension point IDs, and contributions are no longer tied to their contributor.

16.4.2 Named and Anonymous Extensions

Extensions need not have IDs. IDs are useful where direct and distinct extension access is needed, for example, if one part of the system identifies an extension to be used and another contributes that extension. It is convenient and efficient for the party in the middle to simply take the given ID, look up the corresponding extension, and use the data provided. To name an extension, simply give it an ID attribute as shown in this snippet:

The choice of whether or not to have named extensions is completely up to the party defining the extension point. The benefit of having IDs on extensions is that you can call the optimized IExtensionRegistry method, getExtension(String, String), to fetch a specific extension as opposed to having to traverse or remember the extensions yourself. For example, the previous code snippet repeatedly iterates over all contributed configuration elements to find the appropriate action, whereas if an ID were supplied, it could simply access the registry and directly look up the required extension.

While having a system-managed ID can be convenient and efficient, it can also limit you to one contribution per extension. Using anonymous extensions, for example, allows for any number of <action> elements in an extension to the org.equinoxosgi.toast.backend.portal.actions extension point. This is more concise and eliminates the need to manage a set of IDs that are not used.

There is no one answer here. Both anonymous and named extensions are widely used. Use whichever approach best suits your situation.

16.4.3 Extension Factories

In the previous example, the class to instantiate was given as a fully qualified class in the class attribute of the extension. This requires the identified class to implement a public zero-argument constructor and restricts the initialization that can be done on the resultant object. Another approach is to use an extension factory to create or discover the desired object or parameters to inject. Factories are useful as they allow more complex discovery and initialization. For example, one could implement a factory that uses OSGi service lookup or web service discovery to find suitable objects. Factories hide these implementation details from the extension point.

To use an extension factory, ensure that the class identified in the class attribute of the extension implements IExecutableExtensionFactory. When createExecutableExtension is called, this factory class is instantiated and the resultant factory is given the related configuration element. The factory then uses the information given to determine what kind of object to return. Note that the extension factory is free to decide how it creates the requested object. The Javadoc for IExecutableExtension and the individual factories details the acceptable syntax of the data following the factory name in the markup.

16.5 Extension Registry Lifecycle

The Extension Registry is tied to the RESOLVED lifecycle of bundles; that is, the extensions and extension points contributed by a bundle are woven into the Extension Registry when the contributing bundle is resolved—when all of its dependencies are met. Similarly, if the bundle subsequently becomes unresolved—say, when a prerequisite bundle is removed—its contributed extensions and extension points are removed from the registry. Section 23.6, “Bundle Lifecycle,” gives more detail on the lifecycle of bundles.

A bundle is generally resolved only once in a given system—when it is installed—and remains resolved between runs of the system; that is, it is resolved once but started many times as the system is stopped and started. Each time a bundle is (un)resolved, its contributions to the registry are updated, and IRegistryChangeEvent events containing IExtensionDeltas are broadcast to registered IRegistryChangeListeners. The API for these players is summarized in the following snippet. These lifecycle events are key to the proper management of dynamic behavior of your system. This is discussed in detail in the next section.

Given the infrequency of bundle-resolution-related events, and thus registry changes, it is quite easy for a registry implementation to cache its state and greatly reduce startup and access time. Similarly, the registry generates events only when bundles are resolved or unresolved, so there are relatively few registry change events broadcast, and the ones that are, can be batched to further reduce chatter.

16.6 Dynamic Extension Scenarios

Managing dynamic behavior in a system is challenging. We have dedicated an entire chapter, Chapter 21, “Dynamic Best Practices,” to talk about this in detail. Here we talk about the issues specifically related to using the Extension Registry in dynamic environments.

The problems arise as a result of caching information from others in your bundle. Whether it is caching of the extension itself or of any objects created via executable extensions, the cache’s coherence must be maintained; that is, where you use executable extensions to provide code, or descriptive extensions to provide data, dynamism is something to think about.

Extensions are used throughout Eclipse. Equinox includes several extension points for contributed applications, servlets, repository providers, and so on. Section 18.3, “Declarative HTTP Content Registrations,” gives an example of contributing Toast servlets and resources via two extension points:

org.eclipse.equinox.http.registry.servlets
org.eclipse.equinox.http.registry.resources

In that example, contributed extensions identify servlet classes or resource folders to be served by the HttpService. If the HTTP registry bundle was not dynamic-aware for addition, it might miss the addition of these extensions, and the Toast Back End would be unable to handle various client requests. If it was not dynamic-aware for removal, it would miss the removal of the extensions and continue trying to dispatch requests to servlets that are no longer active in the system.

The next three sections enumerate dynamic extension scenarios and how to handle them. In general, they revolve around whether or not the supplied extension is descriptive or executable and whether or not you cache values discovered in the registry or consult the registry each time you need something.

16.6.1 Scenario 1: No Caching

If your bundle consults the Extension Registry each time a value is needed, the burden of being dynamic-aware is substantially reduced. All data lives in and is maintained by the Extension Registry—no work for your bundle. The downside of this approach is that accessing the Extension Registry is likely slower than consulting an optimized, special-purpose data structure or cache in your bundle. If the extensions are accessed infrequently, however, this trade-off is reasonable.

The HTTP registry can use this approach if it registers proxy servlets with the HttpService and then dynamically instantiates the contributed extension servlet class for each request. In this way the HTTP registry never retains an instance of a contributed class. This is fine for small systems, but as the number of contributions increases, it becomes more expensive to find the extension whose alias matches the current request.

16.6.2 Scenario 2: Extension Caching

The performance of extension lookup can be improved by caching the structure of the extensions. For example, the HTTP registry could keep an explicit, in-memory table keyed by servlet alias where the value is the contributing extension. This table needs to be updated accordingly when a bundle contributing a servlet extension is added to the system. Similarly, if an existing bundle is removed, its contributed servlet extension must be removed from the table. Updating this cache is quite trivial—the changed key/value pair is simply added or removed.

This approach improves the time to access the extensions and find the correct servlet or resource class for a given request, but it suffers on two counts: First, it essentially duplicates the extension structure and data; and second, it requires additional infrastructure to implement dynamic awareness.

In some cases, there may be no choice. For example, some extension points such as the Eclipse Help system or the Resources bundle’s markers extension point are used to create complex graphs of contributed elements. These structures must be computed rather than read directly from the Extension Registry, so they inherently require some level of caching. As such, the defining bundle must implement some dynamic-awareness support to clean up the cache when the set of resolved bundles changes. The cache cleanup is more complicated if the cached data structure is inherently interconnected.

As outlined in Section 16.5, “Extension Registry Lifecycle,” the registry supports this need by broadcasting registry change events (IRegistryChangeEvent) to registered registry change listeners (IRegistryChangeListener) whenever Extension Registry contributions are added or removed. The listeners can then query an extension delta (IExtensionDelta) to find out what changed and how. This information is in turn used to update the cached data structures.

Updating the cache need not be a heavyweight operation. For example, if the cache is not critical and rebuilding is not overly expensive, flushing the entire cache on change is reasonable. This approach is sketched in the following code snippet:

Registry change listeners are notified serially, but you still have to be concerned about threads accessing the cache while listeners are clearing it. The code in getCache gets a reference to the cache and uses that reference. The cache may be flushed at any point after that and the old value used. That’s acceptable because there were no guarantees about ordering here anyway. Notice that this coding pattern closes the window between getting and testing the cache state, and returning the cache as the result.

Of course, another situation may not be that easy. If cache entries are expensive to rebuild, it is better to add and remove entries incrementally. The following snippet shows how to handle change events and traverse the deltas:

Here, the listener queries the delta from the change event for any changes to the org.eclipse.equinox.http.registry.servlets extension point. For extension additions, you can choose to aggressively populate the cache with the new extensions or just ignore the additions and look for them later when you have a cache miss. For removals, you need to tear down any data structures and remove the extension from the cache.

16.6.3 Scenario 3: Object Caching

What happens if creating the contributed servlets is expensive? Even with extension caching, the contributed servlet class still has to be looked up and instantiated for each request. Assuming the servlets are properly context-free, one of each type could be cached and used to handle packets as required.

This kind of caching is different from and independent of the extension caching we saw in Scenario 2. Here the HTTP registry would hold on to the servlet class or instances. This prevents the contributing bundles from being properly garbage-collected. Furthermore, the created servlets cannot be left active when the contributor is removed, as they are likely to be invalid because its bundle has been shut down and removed.

If the HTTP registry is to be dynamic-aware, the servlets it instantiates must be tracked so they can be cleaned up if their contributor is removed. More generally, cached objects often play a deeper role in your system; that is, they may be woven tightly into the fabric of the application. The challenge is to understand the interconnections and ultimately reduce them so there is less to clean up.

To update the object structure, you can use the same sort of listener as the one described in Scenario 2. This time, however, the cache contains contributor-supplied objects (i.e., servlets). In the case of the HTTP registry, it turns out that there is only one reference to any given servlet. The Extension Registry change listener simply causes the unregistration of all servlets registered on behalf of the deleted extension. The following code snippet shows this in action:

To handle more complex situations, Equinox has utility classes that help with tracking and disposal of object references. In the next code snippet, the HttpRegistryManager implements IExtensionChangeHandler and registers itself with an IExtensionTracker to get notification of changes in select extensions.

Extension trackers track objects created for an extension. These objects might be the result of using IConfigurationElement.createExecutableExtension or an object created manually. Either way, the objects share the common trait that they should be cleaned up when the related extension disappears.

Looking through the code from the top down, we see that initializeTracker creates an ExtensionTracker and adds the HTTP manager as a handler. Notice that the filter used means that the manager is notified of changes to extension points only in the HTTP registry bundle’s namespace. You can narrow this to individual extension points, but this is good enough here.

The manager is notified of changes through addExtension and removeExtension. Generally it is preferable to create required objects on demand, so there is no work to do when extensions are added.

On removal, however, we do need to ensure that any created servlets are removed. The removeExtensions method receives a list of objects that the tracker is tracking relative to the given extension. This list is populated by the HTTP manager whenever it creates a servlet, as shown in getServlet. Ignoring the detail of how the servlet contribution is discovered, at some point, a class supplied by an extension is instantiated. The resultant object is then registered with the tracker using registerObject. The servlet is then added to the HTTP manager’s internal data structure. It is this data structure that needs to be updated if the extension is removed. You saw that code in the method removeExtension.

Notice also that the tracker uses weak references (REF_WEAK) to track the servlets. This ensures that they do not stay live in the system just because they are being tracked.

The use of extension trackers is to some extent overkill in many cases, but the example gives you an idea of the mechanism’s power. You can use one tracker to track many different extension points and many different objects. You can also use it as your primary data structure by calling getObjects(IExtension) to access all tracked objects for the given extension.

16.7 Services and Extensions

The OSGi service registry and the Equinox Extension Registry are both mechanisms that enable collaboration between bundles in a loosely coupled way. With the advent of DS, both support declarative specification of this collaboration, and both support dynamic behavior. So the obvious question is “Which do I use?” In short, “It depends.”

There are some easy answers. For example, if you are integrating with other components and they already use one approach, consistency is likely a good thing. Some people find one approach simply more appealing than the other. If you can build your system with the level of flexibility and robustness you need, great.

At a purely functional level there are a few characteristics that can help you decide:

Lifecycle—As mentioned previously, the Extension Registry is driven by bundle resolution, whereas the service registry is driven by bundle activation. The former is coarser-grained and somewhat more difficult to control but can be simpler to comprehend and more performant. The latter offers more control but at the cost of more, and more frequent, lifecycle events.

Privacy—An extension point is intended for use by its contributing bundle. As such, extensions contributed to one extension point are not intended for use by other bundles. For example, IPortalAction extensions contributed to the portal’s actions extension point are for use only by the portal. In contrast, the service registry is a global, flat namespace with no formalized partitioning mechanism.

Coupling—Traditionally the IDs attributed to extensions and extension points were automatically prefixed by the Bundle-SymbolicName of the contributing bundle. As such, contributing to a particular extension point implicitly coupled your bundle to the extension point bundle and limited your ability to substitute implementations. This default ID structure is not required by the Extension Registry, however. Extension and extension point contributors are free to define IDs in whatever namespace they want as long as the IDs are globally unique.

Singletons—Given the traditional coupling of contribution ID with Bundle-SymbolicName, bundles contributing to the Extension Registry are typically marked as singletons. Being a singleton means that only one version of a bundle with the associated symbolic name can be resolved at any given time. This is theoretically limiting but pragmatically seldom an issue.

16.7.1 Integrating Services and Extensions

Choosing either services or extensions is relatively easy. More difficult is the case where you need to use services and extensions together. Here there is a mismatch in expectations as both mechanisms vie for control of object discovery and creation lifecycle. Here we outline a few approaches for integrating the use of services and extensions.

16.7.1.1 Integrating Extensions into Service-Based Systems

Since the Extension Registry is driven by the bundle RESOLVED lifecycle and the service registry by the ACTIVE lifecycle, extensions are generally recognized and available before services. As such, integration is more a matter of surfacing extensions somehow as services.

DS enablement—Declarative Services components can be enabled and disabled from code. Using this in conjunction with Extension Registry listeners, it is possible to surface extensions as components, and thus services, by enabling the components whenever related extensions are available. Factory components can also be used in this context. See Chapter 15, “Declarative Services,” for more information on enablement and factory components.

16.7.1.2 Integrating Services into Extension-Based Systems

Integrating the service registry into extension-based systems is relatively easy if the extensions do not consume any services—the executable extension can simply be registered as a service, as we discuss later. If the extension requires some services, however, the lifecycles collide. It is possible to use the factory method to discover required services and inject them into the new object, but what if one or more of the services is not available? Depending on the dynamics of your system, one of the following approaches may work for you:

Executable extension factories—The executable extension factories discussed in Section 16.4.3, “Extension Factories,” can be used to look up and return a service from the service registry rather than instantiating a new object. This approach has the drawback that once a service is handed out to an anonymous Extension Registry user, it is hard to manage further service lifecycle events such as when the service is unregistered. This can be handled using transparent or explicit wrappers but may not be viable in all cases.

Manual service registration—Extension implementations that know about the service registry can manually register services. This approach is taken in the Toast Client application. The application itself is contributed as an extension. Under the covers, our application code registers the implementation as an ICrustDisplay service.

Dynamic extensions—The Extension Registry allows extensions and extension points to be contributed programmatically. This capability can be used from, for example, DS components to register an extension only when its corresponding service is available and to unregister the extension when the service is unregistered.

16.8 Extension Registry Myths

There are a number of misconceptions about the Equinox Extension Registry. While no one tool is good for all purposes, it is worth clarifying the situation somewhat here.

Not pure—Some have argued against the Extension Registry by saying that it is not pure OSGi. It is true that the Extension Registry is not part of the OSGi specification spectrum. That is true, however, of 90 percent of any given OSGi system. OSGi is specifically designed to allow and encourage people to augment systems with more function. The purity discussion seems completely out of place.

Equinox-specific—The Extension Registry sees most of its use in the Equinox context. It has, however, been designed and implemented to be run on any OSGi framework implementation. In fact, it does not actually require OSGi at all and can be run as part of a normal Java application.

Tight coupling—Contributing to an extension point does not couple the contributor to the extension point host. Typically the default name qualification does introduce the host’s symbolic name; however, it is possible for another bundle to supply an extension point with the same name, thus allowing the system to be reconfigured.

Bundle dependencies—A bundle that contributes an extension does not need to specify a dependency on the extension point host. Dependencies are expressed with respect to class loading. Bundles need only depend on packages and bundles supplying required code.

To be clear, we are not arguing for the use of the Extension Registry but rather for the use of the right tool for the job at hand.

16.9 Summary

The Equinox Extension Registry is a powerful, declarative, and lazy mechanism that facilitates collaboration between loosely coupled bundles. It allows for the clear definition of contracts between collaborators and is well supported by tooling that enforces the contracts. The Extension Registry is designed to support highly scalable systems with fast initial loading and incremental caching.

While the Extension Registry is similar in many respects to the OSGi service mechanism, there are semantic and lifecycle differences that make the technologies more or less applicable in different scenarios. There are a number of strategies for integrating the two stories, but none is perfect or universally applicable. In short, both extensions and services are great tools, and developers should choose what best suits their needs and integrate as necessary.