Chapter 1. OSGi, Equinox, and Eclipse

As this book goes to press, both OSGi and Eclipse are celebrating their tenth birthdays as Java technologies. Though they were developed independently in completely different domains, their lineage can be traced back to teams in the same organization with a similar need to provide componentized Java solutions. In the case of OSGi it was home gateways and set-top boxes. For Eclipse, that need was in the tooling space. Both, however, had very similar needs for modularity and extensibility.

In this first chapter of a book dedicated to OSGi and Equinox, we look at some of the history behind these technologies, how they are used, what they are good for, and what they can do for you.

1.1 A Bit of History

For the first few years, OSGi and Eclipse technologies grew up in parallel with only a few passing encounters. The OSGi organization was a loose consortium of embedded and home gateway vendors. Its modular runtime specifications evolved quickly with several major revisions, new services, and expert groups in different domains, particularly vehicle software. Adoption ramped up, and more and more framework implementations appeared. The OSGi community maintained its focus on the embedded market, and those needs continued to be reflected in concise and honed APIs and design.

At the same time, Eclipse was a loose consortium of software tool vendors looking to create a comprehensive tooling platform. Its technology was maturing and quickly dominating the tooling market. Eclipse-based offerings with thousands of components were shipping as flagship products from major software companies. Modularity and the open community, two key ingredients, were working as a powerful pair to drive a revolution in the tooling world. Eclipse also began to reach beyond tools and into rich client applications. This shift drove the need for a more robust modularity mechanism, support for dynamic behavior, and, moreover, standardization.

In 2003 the Equinox project was created at Eclipse. The initial goal was to address the runtime-related issues seen in the Eclipse of the day—static behavior, a nonstandard markup and execution model, and, as a result, the inability to leverage the work of others in areas such as provisioning and management. OSGi was not the only contender.

After a public survey of the available technologies (e.g., Avalon, JMX, and, of course, OSGi), OSGi was identified as the most promising approach—its clear component model and strong execution specification were seen as great assets. The potential for leveraging the standardization and creating an even broader community was clearly evident.

Having decided on OSGi, the team had to get, write, borrow, or co-opt an open-source implementation. At the time there were relatively few choices: the open-source Oscar project at ObjectWeb—now the Felix project at Apache—and IBM’s Service Management Framework (SMF), a shipping commercial framework. Knopflerfish, now an open-source implementation, was not yet open-source and was unknown to the team.

Oscar had lots of great characteristics, not the least of which was its internal simplicity. SMF had the benefit of being an industrial-strength implementation that had been in production for some years and had the backing of a team of developers. In the end SMF was selected as the starting point. IBM donated the code to Equinox, and the transformation began in earnest.

Marrying the two mind-sets and approaches was not easy, but working closely with the OSGi Core Platform Expert Group, the Equinox team helped evolve a number of changes and additions to the OSGi framework specification to cover the new use cases. Lazy activation, bundle fragments, bundle name and version semantics, and bundle dependencies are all fruits of this very successful and productive collaboration. Within six months the original Eclipse runtime was seamlessly replaced with the new Equinox OSGi implementation. From there Equinox evolved to be the reference implementation for the newly minted OSGi R4 framework specification, and the future of both communities changed forever.

1.2 Collaboration

In the years since then, one thing has become clear: The benefits of collaboration are immense. It is no overstatement that Eclipse’s adoption of OSGi put the technology on the map. Google hits on “OSGi” increased by two orders of magnitude in the time around Eclipse’s adoption of OSGi. Most of the hard work was done before Eclipse came along, but with a few additions and some implicit marketing, OSGi was in front of millions of software developers in a new context.

The benefits flow both ways. Equinox is seeing a surge in its use in runtime scenarios from servers to embedded. In that context the shoe is now on the other foot. All of the major Java application server vendors are adopting OSGi. Many of those are using Equinox. Suddenly OSGi is the technology to watch, and Equinox is one of the key implementations.

On a more technical level we have seen a massive transition in the way software systems are put together. Eclipse really showed that componentized software ecosystems work—it changed the tooling landscape. OSGi, Equinox, Felix, and associated projects are now fueling similar transformations in adjacent spaces in the runtime world.

The software industry has a long history of producing component models for various domains—CORBA, SOM, SOA, SCA, to name just a few. Some have gained a persistent following in particular computing categories, but few have a following that spans the computing landscape. What is it about OSGi that makes it interesting in such a wide range of computing environments and application domains?

Perhaps the key factor is that it started in a constrained environment with some hard requirements. The original focus of the OSGi effort can be seen in the organization’s original name—the Open Services Gateway initiative. The initial work was done in the late 1990s to facilitate smart homes, home gateways, set-top boxes, and the like. The requirements of that domain forced an ethic of modularity, compactness, and dynamism on the API design.

While the home gateway industry may yet take off, it did not do so as hoped a decade ago. The core design points have, however, come to be appreciated and are applicable in almost every computing domain.

Modularity, it turns out, is the lubricant of collaboration.

1.3 Modularity and Freedom of Action

Like it or not, the world needs boundaries. The saying “Good fences make good neighbors” has variations in many languages with good reason. Boundaries set expectations and form the basis of contracts between the parties. Defining these boundaries in a formal way enables development-time and runtime monitoring and enforcement of the boundaries.

OSGi has robust boundary mechanisms. Through a series of mechanisms, software components define themselves, what they offer to others, and what they need of others. A component must clearly state a dependency on what some other component is offering before collaboration can happen.

This directly supports implementation and information hiding. Hiding is great because it limits the assumptions you can make and forces the decoupling of system elements. With no knowledge to the contrary, you have to assume anything—or nothing—can happen. With hiding you can do whatever you like in your own yard.

Even the most robust fences have gates. Whereas a fence stops people from traversing the boundary, a gate declares that there is a boundary but allows people to pass, understanding that there are new rights and obligations.

With a formal definition of the fences and gates, components are decoupled and freedom of action is increased. But it is not just the components that are liberated. The development teams, system integrators, and product designers have newfound independence and power. This manifests itself in a number of ways:

Concrete abstraction—Modules add another granularity of abstraction. Just as subroutine calls, separate source files, and classes enable functional abstraction, modules raise the level of abstraction and enable reuse and sharing.

Granular equivalence—Modules add a layer of insulation that makes it easier to substitute equivalent components. The idea of implementation substitution is not new—interfaces, mock objects, and polymorphism are in the same vein. The clear API definition inherent in modularity cranks this up a notch and facilitates the substitution of whole components. This in turn frees teams to act independently, removes bottlenecks, and reduces risk in the software lifecycle. Teams are largely decoupled and can evolve their components independently.

The death of the wad—API and packaging boundaries drive developers to stay in bounds. Similarly, because the API surface area of their bundle is explicit, developers think harder about the contracts they are making with others. This in turn discourages the creation of amorphous wads of tightly coupled code.

Getting there fast—Flexibility and time to market are cool buzzwords, but they are also real assets. Developing business logic and then deploying in a stand-alone server, rich clients, or integrated with existing web infrastructure is a tangible benefit, especially if little change is required. This cuts both ways, too. It’s easier to describe and deploy different functional sets on the same platform.

1.4 Platforms

Modularity brings freedom, but it also enables platforms. The notion of platform is a well-known phenomenon in many domains outside computers. Cars, for example, are very expensive to develop. Manufacturers have long since updated their businesses to develop platforms and then outfit them separately. Initially it was just trim quality and tuning levels—Pontiac and Buick, Ford and Mercury, Honda and Acura. These different product lines shared common designs, components, and assembly-line tooling. With the diversification of the auto market, platforms now extend across “markets”—Accord sedan and Odyssey minivan, Taurus car and Flex crossover—same platform, different market.

One way to think of platforms is as a specialization on the underlying modules. Platforms aggregate a set of components and configure them in a way that is ideal for a particular domain or market. They address the parts of the solution that are largely commoditized. Other teams then build real added value on top of the platform.

Platforms allow companies to move quickly to develop new offerings because they don’t have to develop each from the ground up. End users benefit as well since the underlying technology is generally more widely used and better tested. These effects are magnified as platforms evolve with new capabilities and characteristics and communities of extenders emerge.

1.5 Ecosystems

Popular platforms develop ecosystems—communities of consumers and contributors—that work with a platform and apply it in more specific ways. In the vehicle example the immediate platform ecosystem is internal to the manufacturer and associated companies. Aftermarket parts vendors benefit to a certain degree but have less influence.

With heavy trucks, however, the ecosystem is much closer to the platform. Heavy truck makers produce a modest number of different chassis and drivetrain combinations. In many cases, these roll off the assembly line in forms that are not roadworthy—they lack taillights and other safety equipment. The trucks go to dealers and customers who then customize the platform as a dump truck, goods hauler, tow truck, and so on. In this case the ecosystem is more open and extensive.

Open ecosystems drive innovation. Because everyone has access to the common platform, offerings stand on the merits of their added value. Ecosystem members can compete or collaborate, or both—“co-opetition.” End users can choose how they interact with and consume ecosystems’ output.

1.6 OSGi in Context

What does all this really have to do with OSGi and you? In short, OSGi enables these effects in the software world. We have seen Eclipse evolve as a tooling platform that dominates the overall development environment market. In Section 1.7.1, “NASA’s Maestro and Ensemble,” we look at how the NASA Ensemble project builds on top of Eclipse in the rich client and server space to create a space mission software platform. As of this writing, all major Java application server vendors have adopted OSGi, and in many cases Equinox, as the basis for their flagship offerings.

All of this is possible because of the deep modularity promoted, supported, and enforced by OSGi.

1.6.1 The Java Lie

One of the early slogans of the Java community was “Write once. Run anywhere.” In a certain context that is true, but unfortunately the Java world has been divided in three by Java ME, SE, and EE. These address different computing environments and promote different modularity and execution models. While it is true that the Java code itself likely works everywhere, the overall system does not.

OSGi, on the other hand, makes very few assumptions about the surrounding environment or the context in which it is used. This enables its use in a wide range of scenarios with one consistent programming model. Components written for the server can be run on the client and vice versa. Moreover, the tooling and developer skills are the same across all execution environments. We will see how this works later in the book.

1.6.2 Reality Check

If all this sounds too good to be true, it is. Well, sort of. There is, as they say, no free lunch. Modularity is not easy. It forces you to express your expectations, add rigor, maintain additional metadata, and negotiate contracts. It is simple human nature that every time you draw a box there ensues a discussion about what is in the box and what is not. This is not entirely unhealthy. Most of these discussions result in a deeper understanding of the problem, a more coherent team view of the system, and, indeed, better software.

As always, it is possible to do something bad with something good. Your first, second, and third component sets will be tightly coupled, brittle, and inconveniently granular. You will struggle to identify the players, their roles, and how they interact. These are not problems of OSGi or even modularity—these are design challenges present in any system. OSGi, through its modularity requirements, both forces you and enables you to think about these problems in concrete and tangible terms.

We have encountered a number of teams that, in their adoption of OSGi, have said, “OSGi is hard” or “OSGi does not work for us because <insert some reason why software design best practices do not apply to them>.” This is shooting the messenger. Certainly some OSGi facilities could be better, and no one tool is appropriate in all situations. However, if you view OSGi as a framework for describing the elements of your system, it actually does not even matter if you run OSGi.

One example of this is the Apache Harmony project. This is an effort to create an open-source Java SE 5 implementation. The Java class libraries are a large and complicated software system with many intertwined pieces. To enable multiple independent teams to work on the project, the Harmony team turned to OSGi markup and tooling. The library was broken up into a number of OSGi components with clear API boundaries. OSGi tooling was used to enforce the boundaries at development time. In this context it does not matter that at runtime OSGi is not present.

1.6.3 OSGi Longevity

The OSGi specifications are produced by the OSGi Alliance, an industry consortium of mostly software and embedded systems vendors. It operates as a series of Expert Groups (EGs) focused on particular computing domains: Core, Enterprise, Vehicle, and Mobile. While participation in the specification process is limited to members, the specifications produced are freely implementable. Specification compliance certification is a managed, for-fee service.

Overall the organization has a robust governance structure and a relatively rigorous specification process. The specifications produced are complete and very readable. Each specification has an accompanying reference implementation (RI). In many cases the RI is open-source. For example, the core framework RI is Equinox. In other cases the RI is proprietary to the alliance.

In addition to their own self-proclaimed standards, the OSGi Alliance has been the driver behind Java Community Process standards such as JSRs 232 and 291—OSGi in Java ME and Java SE, respectively.

The current core specifications are widely implemented in a range of open-source and commercial offerings. Adoption is strong and growing. Some new specifications coming out of the Enterprise EG show great promise but are only just released.

The main challenge for OSGi as a specifications body is to maintain relevance as the computing world shifts from Java to so-called dynamic languages. Currently OSGi is firmly fixed as a Java-based technology. Of course the concepts and approaches translate to other languages, and modularity is sorely lacking in those environments—there is ample room for OSGi value to be added; it just needs to be done.

1.7 OSGi and Equinox in Practice

The canonical example of widespread OSGi and Equinox use is the Eclipse Integrated Development Environment (IDE). It is used by millions of software developers around the world. There are thousands of developers writing bundles to extend the platform and hundreds of companies shipping products based on these capabilities. It has revolutionized the tooling market largely on the back of the underlying modularity infrastructure.

Nonetheless, people still do not relate to Eclipse as an example of OSGi’s power because they are not writing large, tooling-related systems. They are writing small to medium end-user or server-side systems. Eclipse, the development environment, is a large and complicated platform that has nothing to offer them. They can, however, still benefit massively from the effects described in this chapter—the same effects that drove the success of Eclipse. Here we look at some uses of these technologies in other domains.

1.7.1 NASA’s Maestro and Ensemble

One of the more appealing uses of OSGi is in NASA’s Maestro and Ensemble projects. Some years ago NASA adopted the Eclipse Rich Client Platform (RCP) as the base for its Maestro space mission software platform. They are using it today to manage the Spirit and Opportunity missions on Mars. Their experience with Eclipse, OSGi, and, moreover, modular software development was so positive that Maestro spawned Ensemble.

Ensemble is a broader effort involving many NASA teams in the creation and use of a space mission software platform. Each mission is made up of many different elements, and each element has its own set of software for planning and executing the required steps. While the domain and content of this software may be quite varied, there is a common set of functions, UIs, and other facilities that are useful across the board. Ensemble captures these commodity components and makes them available to mission software teams.

In the evolution of NASA’s platform there has come to be a rather large element of server-side work. The team has expanded to use Equinox on the server and developed their own RESTlet infrastructure to address scaling and flexibility issues. In their EclipseCon 2008 tutorial on the topic they summarize the approach as follows:

Eclipse RCP + Server-side Equinox = “Tierless Computing”

• Develop many bundles independent of deployment environment.

• Share some capabilities between server and client.

• Freely migrate capabilities back and forth as needed.

• Use a consistent development environment (Eclipse) and component model (OSGi) throughout.

• Debug clients and servers simultaneously!

These directly mirror the benefits and effects discussed earlier in this chapter. The NASA team is living the OSGi effect.

1.8 Summary

OSGi is a powerful, robust modularity framework and runtime. Equinox is a robust, scalable, and comprehensive implementation of the key OSGi specifications. Both the specifications and the implementations are open and driven by multi-vendor organizations with a long track record of delivering innovation.

The value that these technologies deliver enables individual developers, teams, and organizations to change the way they develop and deliver software. This grand claim has been substantiated by numerous examples in open-source and commercial software settings, including the NASA team building Maestro and Ensemble.

As this pair evolves, OSGi coming up from the embedded world and Equinox coming down from the tooling domain, they are spreading into the more traditional server runtime space where Java application server vendors are adopting the approach with gusto. No wonder analysts are calling OSGi one of the hottest technologies to watch.

Throughout the rest of this book we walk you through the details of OSGi, real-world examples of its benefits, and best practices for leveraging these in your projects.