Emerging standard NDC frameworks, bearing many similarities in design and service level, enable a new generation of applications. The degree of interoperability provided by these frameworks both supports and limits the evolution of application development. As in any evolving fitscape, novelty is persistently explored; optional NDC frameworks providing interesting fitscape survival characteristics await discovery.
Vanilla Web Services (those represented by the WUS family of interfaces—WSDL, UDDI, and SOAP) provide an extensible NDC framework for application development. XML ostensibly provides a degree of baked-in interoperability. But without additional layers of definition, too much is left to vendor discretion, and interoperability is not ensured on the application level.
Microsoft has offered BizTalk, a product it markets as a solution to this problem, designed to address concerns specific to b2b transactions on platforms built on top of WUS. A community effort has produced ebXML, which also supports b2b by extending XML specifications over SOAP but which is not specific to any operating system platform. Such efforts in the b2b space are mirrored by similar efforts in application-specific fields such as biotechnology, which has seen several efforts to develop standard extensions in order to facilitate exchange of informatics data. Some of those efforts have been fruitful, while others have found insufficient traction with respect to community support and have since faded. This sort of disappointment points to Reed's law, another important force in the greater NDC fitscape.
In 1999, Dr. David P. Reed, an independent entrepreneur, advisor, and consultant, proffered “Reed's law,”[1] which speaks to community formation, potential value, and Metcalfe's law. The essence of his discovery is that while many kinds of value may be created within networks, there exist certain network structures that create value more quickly than others. Specifically, networks that support the construction of communicating groups create value that can scale exponentially with network size, which is much more rapidly than Metcalfe's law would suggest. Reed calls such networks group-forming networks (GFN).
Metcalfe's law is a mathematical truth: the number of possible links between nodes in a network is a function of the square of the nodes. From an economic perspective, it addresses potential value only. Reed's law not only clarifies another exponential of potential value—The number of nontrivial subsets that can be formed from a set of N members is 2N − N − 1, which grows as 2N)—but points to an NDC application strategy as well.
A GFN approach to applications, application development, developer communities, and business strategies seems plausible in the Network Age and is also very likely an evolutionary mechanism by which optional frameworks will emerge. Consider eBay.com, the definitive example of a GFN portal. Various instantiations of the auction fit well with GFN characteristics, which include such attributes as highly dynamic, scalable, structured, and purposeful. Given the increasing resource base available on the Internet, group formation will trump even peer transaction capabilities as the NDC fitscape evolves. And NDC frameworks that understand and promote group formation will be the means by which it occurs.
To the extent that Jini network technology facilitates group formation and communication, Reed's GFN concepts can be realized in code. Over time, resources on the network increase exponentially. Moore's law spurs processor capabilities and storage capacities on the node level, at the same time decreasing the threshold for node deployment; tiny, embedded processors abound. Gilder's law ensures more and faster links among nodes of all kinds, with or without tether. The flow of research and development in the greater NDC fitscape constitutes a feedback system that itself is subject to and benefits from the results; phase shifts should be expected as autonomous agents innovate. And groups are the means by which this innovation occurs.
Jini network technology is the mind child of Bill Joy, formerly Chief Scientist for Sun Microsystems. Joy, inspired by David Gelernter's Mirror Worlds (discussed in Chapter 3), based Jini on the Java programming language and Remote Method Invocation (RMI), the means whereby one Java Virtual Machine (JVM) exchanges marshalled data with another. The Jini approach was as innovative as it was misunderstood when it was announced in the summer of 1998. The initial announcement heralded Jini network technology as a competitor to Microsoft's then universal Plug and Play, targeting small devices that would connect to a network. The announcement alluded to services that could be built with Jini network technology, turning a laptop into a supercomputer. But there was a catch: there were then no tiny-device JVMs that supported RMI. Alas, a wild commercial uptake of Jini was not to be forthcoming . . . at least not then.
Mispositioning aside, the fundamentals of Jini network technology still allow developers to view resources on the network in a different way. A means whereby NDC application developers can meaningfully address network realities—the ever-present potential of a partial network failure and the lack of a central authority on the network—will always be of interest. Despite commercial challenges, the idea of a self-healing network, imaginable in the Jini environment, has given rise to a flurry of research into the more organic approaches to NDC frameworks, spurred by the Jini community.
A simple set of protocols provide the basis for Jini network technology:
Discovery protocols: Three closely related discovery protocols are used by a device joining a network: one discovers one or more lookup services on a LAN, the second announces the presence of a lookup service on a local network, and the third establishes communications with a specific lookup service over a WAN.
Join protocol: The join protocol makes use of the discovery protocols to provide a standard sequence of steps that services should perform when they are starting up and registering with a lookup service.
Lookup server (LUS): Nodes that would participate in a distributed Jini network technology system (known as djinn) must first obtain references to one or more Jini lookup services, found on one or more LUSs. The discovery protocols govern acquisition of these references. Once the references have been obtained, a number of steps must be taken for entities to start communicating usefully with services in a djinn; these steps are described by the join protocol.
On its maiden voyage into a network, a Jini enabled device utilizes a UDP multicast announcment, a self-referential packet proclaiming its nature and needs. Communication between the device and the LUS is facilitated between JVMs by RMI.[2] Note the underlying assumption that the Java platform is ubiquitous. Any device that would join and consume from or provide a service in a Jini environment must be bound to the Java platform. Note the second assumption: an LUS is listening on the network, awaiting announcements from Jini enabled devices.
“Joining a network” is a state in which many components may find themselves in various deployed instantiations. Indeed, we can envision an LUS beginning from just such a state. Once a device successfully joins a Jini enabled network through an LUS, a unicast approach can be taken, as shown in Figure 9.1.
Another interesting aspect of RMI is its flexibility in playing on the OSI 7 stack, either through HTTP (which sits above TCP) or at the transport layer itself, through UDP, TCP, or other protocols. This dimension of flexibility is not natively present in either CORBA or Web Services.
Between market mispositioning and the inevitable objection to the assumption of ubiquitous JVMs, it's no wonder a wild commercial uptake of Jini network technology was not forthcoming. The JVM assumption can be engineered around by proxies and the Jini Surrogate architecture, which identifies classes of non-JVM devices that can be specified from a communications perspective and therefore proxied. But even while the “Java everywhere” assumption fully addresses Deutsch's Eighth Fallacy (The network is homogenous), it may ignore other realities, Deutsch's and otherwise.
A simple relationship emerges from the Jini protocols and the assumption of an LUS on the network. The essence of the relationship, as shown in Figure 9.2, should look familiar.
Note the structural similarities with the Client-Registry-Service conceptual model; they tell us that the LUS should behave very much like a registry. An important and oft-cited difference between CORBA and Jini network technology is in the representation of objects: With CORBA, a string is used to identify object attributes, whereas on the LUS, objects are represented as extensible Java objects themselves.
In addition to the protocols, a Jini download includes a number of useful programming interfaces, which complete a GFN-capable framework:
Distributed leasing: Leases are grants of a resource or service that are negotiated for a set period of time. Leases can be renewed, cancelled, or allowed to expire, and resources are allocated only so long as appropriate leases are renewed.
Distributed events: An event is the reaction of an object to a state change in another object. On an arbitrary network, timely and reliable delivery of events cannot be guaranteed, according to Deutsch's First Fallacy (The network is reliable). Jini distributed events are based on an observer pattern for distributed systems.
Distributed transactions: For atomic operations, Jini offers a minimal set of interfaces that specifies a two-phase commit protocol and that also provides for nested transactions.
Based on these protocols and programming interfaces, interesting services can be envisioned, exemplified by JavaSpaces, the aboriginal Gelernter-inspired Jini service. Given the recognition of network realities at the interface level, the spaces computing approach programmatically lends itself well to Reed's GFN concept. The leasing interfaces are most instrumental in that regard, providing a conceptual basis for transactions as well as a means to facilitate network self-healing; a framework for dynamic, arbitrary group formation is the result.
“Jini or something like Jini” became the evangelical mantra at Sun. Once the properties that can emerge on a network so configured became clear, a community of interest also emerged, one with sufficient traction to support, refine, extend, and employ the technology. Many developers worldwide have been or are active in the Jini Community. At the same time, further research at Sun coalesced around exploration of “something like Jini,” ultimately leading to Project JXTA and the peer-to-peer (p2p) approach to GFNs, an approach that also boasts an active supporting community of thousands of developers. GFN concepts play at the framework-molding layer even as they provide a model ripe for commercial exploitation. With the fundamental strength and weakness of the assumption that JVMs are ubiquitous in the Jini network, the framework offers a reliability and penchant for GFNs that cannot be matched by any preceding NDC framework, including vanilla Web Services and CORBA.
Strictly speaking, both CORBA and Web Services provide a framework in which some p2p patterns can emerge. But GFN-style applications such as file sharing, instant messaging, collaborative frameworks, and even spaces computing models are simply easier to encode in what is fundamentally a highly interoperable p2p environment. Content delivery networks that use p2p protocols to push value to the network edge will find an increasingly wide array of devices to target, and distributed resource-sharing projects, as exemplified by SETI@Home, are also easily enabled. Imagine the possibilities for p2p in fields such as gaming, ecommerce, grid computing, and even wireless devices. Commercially successful p2p frameworks still await discovery, however, in the wake of the much maligned Napster.
Along with Napster came a realization that in a network that is sufficiently mature with respect to the Nth Laws, interesting properties gleaned with p2p approaches may have significant impact on a number of assumptions. Intellectual property rights notwithstanding, the more popular an item in a p2p network becomes, the easier it is to obtain. It is also true that a p2p network is naturally suited for dynamic group formation. On such a network, a simple p2p model, as shown in Figure 9.3, provides one of the least centralized, most flexible frameworks yet considered, and perhaps one of the most difficult to manage as well.
Another deflection-point-embracing technology closely associated with Bill Joy is Project JXTA. From the perspective of Reed's law, Project JXTA is without peer in the fitscape of frameworks that would compete for NDC dominance. While GFNs are at the heart and soul of JXTA, managing the p2p networks and applications that emerge is another matter.
Where Jini assumes JVMs everywhere, Project JXTA assumes nothing anywhere. A simple set of protocols, bound only to advertisements written in XML, form the basis for the JXTA p2p framework. The following five concepts are key to JXTA nomenclature:
Advertisements—. XML documents (like WSDL) that can describe any resource, node, or service
Peers—. Nodes on a JXTA network that implement JXTA peer protocols
Peer groups—. Dynamic groupings of peers that are not tightly specified and could be a group of services or nodes, a geographically similar grouping, and so on
Messages—. Communications in the datagram style (UDP) for use on unreliable, asynchronous or unidirectional transports (like IP)
Pipes—. Connection of peers in one-to-one, one-to-many, and many-to-many relationships for message passing
Pipe binding—. Runtime binding that allows rebinding if errors occur
Project JXTA defines scores of communication protocols that are independent of each other. That is, if an arbitrary application uses only one of the protocols defined to accomplish its aim, it may still find some homage in a JXTA network as shown in Figure 9.4.
Project JXTA is open source, functionally equivalent to the Apache source license.[4] JXTA defines an application “stack” for p2p communication, which provides much of the underlying interoperable piping. While JXTA was originated by Sun Microsystems (and does begin with a J) it is not based on the Java platform, nor does it require a JVM to run. Although the reference implementation does target J2SE, some JXTA developers have implemented JXTA protocols using C, Python, and Perl as well. Advertisements in XML are specified, ensuring high interoperability.
In p2p networks, peers tend to self-organize into peer groups. A peer group is a collection of nodes that implement the peer protocols and that may have a common set of interests. Each peer group is identified by a unique PeerGroup ID. The JXTA protocols do not mandate the circumstances for peer group creation but only specify how peers may publish, discover, join, and monitor peer groups.
A peer group can provide a set of services. JXTA specifies a core set, called peer group services, the on-the-wire data encoding for which it describes. But because of the community approach taken by the Project JXTA team, a much larger engineering group is involved in the effort, easily allowing additional peer group services to be developed, which add the following to the core set of peer group services:
Discovery service: Used by peer members to search for peer group resources (peers, peer groups, pipes, and services)
Membership service: Used by peer members to reject or accept a new group membership application
Access service: Used to validate requests made by one peer to another
Pipe service: Used to manage and create pipe connections between the different peer group members
Resolver service: Used to address queries to services running on peers in the group and collect responses
Monitoring service: Used to allow one peer to monitor other members of the same peer group
Not all the above services, or the JXTA protocols, must be implemented by a JXTA peer group. By the same token, other services can be envisioned and easily join the JXTA-enabled p2p network; something like the Jini LUS comes to mind, or p2p spaces computing, which too is being explored by a small community worldwide.
Recall that with Jini enabled networks, a device can know nothing about any identified repository and still manage to join a network. With JXTA, at the very least, the identity of something that exercises at least the peer protocol must be known. Once bootstrapped, however, either multicast may be used, usually on a LAN, or unicast to a repository of peers. In JXTA, such repositories are called “rendezvous servers”; these can cache data from and about other peers and offer peer advertisements and proxies and are structured much like a directory from the Client-Registry-Service conceptual model.
Searching for services in a JXTA-enabled network begins with a query of first order peer(s), including known rendezvous servers, a step that begins the exploration process for the network resources in question. First-order peers are those whose existence is known, who are operating in compliance with the JXTA peer protocol, and who are assumed to be available. This constitutes a peer horizon from the perspective of the node initiating the query. Rendezvous servers may ask other rendezvous servers within their horizon if they know of the service, a query that may repeat from server to server, increasing the scope of the search horizon exponentially with each layer, as shown in Figure 9.5. Because of cascading rendezvous requests, a peer must wait some reasonable amount of time before repeating a search. But the power and potential of a p2p network should be apparent.
One of the interesting features of Project JXTA is where on the OSI 7 stack JXTA pipes can play. With greater flexibility than any previously considered framework, p2p application patterns may consider exploitation of potential at each layer. While the ability of an inappropriately encoded p2p application to ignore an arbitrary firewall may at first give developers pause, the opportunity to bake in security at the network or transport layer is also exposed, even while a free ride on HTTP is ensured.
There are a number of motivations for taking a spaces computing approach to NDC application development:
Simplicity: For the most part, steep learning curves are not required. A handful of simple operations provides access to the typical tuplespace.
Expressiveness: Using a small set of operations, a large class of NDC applications can be produced without a lot of code written.
Loosely coupled protocols: A tuplespace must uncouple senders and receivers, to facilitate the composition of large applications. Components can be easily added without redesign of the entire application.
In discussing spaces computing, it is useful to attempt to define the term organic software. In truth, it is not meant to be a candidate for specification as much as it is an allusion to attributes that an NDC framework should exhibit if it is to be vital, in light of Deutsch's Eight Fallacies. A list of such attributes, which apply to any NDC framework, includes the following:
Dynamic service and peer discovery
Self-healing network attributes
Distributed transaction capability
Workflow automation and integration capability
Agent collaboration
Any NDC framework that demonstrates these organic attributes can be tuplespace-capable. Conceptually, applications launch objects to and retrieve them from a space, as shown in Figure 9.6.
JavaSpaces, like other spaces computing approaches, provides a simple mechanism for dynamic communication, coordination, and sharing of objects between JVMs. In an NDC application, a space acts as a virtual space between providers and requesters of network resources or objects. This allows applications participating in a spaces computing solution to exchange tasks, requests, and information. Object persistence is also standardized with JavaSpaces.
JavaSpaces assumes that users are all linked to a JVM. Transport between differenct languages can be implemented through the Java Native Interface (JNI) specification, and if a node is not linked to a JVM, programming links can be made through RMI/IIOP or CORBA transport systems. Although there are conceptually similarities to a database on one level, there are many differences between a spaces computing persistent object and one logged in a database. To be useful, databases require a tremendous amount of overhead in structured programming, interface design, type matching, and indexing. A database client must send specific queries to the server and receives specific answers to those queries. A spaces computing approach is a much more loosely coupled repository of data. The identity of a client or a server in a space is not relevant; data packets are treated just like any other object posted to the space—as an anonymous service.
With JavaSpaces, a system can be thought of as an event-driven distributed cache with behavior transfer capabilities—something not available with CORBA—with functionality well beyond that of a simple messaging system. An application running on a JavaSpaces implementation typically runs in the middle tier of a three tier client-server model (although it can run anywhere).
The essence of spaces computing is the tuplespace (from Gelernter's original Linda sytem): a multiset containing tuples, which can be thought of as objects with exposed attributes. A multiset means that a tuplespace can contain an arbitrary number of identical tuples, which are treated as discrete objects. Some tuplespace implementations provide the ability to nest tuplespaces. The tuplespace can be implemented in a number of ways; both Jini network technology and Project JXTA have framework proponents of a spaces computing implementation for NDC applications.
Most tuplespace systems assume one or more spaces running on one or more nodes; the concept of distributing a single tuplespace over multiple nodes is less common. JavaSpaces supports multiple local and remote shared tuplespaces, but does not explicitly support distributed tuplespaces. IBM's TSpaces has similar limitations.
True distributed tuplespaces offer many advantages in terms of scalability and availability—attributes that would ideally be available in all spaces computing implementations. The Projext JXTA community is pioneering jxtaSpaces,[5] which is intended to provide scalable shared memory services for networks of heterogeneous peers. In large p2p networks, a global tuplespace can be used in many ways to simplify the building of complex NDC applications.
From the perspective of GFNs, a spaces computing approach to NDC application development provides a simple, extensible, organic means of achieving results. To the extent that the attributes of organic software become more important over time, spurred by Nth Law-driven growth and the proliferation of networks of all kinds, spaces computing represents a tractable choice that will bring medium-term competitive advantage to its adherents. Indeed, as sedimentation and consolidation continue, in time a tuplespace too will become a commodity. At the moment, however, Gelernter's vision remains a largely untapped commercial resource.