In essence, the NDC fitscape has given rise to several forms of messaging that are based on standardized protocols that are universally applicable in application development. Communication, which represents not only the transmission of information but the receipt and ultimate understanding of it as well, can be viewed in a manner akin to the OSI 7 conceptual model, as shown in Figure 6.1.

Figure 6.1. Conceptualizing communications

The point above cannot be overemphasized: communication represents both transmission and receipt (and understanding) of information. This may seem obvious, but the point needs be made precisely, if only because the theories of information upon which we rely for modern datacom—which tend not to include transmitter and receiver in the same equation—are descriptive of an approach that could be classified as a “transmission model” for communication. Shannon's information theory gives rise to a conduit metaphor for communication: the source translates thoughts into words and sends the words to a receiver, which then translates them into thoughts.

The notion of transmission and receipt, and the underlying metaphor of placing an object in a container and sending it through a conduit of sorts, may be misleading, however. Even after a message is sent, I presumably still have it; the receiver, who removes the object from the container, has in fact received a copy. This model also raises a question it was never intended to address: how does the object get into the container, and get out? More precisely, how do we put meaning into words, and how does someone else extract that same meaning from those words?

This semantic puzzle is as inherent in node-to-node communication as in head-to-head communication. By design, a transmission model does not deal with meaning, but communication represents receipt and understanding of information as well as transmission. Ironically, Shannon's The Mathematical Theory of Communication^[1] therefore leaves the question of communication itself out of the equations, even while the ability to transmit messages is essentially enabled—another paradox of the Network Age.

Shannon's brilliant work provided the means to quantify and facilitate data communication in real-world environments that regularly include noise, signal decay, and erroneous attributes, a foundation upon which higher-level protocols could be built. Given these protocols and the necessary message transmission capabilities, patterns of messaging grew, eventually giving rise to something in the way of genuine communication. Several winners in the messaging fitscape bear examination—patterns of messaging that are useful, common, and universal.

From the genesis of the ARPANET through the modern Internet and wireless, mobile nodes, the one common thread has been message passing. Even during the dramatic growth of the Internet since the mid-1990s, the single most popular slice of the programming pie has been the essential passing of messages. The very notion of interoperability, the battle cry of Web Services models, is based on the simple concept of putting the object in a container so as to make it a message. That, in fact, is the essence of Web Services model: converting human-readable Web pages into messages. Once data is objectified and transformed into messages, the successful transmission of those messages to one or more receivers (which may also include routing at or above the OSI 7 transport layer) can be undertaken.

Message passing is the basis for all data transfer, which includes RPC as well as executable objects. Once a node decides to access or share data that is not in memory but physically within reach (that is, outside a high-speed bus or proprietary data link mechanism), a message must be passed. While there are many interesting and useful variations, that simple concept is the fundamental constituent of NDC. (Formalization of message passing has been one of the key enablers of parallel processing systems as well, though not necessarily those tied to NDC communication protocols.)

It should be clear that the approach provided by Shannon's simple model is not as robust or useful as those that drive datacom today. Indeed, as we've seen in the discussions of protocols, there is much in the way of error correction, stochastic analysis, and protocol-laden work that takes place on behalf of the message, with more to come as new Internet standards emerge. But message passing will remain NDC's essence, our anchor in a sea of NDC change.

The passing of NDC messages can take many forms. Just as the type of data that is transmitted is germane to the operation, so is the form of transfer. Broadcasting a message, for example, is a much different implementation than a one-to-one transfer. And timing naturally affects the nature of messages. A complex set of implementations arises from the simple transmission model, based on three general message considerations:

These three considerations tend to differentiate the various flavors of message passing patterns that are commonly used today.

In general, NDC messages are used in one of two ways, as shown in Figure 6.2: a node sees either a shared memory, the access semantics for which are not significantly different from those used for local memory, or various “cookings”—manipulations and transformations—of the messages themselves; which become the means of creating a shared memory. Within each of these general patterns are further specializations with respect to the message considerations listed above.

Figure 6.2. Message patterns

The reification of messages at each point along the way is essential, from a conceptual perspective, as a function of the cooking points at which message transformation operations occur. In general terms, the flow of messages over time is not bound to a specific or solitary operation or ordering at any of the potential cooking points. Nor is there any specific limit on cooking-point operations. As such, data-type classification or transformation, for example, can theoretically take place at a number of stops along the way, even in a synchronous message passing environment. The same can be said of potential routing operations.

Recall that the Transport level can provide a number of protocol-level options, including levels of message reliability, nonrepudiation, trustworthiness, and security. Operations can be glued atomically and often can be executed asynchronously, that is, if causality is not implied or required, two similar operations can transform an arbitrary message at two completely different stops along the message conduit. The exercise of such choices is both the art of NDC programming and the bane of framework providers who would sugar Deutsch's Fallacies with interface candy.

Paradoxically, XML is the culture in which we now grow framework-independent data, and in which we can implement operations specific to frameworks dependent on document types. The concept of data-type transformation is probably more accurately described as document-type transformation; data-typing on a language level is orthogonally germane to the data transformation that occurs in message passing, and RPC alludes operationally to that relationship. But the transformation of data on a per-document basis is operationally essential to message passing on the Internet and the nodes that feed it. For example, the encryption and decryption of a message requires sophisticated data transformations that must occur in a reliable, trustworthy, ordered manner. That cooking will, by definition, occur at multiple points distributed over time and resources. Some of these transformations may be more reliable than others, and that in turn affects overall application performance.

Regardless of whether a distributed shared memory view is available, interoperable data may require transformation in order to facilitate processing within a given framework. Beyond encryption, other examples of document transformation include Java-to-XML processing, metadata wrappers and containers, and style sheet transformations with approaches like XSLT.

Custom document generators benefit many NDC projects. (The term document in the context of NDC may include various artifacts.) The wide variety of activities in any software development effort leads to the production of documents in many forms. Whether it be a source programming language file, the compiled version of the source, the control files that facilitate compilation, or the messages generated by the running program, the concept of document transformation is essential. From the perspective of developing code alone, document transformation allows a higher degree of automation. The benefits of this approach can include enhanced provisioning, model-driven programming, and a higher degree of consistency driven by systematic document generation enforced at key points.

With ongoing XML extension, standardization efforts, and the growing availability of tools supporting those standards, it has now become possible to generate multiple types of documents. For example, XML document-transformation technology has made it easier to develop custom code generators in application development projects. As such, the interoperability of platforms and frameworks is theoretically enhanced.

Different types of computers handle data in different ways. One of the most common transformations of network data is from the representation used by the application into a form suitable for transmission over a network, and vice versa. Encoding/decoding data for transmission is sometimes called marshalling/unmarshalling. This terminology comes from the RPC world, in which the client node invokes a procedure with a set of arguments that must then be brought together and appropriately ordered (marshalled) to form a network message.

The big-endian and little-endian transformations—named for a tale from Gulliver's Travels, in which two communities were in conflict over which end of an egg to broach for effective consumption—are a good example of the kinds of data transformations that must occur if interoperability among heterogeneous networked nodes is to be possible. The Motorola 680x0 is an example of big-endian architecture, the Intel 80x86 an example of little-endian architecture; internal storage and processing of data is different on the two different architectures. Take, for example the hexidecimal integer value 0x12345678. The Motorola's big-endian architecture sends that value as 12.34.56.78., whereas the Intel's little-endian architecture sends it as 78.56.34.12. Both are “correct,” depending on the assumptions made; as with the communities in Swift's novel, it is a religious matter and neither approach is sacred.

Another reason for marshalling is that application programs are written in different languages, and—even more insidious—there may be more than one compiler for a given language. Since compilers may have a fair amount of latitude in how structures are represented in memory, such as how much padding they put between fields, the simple transmission of a structure from one node to another requires marshalling; even if both nodes are of the same architectural family and the programs are written in the same language, the compiler on the destination node might align the structure's fields differently.

In addition to marshalling and unmarshalling, other forms of message transformation are required in NDC. To the extent that a message consists of an assembly of structured symbols, document transformation is synonymous with message transformation; the absolute structure of a message is arguably a proper subset of all possible document types, depending on document taxonomy. But it is safe to say that a message can enjoin transformation as can a document. As such, document-transformation technologies can be viewed as message filters as much as processing and storage options.

The instantiations of NDC messaging constitute an active, evolving fitscape. For example, from a Java perspective, relevant document-transformation/messaging technologies include those provided by the Java XML Pack from Sun among which are the following:

The Java XML Pack provides an essential bridge between portable behavior (as exemplified by Java objects) and portable data (as encoded by XML). This combination is essential for NDC interoperability going forward, and is one of the cornerstones of next generation NDC development, whether it be Web Services, the Semantic Web, or other organic approaches covered later in this book.

As discussed in previous chapters, NDC message passing employs two basic forms of synchrony: synchronous and asynchronous. An asynchronous message-passing model has traditionally been applied to loosely coupled nodes, generally found in WANs, whereas a synchronous message passing model is more of an idealization in which some amount of timing information about the network is assumed to be known (such as the upper bounds of message delay). Strictly speaking, an asynchronous model is less realistic in modern NDC environments. Synchronous message passing has a long list of implementations and is somewhat easier from a programming standpoint, giving rise to less complexity. But asynchronous message passing may offer advantages that can make the extra implementation effort worthwhile.

POST /RPC2 HTTP/1.0
User-Agent: Frontier/5.1.2 (WinNT)
Host: betty.userland.com
Content-Type: text/xml
Content-length: 181

<?xml version="1.0"?>
<methodCall>
 <methodName>examples.getStateName</methodName>
 <params>
  <param>
   <value><i4>41</i4></value>
   </param>
  </params>
 </methodCall>

HTTP/1.1 200 OK
Connection: close
Content-Length: 158
Content-Type: text/xml
Date: Fri, 17 Jul 1998 19:55:08 GMT
Server: UserLand Frontier/5.1.2-WinNT

<?xml version="1.0"?>
<methodResponse>
 <params>
  <param>
   <value><string>South Dakota</string></value>
   </param>
  </params>
 </methodResponse>

I. SOAP message
   A. SOAP part
      1. SOAP envelope
         a. SOAP header (optional)
         b. SOAP body
   B. Attachment part (content is plain text)
   C. Attachment part (content is an image file)

Routing is one of the most interesting aspects of NDC messaging. Recall from the OSI 7-layer conceptual model that routing is the job of the Network layer. Navigating the dynamic pathways of the modern Internet is still functionally addressed in the lower datacom protocols of OSI 7. Beyond the network layer, routing is what is also meant in the context of message passing.

In the specific case of ebXML TR&P, routing functions beyond those offered by TCP/IP are provided by the ebXML framework (Electronic Business using eXtensible Markup Language), is sponsored by the United Nations (UN/CEFACT) and OASIS, is a modular suite of specifications designed to enable global enterprises of any size in any country to conduct business over the Internet.^[4] OASIS is an international, not-for-profit consortium that designs and develops industry standards specifications for interoperability based on XML.^[5] Routing at the network protocol layer provides the bedrock for message passing by identifying pathways wherein two (or more) nodes can be connected by a virtual circuit. The routing articulated by ebXML TR&P, which layers well above the OSI 7 Network layer, provides a more application-level flavor of routing, designed for more sophisticated NDC applications.

Consider a “Publish and Subscribe” class of applications. OSI 7 Network layer protocols do not generally provide a means wherein messages may be distributed to members of a list of parties, nor is anonymity necessarily an optional mechanism. By extending the routing metaphor above the OSI 7 stack, ebXML TR&P describes a means for message routing that is extensible for such purposes. Based on interoperability sensibilities and existing in tandem with legacy routing technologies, ebXML provides a more application-oriented set of interfaces that allow developers to implement Publish-and-Subscribe applications.

The earlier example of simple message passing with XML-RPC requires less intellectual investment on the part of the developer than does ebXML. But implementing something like Publish and Subscribe using XMP-RPC would be considerably more difficult because the developer would be required to not only provide a dynamic attachment capability à la SOAP but to implement something in the way of asynchronous message passing capabilities as well. (Publish and Subscribe, by definition, requires more than a simple RPC mechanism in order to function.) At its core, ebXML is based on XMP-RPC, as is SOAP, but using only unextended XML-RPC as the basis for NDC programming would be a limiting choice, depending on the type of application being developed. It's the classic “pay me now or pay me later” question; avoiding the initial cost of learning ebXML, for example, in favor of the less challenging option of XML-RPC, may result in a faster prototype, but will likely result in a longer and more difficult development cycle. Goff's axiom applies here—it all depends on the context.

Most often we think of messages as having one sender and one receiver. The bulk of messages do not fall into that category, however, if we include broadcast communication. In the past two decades the number of broadcasters has increased a hundredfold, if cable television (and satellite) options are any measure. While not NDC communications per se, these digital signals nevertheless betray a clear indication of the growth and penetration of NDC-capable technologies and also serve as an example of message transmission: one sender, multiple receivers.

In NDC, broadcast and multicast can provide flexible one-to-many transmission options. Strictly speaking, three types of transmission options are available on the Internet:

Think of broadcast as one amplified voice that everyone in the world hears, whereas multicast is more like a private party, where only those sitting at a particular table hear what is said. Broadcast NDC systems present many unique challenges for network management. These systems generally have both high bandwidth requirements (up to 20 Mbps per channel) and stringent QoS requirements for reliable, guaranteed delivery; some transmissions can be worth millions of dollars an hour. Some proprietary (that is, nonstandard) solutions exist in this space, but generally speaking, a standardized, reliable broadcast mechanism does not yet exist on the Internet.

TCP/IP based multicast has been utilized by some standard application frameworks such as Jini network technology and, to some extent, Project JXTA. For the most part, however, the standards communities have yet to meaningfully address the general problem of broadcast on the Internet.

One useful perspective on any arbitrary network can be gained by examining the nature of its message passing and the opportunities for message transformations, or cooking points, it provides. The Web Services approach to NDC applications is, at its heart, a set of message passing specifications. To the extent that messages are marshalled and serialized, document transformation technologies can also be viewed as message transformation entities.

A transmission model serves as the presumptive mechanism for NDC message passing, implying both a sender and a receiver who share some semblance of shared processing capability. NDC message passing can be very simple or very complex, depending on the needs of the application and the preference of the developer. Regardless of the developer's messaging choice, a standardized, reliable broadcast specification is not currently available for implementation.