B.2.1 What Is the W3C?

The World Wide Web Consortium (W3C) is a “public consortium” in the sense that it is open to membership by anybody with the resources (financial and otherwise) to participate. At press time, there were over 350 member organizations. According to the W3C’s website,⁴ the mission of the consortium is “to lead the World Wide Web to its full potential by developing protocols and guidelines that ensure long-term growth for the Web.”

In practice, this has meant publishing specifications based on HTML and XML technologies. Among the W3C’s success stories are several of the publications discussed in this book. Some of the principal specifications published by the W3C are:

The W3C publications most relevant to this book, of course, include the entire XPath 2.0 and XQuery 1.0 suite of documents (including the Data Model, Functions and Operators, Formal Semantics, Serialization, and XQueryX), XSLT 1.0 and XSLT 2.0, XPath 1.0, Infoset, and XML Schema (parts 1 and 2).

The W3C was founded by Tim Berners-Lee, who invented the World Wide Web, along with the first web browser, the first web server, HTML, and the HTTP (HyperText Transfer Protocol), while working at CERN (European Organization for Nuclear Research). Sir Tim, as he is sometimes known, remains the Director of the W3Ctoday and takes a very active part in establishing new work and approving the progression of completed work. Berners-Lee spends much of his time working on technology related to the Semantic Web, an effort to add meaning to the web itself to make it more useable and useful to people and to automatic systems.

The W3C, founded in 1994, has published nearly a hundred Recommendations, as its completed specifications are known, ranging from protocols such as HTTP and SOAP, through markup languages such as XML and PNG (Portable Network Graphics), through higher-level specifications such as XML Schema and XQuery. One of the consortium’s most important goals is to make the web accessible to all people, regardless of the language they speak and read or whatever disabilities they might have. Consequently, the Internationalization and Accessibility activities are given significant authority to review and propose changes to the content of specifications before they are advanced to Recommendation status.

B.2.2 The W3C Process Document

The W3C is governed primarily by its Process Document,⁵ which “describes the organizational structure of the W3C and the processes related to the responsibilities and functions they exercise to enable W3C to accomplish its mission.” Among other things, the Process Document sets up the structure of the consortium, including the rights and responsibilities of its members as well as the existence and duties of:

The process document also spells out how W3C Activities (the formal term includes very broad topics, such as “XML,” “Internationalization,” and “Multimodal Interaction”) are created and managed, how Working Groups are established, chartered, managed, and terminated, and how coordination between Working Groups within an Activity is maintained.

Perhaps most importantly, the document establishes the criteria by which Recommendation Track documents are initiated and progressed. We discuss this further in Section B.2.3.

B.2.3 The W3C Stages of Progression

In the context of the W3C, there are two broad types of document to consider. One of these, the Note, has very little official standing and correspondingly has very little process associated with it. A Note can be developed by a Working Group (a Working Group Note), by an Interest Group (Interest Group note), or by a Coordinating Group (Coordinating Group Note). Notes sometimes identify a best practice, and they sometimes describe a specific technology or technique that, while interesting to the web community, doesn’t have enough support to become a more formal Recommendation.

The other type of document is described as “Recommendation Track.” It is this kind that receives significant attention from the Process Document and from the W3C administration.

A Recommendation Track (REC-track) document is one intended to be published eventually as a W3C Recommendation. Most REC-track documents are eventually published as Recommendations, but not all. Some stall during development because of the inability of the responsible Working Group to achieve consensus, while others are overtaken by events and become irrelevant or unneeded. The Recommendation Track process is responsible for advancing specifications from early Working Drafts to Recommendation, for terminating work on a specification before it reaches Recommendation status, for modifying approved Recommendations, and for rescinding approved Recommendations.

There are four “maturity levels” (often called “steps”) along the path toward Recommendation status:

There are well-defined criteria specified for advancing from one maturity level to the next. The entire process, from first draft to Recommendation, might take as little as several months, for very small, very mature specs, and might take as long as six or eight years, for large, complex sets of specifications involving the invention of new technology. Those readers interested in the details are encouraged to read the W3C Process Document.

B.3.1 What Is the JCP?

The Java Community Process (JCP)⁶ is described as “the open, participative process to develop and revise the Java^™ technology specifications, reference implementations, and test suites.” Its goal is to allow Java developers worldwide to participate in the ongoing evolution of the Java platform.

The day-to-day management of the JCP is handled by the Process Management Office (PMO), which is a group within Sun Microsystems. The PMO is responsible for the JCP website, for administering the membership program, for handling press releases, and other such activities. In our view, the PMO is the organ through which Sun exercises its control of the Java Community Process. Sun does this in order to ensure that the Java language and platform are not “hijacked” by entities that do not necessarily have the best interests of the Java community at heart. A cynic might add that Sun may want to be certain that the Java platform evolves in directions that support Sun’s corporate goals. We hasten to note, however, that the JCP is a process, not a consortium or an incorporated entity of any sort.

Participation in the JCP is open to any individual who, or organization that, signs the Java Specification Agreement (JSPA).⁷ The JSPA is a contract (which must be renewed annually) between the JCP member and Sun, setting out the rights and responsibilities of each. Once a JSPA has been sent to Sun and Sun has “executed” it, the new member has the rights to:

There are, in fact, two Executive Committees, one covering Standard and Enterprise Java matters and the other covering Micro Edition Java issues. Each Executive Committee is a body comprising a non-voting chair staffed by the PMO, 10 “ratified seats,” five elected seats, and a permanent seat for Sun. (A ratified seat is one filled by a JCP member nominated by the PMO and elected by the JCP membership. An elected seat is one filled by a JCP member nominated by and elected by the JCP membership.) The responsibilities of each EC include selection of proposed JSRs for inclusion in the JCP, approving draft specifications for review, giving final approval to completed specifications and their supporting material, and providing guidance to the PMO.

(By now, you may well feel as though you are drowning in alphabet soup. It’s unfortunate, but the TLAs⁸ are part of the jargon in many activities and organizations, especially those dominated by us computer software types. With our apologies, we unavoidably use them throughout this appendix.)

B.3.2 JSRs and Expert Groups: Formation and Operation

The term Java Specification Request reminds us of the term “Request For Comment” (RFC) used by the Internet Engineering Task Force (IETF).⁹ Those two terms mean somewhat more than a “request” for anything. They also might refer to the resulting document and to the process by which the document is developed.

The formal process of developing Java specifications is spelled out in the JCP Process Document.¹⁰ In Section B.2.3, you’ll read about the various stages through which a JSR goes from proposal to final specification. In this section, we are concerned with how a JSR comes into being and is set into motion, and with the nature of Expert Groups.

The Process Document defines a Java Specification Request as “the document submitted to the PMO by one or more Members to propose the development of a new Specification or significant revision to an existing Specification.” This definition clearly establishes the “request” aspect of the term.

A JSR is submitted to the PMO by one or more JCP members. The submission identifies the submitters, the person – less frequently, persons – who will be the Spec Leads (that is, the chairs) of the Expert Group that is intended to develop the specification, the initial members of the Expert Group, the reasons for the submission, and an estimated development schedule. Once the PMO approves the format of the submission and establishes that the submission is not redundant with existing or other newly proposed JSRs, the JSR is given to the appropriate EC for its consideration. The EC can reject the JSR or approve it; it can also reject it without prejudice and suggest changes that are likely to result in an approval in a future vote.

When approved by the EC, the JSR is given a number (e.g., JSR 225 is the 225th JSR approved by an EC). The JSR then transforms from a request into a process. The EG is constituted and the Spec Lead sets the development process in motion. In addition to acting as the chair of the EG, a Spec Lead is responsible for ensuring that there is a Reference Implementation (RI) of the specification being developed, as well as a Technology Compatibility Kit (TCK) – essentially a test suite to ensure that the RI actually conforms to the specification. If there are multiple Spec Leads, they may divide the responsibilities for the specification, the RI, and the TCK among themselves.

In practice, EG members frequently have little control over the JSR’s development. They essentially act as advisors to the Spec Leads, who are given complete authority over the JSR’s development. We know of some JSRs in which the Spec Leads have already-completed specifications, RIs, and TCKs when the JSR is submitted; in those JSRs, the EG is little more than a façade to give the appearance of an open process. Other JSRs are run by Spec Leads who are genuinely interested in an open, community-driven process and give EG members considerable ability to guide the specification’s development.

Ultimately, the Spec Lead has the responsibility to deliver – according to the planned schedule – the specification, the RI, and the TCK identified in the JSR’s initial submission, and to ensure that the specification has achieved a consensus from the overall Java community.

B.3.3 The JSR Stages of Progression

As you learned in Section B.3.2, a JSR is established, after submission to the PMO by one or more JCP members, by a vote of the appropriate EC.¹¹ A Spec Lead is named (in the submission), an Expert Group is constituted (starting with the initial members named in the submission), and the development process begins.

The current version of the JCP identifies four steps in the development of a Java specification:

Significant revisions to existing specifications aren’t handled by simple maintenance activities. Instead, they require a new JSR that goes through the entire process described in this section.

B.4.1 The De Jure Process and Organizations

A de jure standard is a specification that has been approved by a “recognized standardization body.” De Jure standards differ from specifications published by consortia and corporations principally because they are entirely public and are meant to be used freely by literally anybody. Although users may find themselves required to pay for copies of these standards, there is no cost associated with using the standard.

By contrast, specifications published by a consortium or corporation remain the property of the publisher, who may place restrictions or licensing obligations on their use. Not all such specifications come with such encumbrances. Increasingly, consortia that develop specifications they wish to become de facto standards are minimizing the restrictions they place on users of those specs. In many ways, the lines between this sort of standard and those published by de jure standards bodies have begun to blur.

One way in which the lines do not blur at all is the cost of participation. The de jure standards organizations commonly make it very easy for members of the general public to participate in their standards development activities. This often means that relatively low fees are charged for participation and that the documents under development have wide public visibility to permit public review, comment, and contribution.

By contrast, it is common for consortia to charge thousands, even tens or hundreds of thousands, of dollars for membership – and membership is always required for participation in the development of the specifications. Thus, even though the specifications might be made readily available for review and comment during their development cycles, only those who have paid for membership actively control the specifications.

There is a wide perception that de jure standards organizations have such burdensome rules and processes that development of their standards takes far too long in today’s fast-paced world. (The words glacial time have been sardonically applied to describe this situation.) We are convinced, however, that this is nothing more than a myth, an urban legend. Because of our extensive participation, spanning decades, in both sorts of organizations, we have good insights into the perceptions.

The standards developed by de jure standards bodies are always done very publicly, from their inception. The processes that seem so cumbersome and impractical have been put in place to ensure that all stakeholders, even those who are not active participants in the development of those standards, are served by due process – that is, they and their opinions are not excluded from affecting the content of the specification. Consequently, a very public development process that takes, say, three years from inception to final approval is all that exists.

Consortia and similar organizations, on the other hand, frequently (not universally) begin development of a specification long before the general public is made aware of the activity. Once the specification has been developed sufficiently, it is announced and often made visible for public review. By that time, however, the spec is usually far along the path toward publication. In addition, voting is done very quickly among the relatively small number of members of the organization – because the larger public isn’t involved. The result is that public perception of the time it takes to publish this sort of specification is shorter than the actual development period.

In the United States, the principle de jure standards organization is ANSI. According to the Wikipedia, ANSI is a “private, nonprofit standards organization that serves as a facilitator for the standardization work of its members in the United States. ANSI accredits standards developing organizations (SDOs) that meet a set of requirements and criteria governing the management of consensus standards development.”¹² ANSI is also the “National Body” that represents the United States in a number of international standards bodies, including ISO, IEC,¹³ and ITU.¹⁴ The standards published by ANSI are American National Standards.

ANSI does not develop standards itself. Instead, it delegates that job to SDOs that it accredits; those SDOs must adopt policies approved by ANSI that ensure complete openness of the standards development process. Examples of SDOs operating under the auspices of ANSI, of which there are scores, are the Acoustical Society of America, the American Boat and Yacht Council, the Clinical and Laboratory Standards Institute, the National Electrical Contractors Association, the Steel Shipping Container Institute, and the U.S. Department of Agriculture. For the purposes of this book, the most relevant ANSI SDO is INCITS, the InterNational Committee for Information Technology Standards.¹⁵ INCITS is responsible for development of and U.S. participation in the development of the great majority of IT (information technology) standards.

INCITS obviously has very broad responsibilities, so it delegates the IT standards development activities to its technical committees (TCs), which range from INCITS B10 (Identification Cards and Related Devices) to INCITS W1 (Office Equipment). It is INCITS H2 (Database) that is responsible for the development of the SQL standard. INCITS H2 is also the United States Technical Advisory Group (TAG) for participation in the corresponding ISO activities discussed in Section B.4.2. INCITS prescribes the procedures under which its TCs operate, initiates and manages public review of proposed American National Standards that it develops, and handles final approval of such standards.

Other countries (including all of the developed countries as well as many developing countries) have their own standards bodies, for example, BSI (the British Standards Institute) in the UK, AFNOR (Association Française de Normalization) in France, and DIN (Deutsches Institut für Normung) in Germany. They develop many standards that are used both nationally and internationally. BSI process management and evaluations standards are particularly well known, as are German safety standards.

Internationally, ISO is the principal standards development organization. ISO’s members are not individuals or commercial organizations. Instead, its members are the National Bodies of countries, such as ANSI from the United States. There are other international standards development organizations, many of them working closely with ISO. Because of space limitations, we focus these paragraphs on ISO alone.

ISO publishes standards on an extremely wide range of subjects, ranging from screw threads to processes for measuring the quality of steel in a furnace to algorithms for recognizing human faces automatically. Analogously to ANSI, ISO does not develop standards itself; instead, it delegates that responsibility to its Technical Committees (TCs). ISO TC 97 was responsible for most IT standardization until the late 1990s, when it joined forces with IEC to form Joint Technical Committee 1 (JTC 1), Information technology. The U.S. TAG for all of JTC 1’s activities is INCITS.

Many ISO TCs develop standards directly. However, like INCITS, JTC 1’s brief is far too broad for it to actually develop standards. It delegates those responsibilities to subcommittees (SCs). SC 32, Data Management and Interchange, is the JTC 1 subcommittee responsible for a number of standards activities related to databases, data management, and metadata management, including SQL. Often, an SCs responsibilities are too great for standards to be developed directly by the SC, in which case, the SC may further delegate that job to one or more working groups (WGs). SC 32 is in that position, and it has delegated responsibility for all parts of the SQL standard to its WG 3, Database Languages.

B.4.2 The SQL/XML Standardization Environment

ISO/IEC JTC 1/SC 32/WG 3 is, as you learned in Section B.4.1, the international organization responsible for SQL standards activities. You also learned that U.S. SQL-related activities are handled in INCITS H2. In this section, we compare and contrast the ways in which these two organizations go about their jobs.

INCITS H2 came into existence in 1978 as X3H2 (X3 being the original name of INCITS). Its members are organizations, such as corporations and even consortia, as well as individual persons. For years, H2 held approximately six meetings every year, at locations selected largely by the individual willing to act as host. (Interesting factoid: H2 is the only INCITS TC to have met in all 50 of the United States.) Membership of H2 has ranged as high as 50 or 60 members at its peak during the development of the 1992 edition of the SQL standard. In recent years, membership has dropped off to the low double digits (between 10 and 20, currently closer to 10) and meetings are held both at some physical location (about one-third of each year’s meetings) and by telephone and Internet (about two-thirds of the meetings each year).

WG 3 meets once each year concurrently, and collocated, with the rest of SC 32 (which includes three other working groups). In addition, it meets one or two additional times per year in order to increase the speed of development. After a ballot has completed, WG 3 meets with the authority of SC 32 in a ballot resolution meeting (often called an editing meeting).

During the development of SQL:1999, WG 3 had nine consecutive meetings of three weeks each! (“Consecutive” in this case does not imply that the start of one meeting began immediately following the end of the preceding meeting. The consecutive meetings were typically spaced about four months apart.) That was an extreme situation, happily, and WG 3 currently meets for only one or two weeks at a time.

One of us (Jim) has been a leading participant in, and editor for, both H2 and WG 3 for nearly 20 years. During that time, the de jure standards development landscape has changed dramatically. Some of those changes are definitely good – for example, WG 3 is able to publish a new standard (a new part of SQL) or to revise an existing standard in only two years. In the “good old days,” the process was more likely to take three to four years. (Some of us believe that the time can be shortened to 18 months, but only if no glitches in the process or in timing of meetings comes into play.)

Both H2 and WG 3 require that all changes to an emerging standard be provided as formal, detailed change proposals that give the editor specific instructions for adding, deleting, or modifying text in the document. By contrast, most standards-developing consortia and even many INCITS and ISO organizations accept conceptual changes and give the documents’ editors broad discretion to make appropriate changes. With a standard as large as SQL, however, the editor (that would be Jim) has declined that authority.

One important consideration of any standards effort is the impact of intellectual property rights, or IPR. IPR includes patents, copyrights, and even trade secrets. Most de jure standards organizations, including JTC 1 and INCITS, require that all participants declare the existence of any IPR concerning the technology that is being proposed for inclusion in a standard – regardless of whether that technology is being proposed by the owner of the IPR or by someone else. This policy avoids the potential disaster of adopting and publishing a standard that contains some organization’s IPR, after which the owner of that IPR can make almost any demand on implementers and users of the standard. Neither INCITS nor JTC 1 prohibits the use of patented or otherwise protected material in standards, as long as the facts – and the terms of its inclusion – are disclosed in advance. In virtually all cases, these de jure standards organizations require that the IPR be made available on reasonable and nondiscriminatory (RAND) terms;¹⁶ some such organizations go a step further and require that the IPR be made royalty-free (RF) as well.

SQL/XML is currently receiving more attention in both H2 and WG3 than any other aspect of the SQL standard. The first edition of SQL/XML was published as part of the 2003 edition of the SQL standard (commonly known as SQL:2003). A second edition was completed very late in 2005, and the third edition is planned for late 2007. The specification was very heavily revised and extended between the first and second editions – as you read in Chapter 15, “SQL/XML,” the 2003 edition was based on the XML Information Set, while the 2005 edition is based on the XQuery 1.0 and XPath 2.0 Data Model. Plans for the 2007 edition include the addition of Full-Text support and the ability to update XML data (see Chapter 13, “What’s Missing?”).

B.4.3 Stages of Progression

Aside from such details as meeting frequency and what sort of members they allow, the ANSI/INCITS and ISO/JTC 1 standards development processes are significantly different in many ways. We discuss first the U.S. process and then the international one.

Development of an American National Standard involves 8 “milestones”:

Slight variations of the milestones exist to accommodate publication of an International Standard as an American National Standard and for “fast-tracking” a standard or specification developed outside of the ANSI/INCITS procedures as an American National Standard.

INCITS’ procedures have been fine-tuned to the point that a standard can be published in as little as 18 months following initial submission of a project proposal.

The JTC 1 process comprises more steps than the ANSI/INCITS one. There are many possible variations, depending on the exact type of document being processed, but the following is typical for processing a specification intended to become an International Standard (IS).

When a specification is intended for publication as both an International Standard and an American National Standard – which is the case with SQL/XML and the other parts of the SQL standard – these two disparate processes must be synchronized to whatever degree possible. That poses some interesting management challenges, but there are formal policies in place (at least within INCITS) to help keep them aligned.