Chapter 1. Schemas: An introduction
This chapter provides a brief introduction to schemas and explains why they are important. It also discusses the basic schema design goals and describes the various existing schema languages.
1.1. What is a schema?
The word schema means a diagram, plan, or framework. In XML, it refers to a document that describes an XML document. Suppose you have the XML instance shown in Example 1–1.
<product effDate="2001-04-12">
<number>557</number>
<size>10</size>
</product>
The instance consists of a product element that has two children (number and size) and an attribute (effDate).
The sentence you just read could be a schema because it describes the instance document, and all other instances of a product with the same kind of children and attribute. However, to be useful with XML and benefit from computer processing, this kind of schema won’t do. The schema must be defined in a schema document using a formal schema language.
Example 1–2 shows a schema document that describes the instance. It contains element and attribute declarations that assign types and names to elements and attributes. The document is written in the XML Schema Definition Language (XSD).
Example 1–2. Product schema in XSD schema language
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="product" type="ProductType"/>
<xs:complexType name="ProductType">
<xs:sequence>
<xs:element name="number" type="xs:integer"/>
<xs:element name="size" type="SizeType"/>
</xs:sequence>
<xs:attribute name="effDate" type="xs:date"/>
</xs:complexType>
<xs:simpleType name="SizeType">
<xs:restriction base="xs:integer">
<xs:minInclusive value="2"/>
<xs:maxInclusive value="18"/>
</xs:restriction>
</xs:simpleType>
</xs:schema>
In contrast, Example 1–3 on p. 9 shows an equally valid schema in the familiar Document Type Definition (DTD) language of XML. The disproportionate sizes of the two schema documents roughly reflect the capabilities of the two languages. The extensive features of XML Schema, when to use them, and how to use them effectively, form the subject matter of this book.
1.2. The purpose of schemas
Schemas are developed for many purposes. How effectively a schema can achieve them depends on the capabilities of the schema language and the schema processor, as well as the quality of the schema design.
1.2.1. Data validation
One of the most common uses for schemas is to verify that an XML document is valid according to a defined set of rules. A schema can be used to validate:
• The structure of elements and attributes. For example, a product must have a number and a size, and may optionally have an effDate (effective date).
• The order of elements. For example, number must appear before size.
• The data values of elements and attributes, based on ranges, enumerations, and pattern matching. For example, size must be an integer between 2 and 18, and effDate must be a valid date.
• The uniqueness of values in an instance. For example, all product numbers in an instance must be unique.
1.2.2. A contract with trading partners
Often, XML instances are passed between organizations. A schema may act as a contract with your trading partners. It clearly lays out the rules for document structure and what is required. Since an instance can be validated against a schema, the “contract” can be enforced using available tools.
1.2.3. System documentation
Schemas can provide documentation about the data in an XML instance. Anyone who needs to understand the data can refer to the schema for information about names, structures, and data types of the items. To include further documentation, you can add annotations to any schema component.
1.2.4. Providing information to processors
Schemas contain information about the type and structure of elements that is useful to have when processing the document. For example, if a processor knows the data type of a value, it knows how to sort it or compare it to other values, and it knows what operations it can reasonably perform on that data. Providing information to the processor can also be useful for debugging: If the processor knows that a certain element is not allowed by a schema, it can signal an error to the processing code.
1.2.5. Augmentation of data
Schema processing can also add to the instance. A schema can insert default and fixed values for elements and attributes and normalize whitespace according to the type.
1.2.6. Application information
Schemas provide a way for additional information about the data to be supplied to the application when processing a particular type of document. For example, you could include information on how to map the product element instances to a database table, and have the application use this information to automatically update that particular table with the data.
In addition to being available at processing time, such information in a schema can be used to generate code, such as:
• User interfaces for editing the information. For example, if you know that size is between 2 and 18, you can generate an interface with a slider bar that has these values as the limits.
• Stylesheets to transform instance data into a reader-friendly representation such as XHTML. For example, if you know that the human-readable name for the content of a number element is “Product Number” you can use this as a column header.
• Code to insert or extract the data from a database. For example, if you know that the product number maps to the PROD_NUM column on the PRODUCTS table, you can generate an efficient routine to insert it into that column.
1.3. Schema design
Schema languages often provide several ways to accurately describe the same thing. The decisions made during the design stage can affect a schema’s usability, accuracy, and applicability. Therefore, it is important to keep in mind your design objectives when creating a schema. These objectives may vary depending on how you are using XML, but some are common to all use cases.
1.3.1. Accuracy and precision
Obviously, a schema should accurately describe an XML instance and allow it to be validated. Schemas should also be precise in describing data. Precision can result in more complete validation as well as better documentation. Precision can be achieved by defining restrictive types that truly represent valid values.
1.3.2. Clarity
Schemas should be very clear, allowing a reader to instantly understand the structure and characteristics of the instance being described. Clarity can be achieved by
• Appropriate choice of names
• Consistency in naming
• Consistency in structure
• Good documentation
• Avoiding unnecessary complexity
1.3.3. Broad applicability
There is a temptation to create schemas that are useful only for a specific application. In some cases, this may be appropriate. However, it is better to create a schema that has broader applicability. For example, a business unit that handles only domestic accounts may not use a country element declaration as part of an address. They should consider adding it in as an optional element for the purposes of consistency and future usability.
There are two components to a schema’s broad applicability: reusability and extensibility. Reusable schema components are modular and well documented, encouraging schema authors to reuse them in other schemas. Extensible components are flexible and open, allowing other schema authors to build on them for future uses.
Since designing schemas well is so important, this book offers sections labeled “Design Hint” to address pros and cons of various alternatives in schema design. In addition, Chapters 21, 22, and 23 pull these principles together into a coherent design strategy.
1.4. Schema languages
1.4.1. Document Type Definition (DTD)
Document Type Definitions (DTDs) are a commonly used method of describing XML documents. The DTD syntax is the original W3C schema language, built into XML 1.0 itself. A DTD allows you to define the basic structure of an XML instance, including
• The structure and order of elements
• The allowed attributes for elements
• Basic data typing for attributes
• Default and fixed values for attributes
• Notations to represent other data formats
Example 1–3 shows a DTD that is roughly equivalent to our schema in Example 1–2.
Example 1–3. Product schema in DTD schema language
<!ELEMENT product (number, size?)>
<!ELEMENT number (#PCDATA)>
<!ELEMENT size (#PCDATA)>
<!ATTLIST product effDate CDATA #IMPLIED>
DTDs have many advantages. They are relatively simple, have a compact syntax, and are widely understood by XML software implementers.
However, DTDs also have some shortcomings. They do not support namespaces easily, and they provide very limited data typing, for attributes only. Also, because they have a non-XML syntax, they cannot be parsed as XML, which is useful for generating documentation or making wholesale changes. However, conversion tools such as James Clark’s open source Trang (www.relaxng.org) and other free and commercial products can convert DTDs to other schema languages for this purpose.
1.4.2. Schema requirements expand
As XML became increasingly popular for applications such as e-commerce and enterprise application integration (EAI), a more robust schema language was needed. Specifically, XML developers wanted:
• The ability to constrain data based on common data types such as integer and date.
• The ability to define their own types in order to further constrain data.
• Support for namespaces.
• The ability to specify multiple element declarations with the same name in different contexts.
• Object-oriented features such as type derivation. The ability to express types as extensions or restrictions of other types allows them to be processed similarly and substituted for each other.
• A schema language that uses XML syntax. This is advantageous because it is extensible, can represent more advanced models, and can be processed by many available tools.
• The ability to add structured documentation and application information that is passed to the application during processing.
DTDs have not disappeared since newer schema languages arrived on the scene. They are supported in many tools, are widely understood, and are still in use in many applications, especially in the publishing arena. In addition, they continue to be useful as a lightweight alternative to newer schema languages.
1.4.3. W3C XML Schema
Four schema languages were developed before work began on XML Schema: XDR (XML Data Reduced), DCD, SOX, and DDML. These four languages were considered, together, as a starting point for XML Schema, and many of their originators were involved in the creation of XML Schema.
The World Wide Web Consortium (W3C) began work on XML Schema in 1998, and it became an official recommendation on May 2, 2001.
On April 5, 2012, version 1.1 of XML Schema became official. It includes several significant enhancements as well as many small changes. One change was the name, which is officially “W3C XML Schema Definition Language (XSD) 1.1.” Understandably, this book follows the common practice of continuing to use the name “XML Schema,” along with “XSD” in syntax tables and other formal language contexts.1
XML Schema 1.1 is backward-compatible with 1.0, and schema authors do not need to specify in their schema documents the version to which they conform. A list of the major changes in 1.1 can be found in Section 23.5.1 on p. 640 of this book.
The formal recommendation is in three parts:
• XML Schema Part 0: Primer is a non-normative introduction to XML Schema 1.0 that provides a lot of examples and explanations. It can be found at www.w3.org/TR/xmlschema-0.
• XML Schema Part 1: Structures describes most of the components of XML Schema. The most recent version (1.1) can be found at www.w3.org/TR/xmlschema11-1.
• XML Schema Part 2: Datatypes covers simple types. It explains the built-in types and the facets that may be used to restrict them. It is a separate document so that other specifications may use it, without including all of XML Schema. The most recent version (1.1) can be found at www.w3.org/TR/xmlschema11-2.
1.4.4. Other schema languages
XML Schema and DTDs are not always the most appropriate schema languages for all cases. This section describes two other schema languages.
1.4.4.1. RELAX NG
RELAX NG covers some of the same ground as DTDs and XML Schema. RELAX NG was developed by an OASIS technical committee and was adopted as an ISO standard (ISO/IEC 19757-2). RELAX NG is intended only for validation; the processor does not pass documentation or application information from the schema to the application. RELAX NG does not have a complete built-in type library; it is designed to use other type libraries (such as that of XML Schema).
Some of the benefits of RELAX NG over XML Schema 1.0 have been addressed as new features in XML Schema 1.1. However, RELAX NG still has some advantages as compared to XML Schema 1.1:
• Many people consider the syntax of RELAX NG to be simpler and more elegant than that of XML Schema.
• It has a convenient, compact non-XML syntax.
• It includes attributes in the elements’ content models. For example, you can specify that a product element must either have an effectiveDate attribute or a startDate attribute. XML Schema does not directly provide a way to do this.
• It allows the definition of components from multiple namespaces in the same document. In XML Schema, multiple schema documents are required.
• It does not require content models to be deterministic. This is explained in Section 12.5.6 on p. 279.
However, RELAX NG also has some limitations compared to XML Schema:
• It has no equivalent of XML Schema 1.1 assertions, which allow complex XPath expressions to be used to determine the validity of an element or attribute.
• It has no type derivation capabilities. XML Schema’s restriction and extension mechanisms allow type substitution and many other benefits, described in Section 13.1 on p. 301.
• It has no equivalent of identity constraints. XML Schema’s identity constraint mechanism is useful in data-oriented applications, as described in Chapter 17.
• Because it is only intended for validation, it does not provide application information to the processor. In fact, the RELAX NG processor passes to the application the exact same information that is available from a DTD. This is not a disadvantage if your only objective is validation, but it does not allow you to use the schema to help you understand how to process the instance.
For more information on RELAX NG, see http://relaxng.org.
1.4.4.2. Schematron
XML Schema, DTDs, and RELAX NG are all grammar-based schema languages. They specify what must appear in an instance, and in what order.
Schematron, on the other hand, is rule-based. It allows you to define a series of rules to which the document must conform. These rules are expressed using XPath. In contrast to grammar-based languages, Schematron considers anything that does not violate a rule to be valid. There is no need to have declarations for every element and attribute that may appear in the instance.
Schematron, sometimes referred to as “ISO Schematron”, is an ISO standard (ISO/IEC 19757-3). Like RELAX NG, Schematron is intended only for validation of instances. It has a number of advantages over XML Schema 1.1:
• It is easy to learn and use. It uses XPath, which is familiar to many people already using XML.
• The use of XPath allows it to very flexibly and succinctly express relationships between elements in a way that is not possible with other schema languages.
• Assertions in a Schematron rule can access XML data anywhere in a particular instance document. XML Schema 1.1 assertions, by contrast, can only be based on the contents of a particular type.
The limitations of Schematron compared to XML Schema are:
• It does not provide a model of the instance data. A person cannot gain an understanding of what instance data is expected by looking at the schema.
• It is intended only for validation and cannot be used to pass any information about the instance, such as types or default values, to an application.
• Anything is valid unless it is specifically prohibited. This puts a burden to anticipate all possible errors on the schema author.
As Schematron and XML Schema complement each other, it makes sense to combine the two. For more information on Schematron, see www.schematron.com.