The .NET Framework XML API
The essence of XML in the .NET Framework is found in two abstract classes—XmlReader and XmlWriter. These classes are at the core of all other .NET Framework XML classes, including the XMLDOM classes, and are used extensively by various subsystems to parse or generate XML text. For example, ADO.NET data adapters retrieve the data to store in a DataSet object using a database reader, and the DataSet object serializes its contents to the DiffGram format using an XmlTextWriter object, which derives from XmlWriter.
XML readers and writers constitute the primitive I/O functions for XML documents and are used to build more sophisticated functionalities. So overall, you have two possible approaches when it comes to processing XML data. You can use any of the specialized classes built on top of XmlReader and XmlWriter as well as document classes that expose the contents through the well-known and classic XMLDOM.
The direct use of readers represents a stream-based, but fast and stateless, approach to XML parsing. The use of XMLDOM classes (for example, XmlDocument) represents the traditional XMLDOM parsing model. Readers are representative of a pull model, as opposed to the SAX parser’s typical push model. You can certainly build a push model atop a pull model–based API. Unfortunately, the reverse is never true, and that’s why there is no SAX support in the .NET Framework. (In Chapter 2, you’ll learn the basics of implementing a SAX parser using .NET Framework XML readers.)
The XML API for the .NET Framework comprises the following set of functionalities:
XML readers
XML writers
XML document classes
All of these functionalities must overcome the rather subtle problem of type mapping. The .NET Framework XML type system has several things in common with the XSD Schema type system, and ad hoc conversion classes in the .NET Framework provide for applicable transformations.
Before we go any further into this overview of the key groups of classes, let’s look at readers and writers in general. Readers and writers represent two rather generic software components that find several concrete (and powerful) implementations throughout the .NET Framework. The reader component provides a relatively common programming interface to read information out of a file or a stream. The writer component offers a common set of methods to write information down to a file or a stream in a format-independent way. Not surprisingly, readers operate in read-only mode, whereas writers accomplish their tasks operating in write-only mode.
.NET Framework Readers and Writers
In the .NET Framework, the classes available from the System.IO namespace provide for both synchronous and asynchronous read/write operations on two distinct categories of data: streams and files. A file is an ordered and named collection of bytes and is persistently stored to a disk. A stream represents a block of bytes that is read from, and written to, a data store. The data store can be based on a variety of storage media, including memory, disk files, and remote URLs. A stream is a kind of superset of a file, or in other words, a file that can be saved to a variety of storage media including memory. To work with streams, the .NET Framework defines several flavors of reader and writer classes. Figure 1-1 shows how each class relates to the others.
Figure 1-1. Streams can be read and written using made-to-measure reader and writer classes.
The base classes are TextReader, TextWriter, BinaryReader, BinaryWriter, and Stream. With the exception of the binary classes, all of these classes are marked as abstract (MustInherit, if you speak Visual Basic) and cannot be directly instantiated in code. You can use abstract classes to reference living instances of derived classes, however.
In the .NET Framework, base reader and writer classes find a number of concrete implementations, including StreamReader and StringReader and their writing counterparts. By design, reader and writer classes work on top of .NET streams and provide programmers with a customized user interface able to handle a particular type of underlying data or file format. Although each specific reader or writer class is tailor-made for the content of a given type of stream, they share a common set of methods and properties that defines the official .NET interface for reading and writing data.
The Cursor-Like Approach
A reader works in much the same way as a client-side database cursor. The underlying stream is seen as a logical sequence of units of information whose size and layout depend on the particular reader. Like a cursor, the reader moves through the data in a read-only, forward-only way. Normally, a reader is not expected to cache any information, but this is only common practice, rather than a strict requirement for all standard .NET readers.
ADO.NET data reader classes (for example, SqlDataReader) are simply .NET readers that move from one record to the next and expose the contents of the current record through a tailor-made interface. The unit of information read at every step is the database row. Similarly, a reader working on a disk file stream would consider as its own atomic unit of information the single byte, whereas a text reader would perhaps specialize in extracting one row of text at a time.
XML readers are simply another, very peculiar, type of .NET reader. The class parses the contents of an XML file, moving from one node to the next. In this case, the finer grain of the information processed is represented by the XML node—be it an element, an attribute, a comment, or a processing instruction.
XML Readers
An XML reader makes externally available a programming interface through which callers can connect and pull out all the data they need. This is in no way different from what happens when you connect to a database and fetch data. The database server returns a reference to an internal object—the cursor—which manages all the query results and makes them available on demand. This statement applies regardless of the fact that the database world might provide several flavors of cursors—client, scrollable, server-side, and so on.
With XML readers, client applications are returned a reference to an instance of the reader class, which abstracts the underlying data stream. Methods on the reader class allow you to scroll forward through the contents, moving from node to node rather than from byte to byte or from record to record. When viewed from the perspective of readers, an XML document ceases to be a tagged text file and becomes a serialized collection of nodes. Such a cursor model is specific to the .NET platform, and to date, you will not find a similar programming API available for other platforms, including Microsoft Win32.
Readers vs. XMLDOM
XML readers don’t require you to keep more data in memory than you actually need. When you open the XML document, a simple logical pointer that corresponds to a node is returned. You can easily skip over nodes to locate the one you need. In doing so, you don’t tax in any way the application’s memory with extra data other than that required to bufferize the currently selected node.
In contrast, the XMLDOM—a full read/write parser model—has the drawback that it might require a significant memory footprint and a long time to set up large documents in memory. Once in memory, however, the document can be easily and quickly read, edited, and serialized. To search a single node, or to change an individual property, you have to load the whole document in memory. As you can guess, this is not necessarily an optimal approach and might not be the appropriate way to go for most applications.
Taking the cursor-like approach to its limit, you can also observe an interesting convergence between readers and the XMLDOM. In fact, by visiting all element and attribute nodes in the stream and storing in a memory tree the related data, you build a dynamic and customized XMLDOM. Incidentally, this is just what happens in the .NET Framework when XMLDOM classes are instantiated using readers to load data and are serialized to disk using writers.
Readers vs. SAX
A SAX parser directly controls the evolution of the parsing process and pushes data to the client application. A cursor parser (that is, an XML reader), on the other hand, plays a more passive role and leaves client applications to control the process.
Giving applications, not the parser, control over the parsing process promotes the pull model (as opposed to the SAX parser’s push model), in which the parser is invoked to obtain a reference to the underlying XML document. The parser also exposes methods for the client to navigate through the obtained document.
In addition to providing a simplified programming interface, the pull model is on average more efficient than the push model. For example, the pull model allows client applications to implement selective node processing and just skip over unneeded nodes. With SAX and the push model, all data has to pass through the application, which is the only entity that can reliably determine what is of interest and what can be discarded.
Note
The push model, at least as implemented in SAX, can also be quite boring to code. SAX works by passing node contents to application-defined handlers. A handler is a living instance of an object that implements one or more interfaces according to the specification. So an application that needs to parse XML documents using SAX assigns instances of these objects to ad hoc properties on the SAX parser. Once started, the parser calls back the handlers through the predefined interfaces whenever it parses some content that relates to a given handler.
XML Writers
The .NET XML API separates parsing from editing and writing and offers a set of methods that provides effective results for performance as well as usability. When writing, you create new XML documents working at a considerably high level of abstraction and explicitly indicate the XML elements to create—nodes, attributes, comments, or processing instructions. The writer works on a stream, dumping content incrementally, one node after the next, without the random access capabilities of the XMLDOM but also without its memory footprint.
To grasp the importance of XML writers, consider that, in general, the only alternative you have for writing XML contents to any storage media consists of preparing the entire output as a string and then writing it off. In this case, the markup nature of XML is more hindrance than real help, because you must yourself take care of the intricacies of quotation marks, attributes, indentation, and end tags.
In the .NET Framework, XML writers come to the rescue and let you write XML documents programmatically in much the same way you write them through text editors. For example, you can specify whether you want a namespace prefix, the padding character and the size of the indentation, the quotation mark and the newline character, and even how you want white spaces to be treated. To create nodes, you simply use ad hoc methods to write comments, attributes, and element nodes. The overall method of working is simple and extremely effective.
The .NET Framework provides several types of writers that use heterogeneous output devices—strings, HTTP response, and HTML documents. You could also use an XML text writer to dump contents to a stream object or a new text file. In the latter two cases, you could also specify character encoding. If the encoding argument is null, the Unicode 8-bits-per-character schema (UTF-8) will be used.
XML writers, and in particular the XmlTextWriter class, are used throughout the .NET Framework for creating any sort of XML output. We’ll look at XML writers in detail in Chapter 4.
The XML Document Object API in .NET
As mentioned, along with XML readers and writers, the .NET Framework also provides classes that load and edit XML documents according to the W3C DOM Level 1 and Level 2 Core. The key XMLDOM class in the .NET Framework is XmlDocument—not much different from the DOMDocument class, which you might recognize from working with MSXML.
The XMLDOM supplies an in-memory tree-based representation of XML documents and supports both navigation and editing of the document. In addition, the XMLDOM classes can handle both XPath queries and XSLT.
Tightly coupled with the XmlDocument class is the XmlDataDocument class. It extends XmlDocument and focuses on XML storage and retrieval of structured tabular data. In particular, XmlDataDocument can import data from an ADO.NET DataSet object and export regular XML contents to the DataSet relational format. Regular XML content is a set of nodes with exactly one level of subnodes, with each node having the same number of children. The ultimate goal of this requirement is enabling the XML contents to fit into a relational table.
The XMLDOM representation of an XML document is fully editable. Attributes and text can be randomly accessed, and nodes can be added and removed. You perform updates on a loaded XMLDOM document by first creating a node object (the XmlNode class) and then binding it to the existing tree. All in all, the underlying writing pattern is close to that of XML writers—you write nodes to the stream in one case, and you add nodes to the tree in the other. Of course, if you are using the XMLDOM, bear in mind that all changes occur in memory and must be flushed to the storage medium prior to return. (The XMLDOM API is described in detail in Chapter 5.)
XPath Expressions and XSLT
In the .NET Framework, XSLT and XPath expressions are fully supported but are implemented in classes distinct from those that parse and write XML text. This is a key feature of the overall .NET XML API. Any functionality is provided through a small hierarchy of objects, although each subtree connects and interoperates well with others. Figure 1-2 demonstrates the interconnection between constituent APIs.
Figure 1-2. The XMLDOM API is built on top of readers and writers, but both XSLT and XPath expressions need to have a complete and XMLDOM-based vision of the entire XML document to process it.
XML readers and writers are the primitive elements of the .NET XML API. Whenever XML text must be parsed or written, all classes, directly or indirectly, refer to them. A more complex primitive element is the XMLDOM tree. Transformations and advanced queries must rely on the document in its entirety being held in memory and accessible through a well-known interface—the XMLDOM.
The XSLT Processor
The key class for XSLT is XslTransform. The class works as an XSLT processor and complies with version 1.0 of the XSLT recommendation. The class has two key methods, Load and Transform, whose behavior is for the most part self-explanatory.
Once you acquire an instance of the XslTransform class, you first load the source of an XSL document that contains the transformation rules. By calling the Transform method, you actually perform the conversion from native XML to the output format. Prior to applying the transformation, the underlying XML document is loaded as a kind of XMLDOM tree. (The details of XSLT are covered in Chapter 7.)
The XPath Query Engine
XPath is a language that allows you to navigate within XML documents. Think of XPath as a general-purpose query language for addressing, sorting, and filtering both the elements and the text of an XML document.
The XPath notation is basically declarative. Any XPath expression is a path within the XML document that identifies the information with the given characteristics. The path defines a pattern, and the resulting selection includes all the nodes that match it. The selection is expressed through a notation that emphasizes the hierarchical relationship between the nodes. It works in much the same way files and folders work. For example, the XPath expression “book/publisher” means find the “publisher” element within the “book” element. The XPath navigation model works in the context of a hierarchy of nodes in the XML document’s tree. XPath makes use of a variation of the XmlDocument class, named XPathDocument.
Running an XPath query is not actually different from executing a Transact-SQL (T-SQL) query on SQL Server. Instead of getting back a collection of rows, a valid XPath expression returns a collection of nodes. To scroll the returned nodes, you just use an XPath-customized version of a reader. We’ll look at XPath in more detail in Chapter 6.