The .NET XPath Navigation API

The XML DOM support for XPath expressions has a double goal. First, it smooths the transition from MSXML COM code to the .NET Framework. Second, it gives you a built-in and easy-to-use mechanism to search for nodes in a memory-mapped XML document. As mentioned, the core .NET API for processing XPath expressions is built into a tailor-made class named XPathNavigator.

You access the navigator object either from the XmlDocument class or from the newest XPathDocument class. Figure 6-5 illustrates the relationship between the two ways of accessing XPath functions in the .NET Framework.

As you can see, the XmlDocument and XPathDocument classes have different internal layouts. XmlDocument implements XML DOM, whereas XPathDocument provides a more agile and compact structure, designed to speed XPath-driven navigation. (Later in this chapter, in the section “The XPathDocument Class,” on page 281, I’ll have more to say about this.)

No matter the application-level API, the sequence of steps necessary to execute XPath queries on an XML data source is always the same:

The XPathNavigator Class

The programming interface of the navigator object is defined in the XPathNavigator abstract class. The XPathNavigator class represents a generic interface designed to act as a reader for any data that exposes its contents as XML.

Functionally speaking, the XPathNavigator class is not much different from a pseudo-class that simply groups together all the XML DOM methods (ChildNodes, SelectNodes, and SelectSingleNode) to navigate the document contents. The big difference lies in the fact that XPathNavigator is a distinct component completely decoupled from the document class. As mentioned, XPathNavigator represents a generic interface to navigate and read data from any XML-based, or XML-looking, contents.

The XPathNavigator class enables you to move from one node to the next and perform XPath queries. In the .NET Framework, only three classes support XPath navigators: XmlDocument, XPathDocument, and XmlDataDocument.

An XPath navigator works on top of a special breed of XML document class that is generically referred to as an XPath data store. An XPath data store is simply any .NET Framework class that exposes its contents as XML and that can be queried using XPath expressions. An XPath data store can be based on a native XML stream or other data sources exposed as XML. For example, both the XmlDocument and XPathDocument classes are built from well-formed XML data. In contrast, the XmlDataDocument class exposes as XML the contents of an ADO.NET DataSet object. In all cases, however, the XPath query and navigation API works just fine.

As a stand-alone class providing a programming interface, the navigator is much more than a simple collection of XPath-related methods. The XPath navigator is not bound to a particular class document and can be associated with a number of data container classes.

A .NET Framework class becomes XPath-enabled simply by implementing the IXPathNavigable interface. This interface consists of a single method, CreateNavigator, that creates and returns an instance of a document-specific navigator object, as shown here:

public interface IXPathNavigable 
{
    XPathNavigator CreateNavigator();
}

All document-specific navigators derive from the XPathNavigator abstract class.

XPath Navigators and XML Readers

The MSDN documentation defines an XPath navigator as a class that reads data from an XML-based data store using a cursor model. XPathNavigator, therefore, provides read-only, random access to the underlying XML-based data. The navigator has a notion of the current node and advances the internal pointer using a series of move methods. When the navigator is positioned on a given node, all of its properties reflect the value of that node. How is this different from the XML readers that we encountered in Chapter 2?

XPath navigators and XML readers are radically different objects, although both look like client-side cursors for reading XML data. Let’s review the key differences:

• Connection model

  Both readers and navigators work on top of a data source. Readers, however, work connected to the input stream, which is often a persistent storage medium like a disk file. Navigators always work on memory-mapped data sources like XML DOM or more optimized and specialized structures. Readers must be closed when you have finished with them; navigators are simply garbage-collected when they go out of scope. A parallel can be drawn with ADO.NET data readers and DataSet objects. An XML data reader object, like the SqlDataReader base class, is connected to the data source, whereas a DataSet object is a disconnected object.

• Navigation interface

  Readers are simple read-only and forward-only cursors. Navigators too are read-only, but they let you move forward and backward. The navigator’s set of move methods is significantly richer. In particular, the set includes methods for going to the root of the underlying document, for reaching the parent node, for reaching the next and the previous sibling, for reaching the node where the given namespace is defined, and even more. In addition, you can synchronize the navigator position with the current position on another navigator object.

• Programming interface

  Navigators provide rich XPath capabilities and supply methods that perform XPath queries and return groups of related nodes. You have a generic Select method but also ad hoc selection methods that specialize on the most common XPath axes, such as descendant, ancestor, and child. In addition, navigators can simply evaluate an XPath expression and return the value.

Conceptually, XPath navigators and XML readers occupy diametrically opposed positions in the .NET XML puzzle. Moreover, this difference clearly stems from their names. Navigators are thought to traverse XML-based or XML-looking data. XML readers are simply lower-level tools that you can use to read XML-based or XML-looking data and build in-memory data structures that navigators rely on.

The XPathNavigator Programming Interface

Let’s briefly review the properties and methods that form the programming interface of the XPathNavigator class. A valid instance of the class can be obtained by calling the CreateNavigator method on any .NET Framework class that implements the IXPathNavigable interface.

Properties of the XPathNavigator Class

Table 6-3 summarizes the properties of the XPathNavigator class. As you can see, most of these properties reflect the characteristics of the currently selected node.

Like XML readers and XML DOM documents, an XPath navigator employs a name table to more efficiently store strings. The set of properties looks like the subset of properties that in the XmlTextReader class characterizes the current node.

Methods of the XPathNavigator Class

The tables in this section group the methods available in the XPathNavigator class into three main categories: move methods, selection methods, and miscellaneous methods. Table 6-4 lists the move methods.

The MoveTo method attempts to synchronize the current instance of the XPathNavigator object with another instance. MoveTo returns true or false depending on the success or failure of the operation. Note that the synchronization always fails if the two navigators are actually implemented through different and incompatible classes. Two navigators have different implementations if the other navigator can’t be cast to the current type.

Consider the following pseudocode:

public bool MoveTo(XPathNavigator other)
{
   InternalXPathNavigator nav = other as InternalXPathNavigator;
   if (nav == null)
      return false;
   ⋮
}

In C#, the as operator behaves like a cast except that, when the conversion fails, it returns null rather than raising an exception. In the preceding pseudocode, the InternalXPathNavigator class represents the actual (and internal) navigator class you got from the document’s CreateNavigator method. Each XPath-enabled document class actually instantiates a custom navigator class and returns that class when you call its CreateNavigator method.

The MoveTo method also might fail when the two navigators share the same implementation but point to different document instances. What happens in this case, however, depends on the specific implementation. In particular, MoveTo fails when the document class is XmlDocument or XmlDataDocument, but not when the underlying data object is an instance of XPathDocument.

Table 6-5 lists the XPathNavigator class’s methods for selecting nodes through XPath queries.

None of these methods produces any effect on the state of the XPathNavigator object. The following code snippet demonstrates how to select the descendants of a node. The code to get the ancestors is nearly identical.

// Create the underlying XPath-enabled document object
XPathDocument doc = new XPathDocument(fileName);

// Create the navigator for the specified object
XPathNavigator nav = doc.CreateNavigator();

// Select the descendants of the current node that match
// the specified criteria
nav.SelectDescendants(nodeName, nsUri, selfIncluded);

SelectDescendants, as well as SelectAncestors, has the following two overloads. The former takes a node type and returns only the nodes of that type, if any. The latter takes a node name and a namespace URI.

XPathNodeIterator SelectDescendants(XPathNodeType, bool);
XPathNodeIterator SelectDescendants(string, string, bool);

If you pass both the node name and the namespace URI as empty strings, all descendant nodes with no namespace information are selected. This method, and the homologous SelectAncestors and SelectChildren methods, is a specialized query performed along the corresponding XPath axis.

The Boolean argument you specify in the method signatures indicates whether the context node must be included in the final node-set. Setting the argument to true is equivalent to working along the descendant-or-self axis.

Table 6-6 lists the remaining XPathNavigator methods.

As you can see, several methods have to do with XPath expressions that are often rendered as instances of the XPathExpression class. But why do we need to express an XPath command using a new class?

XPath Expressions in the .NET Framework

An XPath expression is first of all a string that represents a location path, but an XPath expression is a bit more than a plain command string. It has a surrounding context that is just what the .NET Framework XPathExpression class encapsulates. The context of an expression includes the return type and the namespace information to handle the involved nodes.

The XPathExpression Class

Table 6-7 lists the methods and properties that characterize a .NET Framework XPath expression.

Looking at the programming interface of the XPathExpression class, you’ll notice the methods Clone and AddSort. As its name suggests, Clone makes a deep copy of the object, creating a brand-new and identical object. AddSort, on the other hand, associates the expression with a sorting algorithm that will be automatically run once the node-set for the expression has been retrieved.

The XPathExpression class is not publicly creatable. To get a new instance of this class, you must take a plain XPath string expression and compile it into an XPathExpression object.

Compiling Expressions

Both the XML DOM SelectNodes method and the navigator object’s Select method let you execute an XPath query indicating the expression as plain text. In spite of this simplified programming interface, in the .NET Framework, an XPath expression can execute only in its compiled form. This means that both the aforementioned methods silently compile the provided text into an XPathExpression before proceeding.

There are several advantages to compiling the expression yourself. For one thing, you can reuse the compiled object over and over. If you repeatedly call an XPath selection method to work on the same expression, each time the method will instantiate the same object. If you have a compiled expression, you save a few operations.

In addition, a compiled expression lets you know in advance about the expected return type. The return type is one of the values defined in the XPathResultType enumeration, shown in Table 6-8.

The Boolean, NodeSet, Number, and String types come directly from the W3C specification; the others represent extensions. However, Any and Error do not introduce any new functionality but simply make more consistent the enumeration type.

If you use a compiled expression, you can add namespace information to process the nodes and define a sorting algorithm for the resultant node-set. All this extra information remains associated with the XPathExpression object and can be reused at will.

To compile an expression, you use the Compile method of the XPathNavigator class. The method takes a string and returns an XPathExpression object, as shown here:

XPathDocument doc = new XPathDocument(fileName);
XPathNavigator nav = doc.CreateNavigator();
XPathExpression expr = nav.Compile(xpathExpr);

// Output the expected return type
Console.WriteLine(expr.ReturnType.ToString());

// Execute the expression
nav.Select(expr);

A compiled XPath expression can be consumed by a few navigator methods, including Select, Evaluate, and Matches.

Setting Namespace Information

The information you can pass through the SetContext method helps the XPath processor to resolve any namespace references in the expression. If no prefix appears in the expression, it is assumed that the namespace URI for all nodes is the empty namespace. Otherwise, you must let the processor know about defined prefix and namespace URI mappings.

You create an XmlNamespaceManager object, pack it with all the needed information, and then use the SetContext method to register it with the XPath expression object, as shown here:

// Create the navigator
XPathDocument doc = new XPathDocument(fileName);
XPathNavigator xnm = doc.CreateNavigator();

// Create and populate the XML namespace manager
XmlNamespaceManager xnm = new XmlNamespaceManager(nav.NameTable);
xnm.AddNamespace("dd", "urn:dino-e");
xnm.AddNamespace("es", "http://www.contoso.com");

// Set the expression’s context
XPathExpression expr = nav.Compile(xpathExpr);
expr.SetContext(xnm);

The .NET XPath processor is designed to look for the namespace manager on the XPath expression object prior to proceeding.

Evaluating Expressions

As mentioned, when evaluated, an XPath expression can return any of four basic types: node-set, Boolean, number, or string. If the return type is a node-set, you can run the expression through both the Select method and the Evaluate method.

The Select method returns an object of type XPathNodeIterator that you can use to walk your way through the members of the node-set. Unlike Select, the Evaluate method returns a generic object type, which it is your responsibility to cast to the correct strong type, as in the following example:

XPathNodeIterator iterator = (XPathNodeIterator) nav.Evaluate(expr);

Expressions that do not return a node-set can be used only with the Evaluate method. In this case, however, you must also cast the returned object to a strong type, as shown here:

string buf = (string) nav.Evaluate(expr);

The Evaluate method has no effect on the state of the navigator. An interesting overload for the method is shown here:

public object Evaluate(
   XPathExpression expr,
   XPathNodeIterator context
);

Normally, the expression is evaluated using the current node in the navigator as the context node. Using this overload, however, you can control the context node for the expression. If the context argument is null, the method works as usual. Otherwise, if context points to a valid iterator object, the current node in the iterator is used to determine the context node for the XPath expression.

Sorting the Node-Set

An interesting extension to the XPath programming model built into the XPathExpression class and the XPath processor is the ability to sort the node-set before it is passed back to the caller. To add a sorting algorithm, call the AddSort method of the XPathExpression object. AddSort allows for two overloads, as follows:

public void AddSort(
   object expr,
   IComparer comparer
);
public void AddSort(
   object expr,
   XmlSortOrder order,
   XmlCaseOrder caseOrder,
   string lang,
   XmlDataType dataType
);

The expr argument denotes the sort key. It can be a string representing a node name or another XPathExpression object that evaluates to a node name. In the first overload, the comparer argument refers to an instance of a class that implements the IComparer interface. The interface supplies a Compare method that is actually used for comparing a pair of values. Use this overload if you need to specify a custom algorithm to sort nodes.

expr.AddSort("lastname", Comparer.Default);

class MyOwnStringComparer : IComparer
{
   public int Compare(object x, object y)
   {
      string strX = (string) x;
      string strY = (string) y;

      // 0 if equals, >0 if x>y, <0 if x<y   
      return String.Compare(strX, strY);
   }
}

The second overload of the AddSort method always performs a numeric or text comparison according to the value of the dataType argument. In addition, you can specify a sorting order (ascending or descending) and even the sort order for uppercase and lowercase letters. In practice, you can decide whether lowercase letters must come before or after uppercase letters. The constant XmlCaseOrder.None simply ignores the case. Finally, the lang argument indicates the language to use for comparison—for example, “us-en” for U.S. English.

The following code snippet selects all the <Employee> nodes from our original sample XML file. This time, we make use of a compiled expression with sorting capabilities.

XPathDocument doc = new XPathDocument(fileName);
XPathNavigator nav = doc.CreateNavigator();
XPathExpression expr; 
expr = nav.Compile("/MyDataSet/NorthwindEmployees/Employee");
expr.AddSort("lastname", 
             XmlSortOrder.Ascending, XmlCaseOrder.None, 
             "", XmlDataType.Text);
XPathNodeIterator iterator = nav.Select(expr);

The iterator now returns nodes sorted in ascending order on the values stored in the lastname child nodes.

Is there a way to sort by multiple fields? As mentioned, the expr argument of the AddSort method can also be an XPath expression, and by exploiting this feature, you can involve more nodes in the sort process. When sorting database tables, you normally indicate the sortable columns in a comma-separated string. In this case, you must provide a valid XPath expression. The expression will be evaluated to a string and the actual value used to sort nodes. To concatenate the contents of two or more nodes, you must resort to the XPath concat core function—the only XPath way to concatenate strings. The following code sorts by title and lastname. To demonstrate the flexibility of the solution, the node contents are separated with an unnecessary comma.

string sortKey = "concat(concat(title, ’,’), lastname)";

Figure 6-6 demonstrates that using the AddSort method does change the structure of the final node-set.

To generate the output shown in this figure, I made use of an XPath iterator to visit all the nodes and their own subtrees. We’ll examine this code in detail in the section “Visiting the Selected Nodes,” on page 286, but first we’ll take a look at the internal layout of the XML document classes the navigator relies on.

XPath Data Stores

As mentioned, an XPath navigator works on top of an ad hoc document class. The .NET Framework provides three XPath-enabled classes: XmlPathDocument, XmlDocument, and XmlDataDocument. These classes have in common the IXPathNavigable interface.

In theory, each .NET Framework class can become XPath-enabled. In practice, however, only a subset of classes is a good candidate. In the first place, the class must act as the in-memory repository of some sort of content. Second, this content must be, or must be exposed as, XML. When these two prerequisites are met, classes can reasonably implement the IXPathNavigable interface and create their own navigators.

An XPath navigator is always class-specific and is built by inheriting from the abstract class XPathNavigator. Although in practice you always use navigators through the generic reference type of XPathNavigator, each class has its own navigator object. Table 6-9 lists these internal, undocumented classes; they are programmatically unaccessible, and often each is implemented in a different way. Despite this complexity, however, the classes’ application-level programming interface is common and is based on their base class XPathNavigator.

The document-specific navigator exploits the internal layout of the document class to provide the navigation API. A document-specific navigator can also have new methods and properties that make sense to a particular implementation. In this case, however, the navigator’s author must carefully document the new features; otherwise, it would be hard for a caller to exploit them through the generic XPathNavigator interface.

In the following sections, we’ll review the characteristics of the various XPath-enabled document classes.

The XPathDocument Class

The XPathDocument class provides a highly optimized, read-only in-memory cache for XML documents. Specifically designed to serve as an XPath data container, the class does not provide any information or identity for nodes. XPathDocument simply creates an underlying web of node references to let the navigator operate quickly and effectively. XPathDocument does not respect any XML DOM specification and has only one method—CreateNavigator.

The internal architecture of the XPathDocument class looks like a linked list of node references. Nodes are managed through an internal class (XPathNode) that represents a small subset of the XmlNode class, which is the official XML DOM node class in the .NET Framework. You can access the XML nodes of the document only through the properties exposed by the navigator object. (See Table 6-3.)

The following code shows how to create a new, XPathDocument-driven navigator object:

XPathDocument doc = new XPathDocument(fileName);
XPathNavigator nav = doc.CreateNavigator();

The returned navigator is positioned at the root of the document. The XPathDocument class supports only XML-based data sources, and you can initialize it from disk files, streams, text, and XML readers.

SqlCommand cmd = new SqlCommand(query, conn);
XmlTextReader reader = (XmlTextReader) cmd.ExecuteXmlReader(); 
XPathDocument doc = new XPathDocument(reader);

The XmlDocument Class

XmlDocument is the class that represents the .NET Framework implementation of the W3C-compliant XML DOM. This aspect of XmlDocument was covered in detail in Chapter 5.

Unlike XPathDocument, the XmlDocument class provides read/write access to the nodes of the underlying XML document. In addition, each node can be individually accessed and sets of nodes can be selected through XPath queries run by the SelectSingleNode and SelectNodes methods, respectively.

The XmlDocument class also enables you to create a navigator object. In this case, however, the navigator will work on a much more rich and complex web of node references. The following code shows how to get the navigator for the XmlDocument class:

XmlDocument doc = new XmlDocument();
doc.Load(fileName);
XPathNavigator nav = doc.CreateNavigator();

In particular, XmlDocument’s navigator class extends the interface of the standard navigator by implementing the IHasXmlNode interface. This interface defines just one method, GetNode, as shown here:

public interface IHasXmlNode 
{
   XmlNode GetNode();
}

Using this method, callers can access and query the currently selected node of the navigator. This feature is simply impossible to implement for navigators based on XPathDocument because it exploits the different internal layout of the XmlDocument class. By design, the XPathDocument class minimizes the memory footprint and does not provide node identity.

If the GetNode method is an extension to the XPathNavigator base class, how can callers take advantage of it? Here’s a code snippet:

XmlDocument doc = new XmlDocument();
doc.Load(fileName);
XPathNavigator nav = doc.CreateNavigator();
XmlNode node = ((IHasXmlNode) nav).GetNode();

At this point, the caller program has gained full access to the node and can read and update it at will.

The XmlDataDocument Class

The XmlDataDocument class is an extension of XmlDocument designed to allow the manipulation of a relational DataSet object through XML. The class also allows for rendering XML data as a relational DataSet object; but this aspect is less important here. (We will return to this topic in Chapter 8.)

The XmlDataDocument class provides a CreateNavigator method to let callers navigate the XML representation of an ADO.NET DataSet object. This is a neat example of the fact that the .NET Framework navigation API can be indifferently applied to XML-based data as well as XML-looking data. Like the XmlDocument navigator, the XmlDataDocument navigator also is not positioned on the root of the document but is positioned on the node from which the CreateNavigator method was called.

Custom Navigator Objects

The .NET Framework navigation API is extensible with navigator objects that work on top of particular XML documents or any other data exposed through a virtual XML node structure. To XPath-enable a given data source, you create a class that inherits from XPathNavigator. You can associate this new navigator class with a document class or make it a stand-alone creatable class. The MSDN documentation includes an example class named FileSystemNavigator. I extracted it from the documentation and compiled the C# and Microsoft Visual Basic code into an assembly. The assembly is available in this book’s sample files.

The file system navigator supports a virtual node structure similar to the following:

<root Name="…" CreationTime="…">
   <folder Name="…" CreationTime="…" />
   <folder Name="…" CreationTime="…">
   <file Name="…" CreationTime="…" Length="…" />
   <file Name="…" CreationTime="…" Length="…" />
   ⋮
   </folder>
</root>

Notice that the sample file system navigator places all subfolders of the context folder at the same level, thus losing any hierarchical information. The following code snippet shows how the custom navigator can be created and used:

XPathNavigator nav = new FileSystemNavigator("c:\folder");

// Exclude the folder itself but not all the subfolders.
// (If you run this on c: a VERY LONG list of nodes is returned...)
XPathNodeIterator it = nav.Select("descendant::*[position() >1]");
while(it.MoveNext())
   Console.WriteLine(it.Current.Name);

In this case, the architecture of the sample code makes it significantly harder to execute a query that selects only the children of the context folder. The preceding listing returns all the folders and files below the c: folder despite the effective parent folder. The predicate [position() >1] skips over the context folder name.