Preface
Extensible Markup Language (XML) is everywhere. In an astonishingly short period of time, XML has worked itself into the nooks and crannies of corporate IT departments, academic research institutions, and small-shop programming operations everywhere. As one of the authors of this book points out, XML now encodes a bewildering array of datatypes scattered across a large number of diverse application domains.
Where there's information, there are people who are going to need to query that information. Happily, XQuery has come along to provide a powerful and standardized way of searching through all that XML-encapsulated data. Going beyond Google, XQuery provides a flexible and easy-to-use mechanism for querying not only content, but structure as well. Drawing from XSLT, XQuery doesn't content itself with simply providing a query capability; it does transformations too. I believe that it will increasingly replace XSLT in a number of application areas (though XSLT aficionados needn't fear—there are plenty of things to do for both technologies). Most interestingly, some of the data that XQuery will manipulate won't even have started life as XML: A large portion of the data that XQuery deals with will have started life in the form of rows and columns in corporate relational databases.
This Book
Most technical books are either tutorials or reference works. You turn to the former if you're looking for a gentle introduction to a new subject; you look to the latter if you already know something about the subject and want a minimum of hand-holding while you seek out the information you need. This book is unusual in that it shares some of the characteristics of each. It consists of a series of technical essays and perspectives, some of them tutorial-like, on various facets of XQuery from members of the World Wide Web Consortium's (W3C) Query working group, the standards body that has been hard at work creating and crafting the formal definition of the language since the fall of 1999. Their intimate knowledge sheds a lot of light on a technical topic that some find daunting, given the weight and density of the formal spec.
Members of the working group would probably be embarrassed to hear themselves referred to here as “experts,” as the title implies, but that's what they are. Nobody knows the material better than this group—after all, they created it! In this case you could say (pun intended) that they “wrote the book.” And not just the one you're reading: They also wrote most of the specification itself, since nearly every one of the authors is also an editor of one or more of the numerous documents that comprise the formal specification.
In addition to being intimately involved in the creation of XQuery and the specification documents that define it (twelve at this point and counting), the credentials shared by the collective authorship of this book is impressive. They are researchers and theoreticians, software architects, product managers, educators, and database implementers. They are the designers of no less than six influential computer languages, including Haskell, Quilt, XML-QL, YATL, XQL, and SQL. Their views on query languages and XML are helping to shape the technology strategies of some of the most influential computer companies in the world. All in all, an impressive group! The “Contributors” section gives the bona fides of all our experts. It's worth perusing.
This book provides an opportunity for these people to write individually about the XQuery topics they know best and about which they are most passionate. It allows each of them to write with a personal voice and perspective in a way that's not possible within the carefully and consensually crafted framework of a formal standards specification.
Who Should Read This Book?
You should read this book if you're interested in XQuery—a statement that describes numerous technophiles these days. The members of two programming disciplines in particular, however, will find this volume especially valuable. XSLT developers will be interested in these essays. As noted above, XQuery shares a number of capabilities with XSLT and is likely to replace it in some areas. Michael Kay's chapter on “XQuery, XPath, and XSLT” (Chapter 3) examines the commonalities and differences between the two languages in some detail.
This book will also interest those who are corporately or otherwise involved with SQL and relational databases. XQuery has often been called SQL for XML, and that's no accident, given that several members of the working group come from a relational database background. XQuery was designed from the ground up to work with relational data. Two of the chapters in Part IV, “Databases,” examine the connections between XQuery and relational data.
Whatever your specific background, you should be comfortable and familiar with XML basics to benefit most fully from this book. You'll do best if you have some prior exposure to XQuery, although it needn't be extensive. At a minimum, the book assumes that you understand the fundamentals of XPath and know what a location path is. Jonathan Robie's introductory guided tour provides a good overview of XQuery for those who aren't already familiar with the technology.
Organization and Roadmap
Your approach to this material will depend on your specific experience and interests. Here's a quick roadmap to the four parts of the book and a look at the individual chapters to help you figure out where to dive in first.
Part I: Basics
Jonathan Robie's “XQuery: A Guided Tour” provides a good starting point if you're new or relatively new to XQuery. Its easy, tutorial-like style invites you to read it straight through. Jonathan provides a compact and convenient reference to XQuery's key features that newcomers and experts alike can return to periodically for a quick refresher on the basics while working through the detail in the other chapters. Jonathan's coverage is extensive; most of the entries in the glossary (a useful resource when you're stumbling across new terminology) are introduced in this chapter. (Note that glossary entries are bolded the first time they appear within the text.)
Part II: Background
The two chapters in Part II provide historical context and a detailed rationale for many of the complex technical decisions the Query working group has had to make as the language has evolved.
As Don Chamberlin explains in Chapter 2, “Influences on the Design of XQuery,” the process of designing XQuery has been one of resolving the tensions between conflicting goals, and his chapter provides a historical and technical description of that dialectic in action. This chapter also places XQuery in the context of the other related languages and standards with which it coexists. The second half of the chapter deals in cogent detail with the intricacies of what Don calls “watershed issues”: eight complex and often controversial issues that have had a major impact on the design of the language. Don goes a long way to teasing apart the complexities.
As Michael Kay notes in Chapter 3, “XQuery, XPath, and XSLT,” the fact that both these languages share a common data model, as well as a common sublanguage in XPath, is a major achievement of the W3C. His chapter explores that commonality and the overlap in functionality between the two languages, as well as their differences. This chapter describes where each language might be employed to best advantage. Mike addresses some of the same issues discussed by Don Chamberlin in the previous chapter but from the particular perspectives of XPath and XSLT.
Part III: Formal Underpinnings
Both chapters in Part III were written by Mary Fernández, Jérôme Siméon, and Philip Wadler. Chapter 4, “Static Typing in XQuery,” is a gentle tutorial that explores the ramifications of static typing in a language dealing with XML data. Despite the term formal in the title of this part, this chapter fits naturally here because it provides such a great lead-in to the chapter on the formal semantics—the topic isn't difficult or particularly heavy. Static typing is important to XQuery developers because it helps to guarantee program correctness, and it helps significantly to speed up the development cycle, as this chapter attests. Users need to understand the concept of static typing in order to better understand the error messages returned by the query system they're working with.
Chapter 5, “Introduction to the Formal Semantics,” is a tutorial-like introduction to the formal semantics, the formal mathematical system on which XQuery is built. This is the most challenging technical component of XQuery. This chapter should appeal to implementers, language theoreticians, and those just terminally curious about this aspect of the language. They will find here a much less daunting approach to the formal semantics than that provided by the specification.
Part IV: Databases
Most major relational database vendors already have or will shortly provide support for XQuery as a front end for relational data, and interest in XQuery from this market sector was one of the driving forces behind its development. Chapters 6 and 7 deal with the topic of XQuery and relational data. The authors collaborated on the content so that the chapters would fit seamlessly together. The last chapter in this part describes the features of what its author refers to as an XML database management system.
XQuery was designed to query XML, and using it to query and pull data out of relational databases that store information in tabular format poses a number of interesting challenges. Denise Draper explores these issues in “Mapping between XML and Relational Data.”
Michael Rys's chapter, “Integrating XQuery and Relational Database Systems,” builds on the foundation laid in Chapter 6. Michael shows two methods of accessing XML stored in a relational database. One approach uses the XML datatype to store XML into the database as an LOB (large object) and accesses that data using a combination of both XQuery and SQL. He also describes a so-called “top-level XQuery” approach that obviates the need to use SQL altogether.
Jim Tivy's concluding chapter, “A Native XML DBMS,” rounds out the offerings in this part. Jim describes the key features of what he calls an XML database management system and explores what that means. He looks at the use of the XQuery data model in that context and explores other XML DBMS features such as command languages and APIs, drawing on his experience with XStreamDB (his own native XML database product), Tamino, and the XML:DB Initiative.
Software
Let's not close without mentioning an extremely valuable resource if you're tracking what's going on with XQuery, http://www.w3.org/XML/Query, the W3C XML Query website. This is the official website of the working group, and just about everything relevant to XQuery gets posted here sooner or later, whether it's pointers to the latest specifications, news and articles on XQuery in the trade and technical press, information on the various discussion groups you can join or monitor, and an ever-growing list of real-life XQuery implementations.
Some of these are free, some are open-source (not always the same thing), some are full-strength, industrial products, and some come in the form of online demos. All the authors in this book are involved either directly or indirectly in XQuery-related software efforts at their various companies and research institutions (which shouldn't surprise anybody if you think about it, since most companies on the working group have XQuery projects under way), and a number of the authors of this book have been involved in the development of the software on this list.
Just to mention a few of these author/software connections: Mary Fernández, Jérôme Siméon, and Phil Wadler have all been involved in Lucent's implementation of Galax (which lets you make queries directly in the syntax of the formal semantics). Jim Tivy has already been mentioned in connection with XStreamDB. Jonathan Robie was active in Software AG's development of QuiP (which is a great reference implementation for hands-on exploration of XQuery syntax and semantics, although I'm told it's no longer being maintained), and even I have an open-source, Java-based implementation called XQEngine. Please visit the XML Query website for an up-to-date listing of all the XQuery software that's currently available. The list is growing daily.
Cover Photograph
The photograph on the cover was taken by Don Chamberlin on a visit to the Bronx Botanical Garden in 1972. The tulips are reminiscent of XQuery's FLWOR expression, one of its central and most powerful constructs.
Ongoing
XQuery is alive and still under development, and some of the information in this book is accordingly subject to change. Current information on the status of the specification, as well as corrections to the inevitable typos in the book, will be posted on an ongoing basis at http:www.fatdog.com/experts.html.