Historical Perspective

It is hard to believe that it has been ten years since I was offered a job to work at Microsoft to support its new SQL Server Windows NT product. I never thought I would find myself or our product in the position we are today. My previous experience was as a C programmer and database developer on UNIX systems mainly working with Oracle and Ingres. My perception of Microsoft was purely as a desktop company. The only database I had even seen on a PC was dBase. As I contemplated the job offer, I was naturally skeptical. How could Microsoft even create a product that could compete with the biggest names in the database industry? Fortunately for me, Andrea Stoppani, the director of SQL Support for Microsoft in 1993, convinced me that not only would Microsoft and SQL Server be successful but also that I would find a rewarding career with Microsoft because I would have an opportunity like never before: to train, learn, debug, and dig into the internal “nuts and bolts” of a relational database engine.

Ten years later, that promise has held true. In my role as an escalation engineer at Microsoft, I've had to train, learn, debug, and dig into the internal mechanics of the engine that drives SQL Server. Because of that gained knowledge, I've been asked to advise and provide feedback and insight to the development team with each new release. Through these years, I've witnessed an evolution and revolution with this product, from the early years of supporting SQL Server 4.20 for OS/2 when customers on single processor machines were limited to 16MB of RAM to the enterprise-ready, TPC record-breaking SQL Server 2000 Enterprise Server running on a machine with 64CPUs and 512GB of RAM.

The engine itself has clearly evolved from its early origin. The storage engine and query processor for SQL Server 4.20 through SQL Server 6.5 were all based on the original architectural design that came from the port of the Sybase engine on OS/2. During these years, remarkable changes and additions were performed to make the engine run faster and become a reliable, affordable platform for many users looking to deploy database systems. However, these efforts ultimately reached their limits. This is why SQL Server 7.0 was so significant. Microsoft attracted some of the leading developers in the database industry to design and implement a new architecture for the engine, a foundation to build on for years to come.

The changes and evolution have not just been with the SQL Server engine. In fact, in my early years of supporting SQL Server, the engine was the primary focus of the job because the product was pretty much the engine, sqlservr.exe. The development community back then focused its efforts on Visual Basic or C applications using DB-Library communicating over named pipes or IPX/SPX. ODBC was just an idea on Kyle Geiger's computer. Today, it is more common for Microsoft support engineers to deal with multitiered Web-based applications supporting online business retail applications with thousands of users all communicating over TCP/IP. The mind-set of supporting “just the engine” no longer applies. The SQL Server product has expanded to include a rich framework of data services including Multi-Server Job Scheduling, Data Replication, XML, Data Transformation Services, and Notification Services.

As I reflect on the changes to the engine and the core additions that have made it such a popular product, I think about the common questions I get from customers and other Microsoft employees: “How can I learn more about what makes SQL Server so powerful? How can I gain expert knowledge of some of the internals of the SQL Server engine in order to maximize the usage of the product?” My answer is always, “Think like a programmer.” To be more specific, “Think like a Windows programmer.”

I have learned the importance of gaining a solid understanding of the foundation of technology that the engine uses to perform its work. This includes a range of Windows programming topics such as processes, threads, synchronization, asynchronous I/O, dynamic linked libraries, virtual memory, networking, and COM. Regardless of the various SQL Server releases over the years, learning these topics has been essential to my understanding of the internals of the product. Learning these topics takes much more than just reading about them. You must apply the knowledge and truly understand the meaning behind the concepts. Don't just read about what structure exception handling is—understand why it has become an important feature for the SQL Server engine to use. Part I of this book can help guide you toward that goal. It provides concise, comprehensible coverage of Windows programming fundamentals. But don't just read those chapters. Go through the examples and be sure you understand the answers to the questions in each chapter. Once you master a solid knowledge of these concepts, you will have the right foundation and frame of mind to understand Part II of this book, which covers the internals of the core components that make up the SQL Server engine. Armed with this information, you will be able to broaden and round your skills by understanding the technologies covered in Part III that complement the engine and provide the complete database services product that SQL Server has become.

SQL Server has grown as a technology and as a force in the database industry. The number of high-quality books on this product alone is a leading indicator. When I started at Microsoft in 1993, there were no books on Microsoft SQL Server (and only one was produced within the next year). Today you can search the Web or go to your local bookstore and find dozens of books dedicated to this product ranging from topics on performance tuning to database administration to XML. The development of this book is a testament to the product's success. The book seeks to expand the knowledge of important topics about SQL Server in order to broaden the level of expertise worldwide. With knowledge there is power, and this book is about empowering SQL Server users, developers, and administrators to get the most out of the product.

Bob Ward
June 2003

Bob Ward joined Microsoft in 1993 as a support engineer for Microsoft SQL Server. He is currently an escalation engineer in SQL Server Support.