Microsoft Office 2007 significantly changed the way developers look at Office files. In previous versions, Office files were stored in a proprietary binary format that captured serializations of in-memory data structures. The advantages of this technique were that it conserved space within the file and increased the speed at which the Office application could parse its contents. With Office 2007, Microsoft switched the default file format to an open, XML-based standard. This standard (called Office Open XML or just Open XML) was an output of Ecma International's Technical Committee 45; it was approved by the Ecma General Assembly on December 7, 2006 and became an ISO standard in April 2008. Since then, many companies have announced support for the Open XML format, including Novell's OpenOffice and Corel's WordPerfect Office X3 suite.

Within Ecma, Technical Committee 45 took on the responsibility to standardize Office Open XML so that it can be used by other productivity applications and platforms. This committee is responsible for the standard, its documentation, and the comprehensive set of schemas. The committee includes representatives from Apple, Barclays Capital, BP, The British Library, Essilor, Intel, Microsoft, NextPage, Novell, Statoil, Toshiba, and the United States Library of Congress. To read more about this committee, including updates on their activities, visit its web site: http://www.ecma-international.org/memento/TC45.htm.

So why the change? One of the most interesting trends over the past few decades is the pace at which we are generating electronic information. This trend becomes disturbing when you realize that with a binary file format, the information is strongly tied to the specific versions of applications that generated them. With the long document retention policies many companies have, it becomes more difficult to maintain these documents reliably. Imagine if a document as important as the Declaration of Independence were written using Microsoft Word 95. For this reason, organizations and governments have been requesting that the technology industry reduce the dependencies of files on specific versions of specific applications by switching to an open format.

Of course, the new file format had to do more than fix such dependencies moving forward; it also had to address the billions of documents that already exist. This was an important goal of the Open XML initiative. The schemas for the format were designed to reflect existing documents. Microsoft has several efforts that will help in terms of adoption and backward compatibility. The Office Migration Planning Manager (OMPM) contains a set of tools to help organizations scan their documents and identify migration issues. One of the important tools in this set is the Office File Converter, which enables you to convert large quantities of files that use the binary format to the XML one. You can find more information about OMPM at http://technet.microsoft.com/en-us/library/cc179179.aspx.

Microsoft also provides an extension to older versions of Office that enables people to work with Open XML–formatted files. This extension is called the Microsoft Office Compatibility Pack, and it lets users open, edit, and save files in the Open XML format using Microsoft Office 2000, Office XP, or Office 2003. This is important because organizations often take more time to update their standardized desktops than to deploy new application servers. It means that developers will be able to use the new format on the server and in custom development projects even if the end user does not have Office 2007 or 2010. This add-on can be obtained at http://www.microsoft.com/downloads/details.aspx?FamilyId=941B3470-3AE9-4AEE-8F43-C6BB74CD1466&displaylang=en.

What does the Office Open XML file format mean to developers? It opens the possibility of generating or modifying an Office file from code without having to automate the Office application. Previously, developers would have to code against the Office application's API. When this was then run on a server, it was equivalent to a user launching the application and performing the necessary steps. These solutions did not scale well and Microsoft backed off of supporting them. However, with the Open XML format, Office files can be manipulated by modifying their XML structures. The Office application does not even have to be installed on the server. Even in Ecma's Open XML white paper (referenced in the "Further Reading" section of this chapter), the committee anticipated what developers would do with the format. Some of the new applications called out in the document were

The best way to understand the Open XML file format is to make a copy of an existing file and explore how it is put together. Whether the file is a spreadsheet, a presentation, or a document, it is really a package of parts and items. Parts are pieces of content for the file, whereas items are metadata describing how the parts should be assembled and rendered. Most of these components are XML, making it possible for them to be manipulated through code. You can gain insight into the structure of an Open XML–based file by replacing its file extension with .zip since the file is really an ordinary Zip archive. We will use a PowerPoint presentation as an example to illustrate the structure of an Open XML file. Follow along by creating your own presentation with a few slides of content. Figure 4-1 shows the root of the archive for a PowerPoint presentation.

Figure 4-1. Examining the archive of a PowerPoint presentation

The XML file in the root is named [Content_Types].xml and it stores content-type directives for all the parts that appear in the archive. A content type contains metadata about a particular part or groups of parts and, more importantly, contains a directive as to how the application should render that part. For example, Listing 4-1 shows just a few lines from the file, but it clearly delineates how this file tells the rendering application which parts are slides, images, note pages, etc.

<Override PartName="/ppt/slides/slide1.xml"
  ContentType="application/vnd.openxmlformats-
   officedocument.presentationml.slide+xml" />
<Default Extension="jpeg"
  ContentType="image/jpeg" />
<Override PartName="/ppt/notesSlides/notesSlide1.xml"
  ContentType="application/vnd.openxmlformats-
   officedocument.presentationml.notesSlide+xml" />

Of the parts in Listing 4-1, the slide parts are the most interesting to us. They store the presentation's content described using a markup language specific to presentations. This language is named PresentationML. It is XML-based and one of the easier Open XML file formats to understand because presentations have an obvious composition. A presentation is made up of the following:

Open the presentation.xml file that is located in the ppt folder of the archive. Viewing this file in Internet Explorer displays XML similar to what's shown in Figure 4-2.

Figure 4-2. The presentation.xml file

The presentation.xml file contains the list of slides (<p:sldLst>), references to the slide and notes masters, and the sizes of the slides (<p:sldSz>) and notes pages (<p:notesSz>). The sequence of the slide nodes is important; the rendering application will display them in the order they are listed here. Notice that each slide node, as well as the master nodes, has an r:id attribute. This identifier is a key to a specific relationship that will tell the rendering application where the part for this item is located. The relationship information for this presentation is stored in the presentation.xml.rels file located in the ppt\_rels folder of the archive. Listing 4-2 is the XML node in this file that tells the application that the first slide's content is stored in slide1.xml.

<Relationship
Id="rId3" Type="http://schemas.openxmlformats.org/officeDocument/

   2006/relationships/slide" Target="slides/slide1.xml"
/>

Following the relationship links, you can see that the contents of the slides are stored in the pptslides folder of the archive. When building solutions that work with the Open XML file formats, it is often useful to explore the archive to examine its contents before trying to manipulate it through code. In fact, you could extract these contents to a folder, modify them, and repackage them to a new Zip archive. Renaming this archive with a .pptx extension creates a new presentation, allowing you to see the changes in PowerPoint.

Within a slide, there are shapes marked up as <p:sp> in the XML. A shape could be the slide's title, a bulleted list of text, or even a table. Shapes are grouped into trees that you'll see as <p:spTree> elements in the XML. Figure 4-3 shows a sample slide that has a shape containing a bulleted list of three items.

Figure 4-3. A sample slide's XML

WordprocessingML is the Open XML markup language for documents. It defines a document as a collection of stories. Stories within a document can refer to the main document contents, its glossary, a header, or its footer. By examining the main document, document.xml, you'll find a body that contains paragraphs, each of which contains a collection of runs. A run is a continuous section of text that has the same properties. The text itself is stored within a text range, which is restricted to containing text only and can't have additional formatting, line breaks, tables, or graphics. The text range gets its formatting from the run or paragraph it is contained in. Figure 4-4 shows a portion of a WordprocessingML file. Notice the paragraph delimiters, the runs, and the text ranges.

Figure 4-4. Sample WordprocessingML

SpreadsheetML files are defined by a workbook part. Like a PowerPoint file whose presentation is made up of slides, workbooks are made up of worksheets. The workbook part includes information such as the file version and its collection of worksheets. Within a worksheet part, you'll find the sheet's data, containing rows of cells that have values. Figure 4-5 shows a simple spreadsheet we created to explore its Open XML structure.

Figure 4-5. A simple Excel spreadsheet

Figure 4-6 shows a portion of the markup. Notice how the sheetData element contains row elements. Within each row element, there are cells (<c>) that have values (<v>). The reason you don't see the company names is that they are stored separately in a sharedStrings.xml part. This separation enables reuse since the file needs to store a string value only once, regardless of the number of cells that contain it.

Figure 4-6. Sample SpreadsheetML

Working with these large XML files can be a burden. To make it easier, download the schemas for the Open XML file formats so that Visual Studio's IntelliSense can help you create valid structures as you're modifying the files. The schemas are available for download at http://www.microsoft.com/downloads/details.aspx?familyid=15805380-F2C0-4B80-9AD1-2CB0C300AEF9&displaylang=en.

To let you build .NET applications that operate on Open XML files, Microsoft added a namespace to the .NET 3.0 Framework. The namespace is System.IO.Packaging and its classes are contained in the WindowsBase.dll assembly. You may have this assembly in your references dialog box in Visual Studio; if not, you can browse to it at C:Program FilesReference AssembliesMicrosoftFrameworkv3.0WindowsBase.dll. Code that acts on Open XML files works much as you would if you were editing the file by hand. First the package must be opened and the file's main part located. In PowerPoint, the main part is the presentation.xml file. For spreadsheets, the main part is workbook.xml and for documents, it is document.xml. Once the main part is found, the relationship information helps the code locate the XML part with the content it wants to operate on. When this part is located, the XML part is loaded into an XML document, where it can then be read, manipulated, and saved back.

Fortunately, you won't have to do a lot of coding against the raw XML, an approach that was rather tedious and prone to error. To find areas of the files, you had to rely on XPath queries; and when you did make a change, there was no validation of your modified XML. To avoid these difficulties, we will leverage the Open XML Format SDK 2.0, which provides an object model that more closely represents the familiar objects of Office files. We will cover this SDK in the next section.

In addition to allowing you to modify or generate documents using the different markup languages, the Open XML standard supports having a custom XML folder that is capable of storing information. Within this folder, you or your applications can store your own XML data islands. Though all of the Office 2007 and 2010 files can store custom XML, Microsoft Word 2007 and 2010 have the added benefit of having a set of controls that can bind to the custom XML parts, displaying the data inline with the document. These controls are called content controls and they can be used to display data in the custom XML part, as well as serve as an input device for modifying the values there.

Several solutions in this book leverage the benefits of the Open XML file format. Chapter 6 works with Open XML Word documents and shows how to add content controls to the document to allow users to interact with its contents. The solution in Chapter 7 facilitates a document assembly scenario which also uses content controls that are placed in the document by a custom task pane. Chapter 9 leverages PresentationML to dynamically build a PowerPoint presentation by merging SharePoint list content with a presentation template.

As you can see, working with the XML of Office files could involve a lot of XML manipulation. Once exposed to the format, developers usually raise several questions. How can I be sure my injected XML is valid? What about variations of the file format, since it continued to evolve after being released with Office 2007? Is there any better way of finding my way around than XPath queries? Isn't Microsoft going to make this any easier?

The answers to all of these questions are enhancements provided by the Open XML Format SDK 2.0 (which is used heavily in the solutions in Chapters 6, 7, and 9). You can download and install the SDK from http://msdn.microsoft.com/en-us/library/bb448854(office.14).aspx. The SDK includes an assembly, DocumentFormat.OpenXML, which provides an object model of strongly typed classes that are an abstraction from the actual XML markup, making your code easier to read and write. Listing 4-3, for example, takes just a few lines of code to create a new Microsoft Word document with a paragraph of text.

using (WordprocessingDocument doc = WordprocessingDocument.Create

        ("test.docx", WordprocessingDocumentType.Document))
{
  // Add main document part.
  doc.AddMainDocumentPart();

  // Create the Document DOM.
  doc.MainDocumentPart.Document =
      new Document(
        new Body(
          new Paragraph(
            new Run(
              new Text("Hello World!")))));

    // Save changes
    doc.MainDocumentPart.Document.Save();
}

It is important to realize that you still need to understand what the Open XML file format is all about in order to use the SDK. The classes it provides make it easier to write the code, but the underlying relationships of the package, its parts, and the content remain. This is why we took the time in the previous section to give you an overview. The SDK does reduce a lot of complexity, and as long as you refrain from writing code that modifies the XML directly, your changes will still be valid Open XML files. The classes provided by the SDK also support LINQ, so searching through parts to find specific elements is also easier than writing XPath queries. Your code will be more in the style of LINQ to XML. For example, the code in Listing 4-4 shows you how to query to find all tables within a slide of a PowerPoint presentation.

var tables = from table in slidePart.Slide.Descendants<Drawing.Table>()
                 select table;

In addition to the object model, the Open XML Format SDK provides a tool to assist developers working with the Open XML file format. The Open XML SDK 2.0 Productivity Tool for Microsoft Office is installed by default at C:Program Files (x86)Open XML Format SDKV2.0 ool. This tool can perform several functions. We will focus here on its ability to generate code. When you open an Open XML-based file, you can browse the structure of the file. Selecting a specific item shows you the XML for that portion of the file, as well as a generated function for creating that part of the file. We will use this repeatedly in the chapters where we manipulate Open XML files. Figure 4-7 shows this tool being used to create a function that creates a paragraph node for a bulleted list.

Figure 4-7. Using the Document Reflector tool

Microsoft Office is one of the world's most-used information-worker applications. That level of exposure and recognition has made these tools an enticing area for customization as developers like to extend a tool that users are already comfortable with. Developing with Microsoft Office has traditionally meant developing in a COM-based world. This began to change with Office 2003 when Microsoft shipped primary interop assemblies (PIAs) as part of the advanced installation. These PIAs opened Office to the .NET developer community and their managed code projects. A primary interop assembly enabled a Visual Studio developer to write code against the Office applications by adding a reference in the project. The assembly took on the responsibility of translating between the COM and managed code environments. Even with the PIAs, such development was not for the faint of heart. .NET developers often struggled—their code had to deal with the way Office maintained object lifetimes and they had to contend with lots of COM plumbing. Moreover, the managed code solution was limited to running outside the process of the Office application. What was missing was a layer between the PIAs and the developer's custom application. Microsoft Visual Studio Tools for Office (VSTO) fills that void. VSTO extends the Visual Studio environment to support the development of managed code solutions for Microsoft Office. With VSTO, the developer can create solutions that leverage Microsoft Office applications, including the construction of add-ins, custom task panes, action panes, ribbon customizations, and smart documents.

The first version of VSTO was called Visual Studio Tools for Office 2003. This version was an add-on to Visual Studio .NET 2003 and it gave developers project types that let their code run in process with Word 2003 and Excel 2003. These project types let developers create applications by building on top of documents. You could select your project type as an Excel workbook, Word document, or Word template. By writing their customizations in the managed-code world, developers could leverage the power of the .NET languages C# and VB.NET. These languages were more appealing than VBA and provided powerful access to manipulate XML and connect to data stores through ADO.NET, plus support for web services. By running within Office, the resulting assembly could interact with application- and document-level events, command bars, and the document itself. An important limitation of these solutions was that they were not compatible with Office Standard Edition 2003 because some of the necessary elements, such as XML and smart documents, were not supported.

Over time, new versions of the Office, the .NET Framework, Visual Studio, and the VSTO runtime appeared and each improved the developer experience, bringing it closer to the experiences of developers using Visual Studio for web and Windows development. First appearing in Visual Studio 2005, VSTO literally brought Microsoft Word and Excel into Visual Studio and presented them as a designer surface (known as a document-level project or smart document). This continues today. As Figure 4-8 shows, when a developer is working on a workbook solution, Excel appears just as a form surface would to a Windows developer.

Figure 4-8. Microsoft Excel as a visual design surface

With the application as a design surface, the developer could drag Windows forms controls from the Visual Studio toolbox directly onto the spreadsheet or document. Double-clicking a button control took the developer to an event handler in a code-behind file. This made the Office development experience similar to that of Windows forms or ASP.NET, and gave Office developers a rapid application-development environment. In addition to the design surface, developers also gained access to application-level add-ins, task panes, smart tags, and the Ribbon interface, as well as expansion into other Office applications such as Visio and Outlook. However, with each release, a tighter relationship developed between Visual Studio, the .NET framework, Office, and the VSTO runtime. This means that as a developer, you need to be aware of dependencies on the applications you are writing.

Add-ins are defined by Microsoft Office Help as "supplemental programs that you can install to extend the capabilities of Microsoft Office by adding custom commands and specialized features." These programs run at the application level and are not tied to a specific document, spreadsheet, or presentation. In fact, the program can be loaded when the Office application starts, before any of these files are opened. Figure 4-9 shows the New Project dialog in Visual Studio 2010.

Figure 4-9. Creating an Office add-in project

Every add-in project includes a ThisAddin (.cs or .vb, depending on the language) file. This file includes the ThisAddin class, which is the root of your application. Within this class you can respond to events such as startup and shutdown, where you can include initialization and clean-up commands. Listing 4-5 shows some sample code from a ThisAddin class that sets up a task pane. This task pane is a user control that loads within the Office application, allowing a developer to display a custom user interface.

public partial class ThisAddIn
{
  public CustomTaskPane ctp;
  private void ThisAddIn_Startup(object sender, System.EventArgs e)
  {
    ctp = Globals.ThisAddIn.CustomTaskPanes.Add(new ucTaskPane(), "Custom Briefing");

ctp.DockPosition = Office.MsoCTPDockPosition.msoCTPDockPositionRight;
    ctp.Width = 250;
  }
 private void ThisAddIn_Shutdown(object sender, System.EventArgs e)
 {
 }

With just a quick search on the Web, you can find a wide variety of Office add-ins for removing duplicate contacts in Outlook, creating surveys, redacting text in Word documents, and more. In this book, the solution in Chapter 9 creates a PowerPoint 2010 add-in to build presentation slides from the content in a SharePoint site. In Chapter 10, Outlook 2010 is extended, enabling customer data to be maintained as Outlook contacts, along with data visualizations of order trends.

Before Office 2007, developers spent most of their efforts writing code utilizing the CommandBars object to create new buttons, extend menus, hide existing options, and make other user interface customizations. Microsoft Office 2007 introduced a new user interface for organizing these commands, called the Ribbon. The Ribbon occupies the top portion of most Office applications and provides a different experience from the layered menus and toolbars of previous Office releases. Figure 4-10 shows Microsoft Word 2010's Ribbon.

Figure 4-10. The Ribbon in Microsoft Word 2010

This interface provides a simpler experience for the user by improving the organization of commands and displaying a more graphical representation of options. One of the primary goals of the Ribbon interface is to surface functionality that users wanted but didn't know was there because of how buried or obscure that menu item was. The Ribbon also provides context sensitivity. That is, it will highlight certain tabs depending on the object you're focused on. For instance, the table tools are brought to your attention when you are in a table in a Word document. The controls on the Ribbon are broken up into tabs (Home, Insert, Page Layout, and so forth) and groups (Clipboard, Font, Paragraph, etc.).

VSTO opens the Ribbon interface to customization by the developer. In a project, you can select a custom ribbon from the Add New Item dialog of Visual Studio, as shown in Figure 4-11. There are two possible choices for building ribbon customizations: using a designer and working with the ribbon XML markup. The designer is adequate for most common scenarios and is used in Chapter 8. The XML markup option is useful when you need access to functionality that is not supported by the designer, which includes several 2010-specific features that we'll describe later. This approach is used in Chapter 7.

Figure 4-11. Adding a custom ribbon

The concept of adding panes inside Office applications was introduced in Office 2003. The panes display in the same window as the application, usually off to the right-hand side of the document, e-mail, spreadsheet, or presentation. These panes can be scoped to the current open file or at the application level. When the pane is scoped to a file, we refer to it as an actions pane. When it is scoped to the application, we refer to it as a task pane. Actions panes are supported as part of the document-level projects. Task panes are associated with add-in projects. Panes provide an interface for the user to interact with, where the result usually has some impact on the user's current document. Office uses the pane interface for many tasks, such as browsing for clip art, searching through the Research pane, or displaying SharePoint site data. For developers, panes can be a method of extending the Office application with your own interface and process.

Office document-level projects can include custom actions panes. Fortunately, VSTO does most of the heavy lifting and provides you with an actions pane item template that can be added to your project. Figure 4-12 shows a sample actions pane control; the code necessary to load it is shown in Listing 4-6.

Figure 4-12. Constructing an actions pane control

public partial class ThisWorkbook
{
   ActionsPaneControl1 paneCtl1 = new ActionsPaneControl1();
   private void ThisWorkbook_Startup(object sender, System.EventArgs e)
   {
      this.ActionsPane.Controls.Add(paneCtl1);
      this.ActionsPane.Show();
   }

Once the control is loaded as an actions pane, it can interact with the host application (Excel) through Globals.ThisWorkbook. By placing code behind the control, you can manipulate the application or its document. For example, the code in Listing 4-7 responds by placing the selected date in the A1 cell of the current worksheet when the user clicks the Insert button.

private void btnInsert_Click(object sender, EventArgs e)
{
   if (Globals.ThisWorkbook.ActiveSheet != null)
   {
      Microsoft.Office.Interop.Excel.Worksheet current = null;
      string val = monthCalendar1.SelectionRange.Start.ToShortDateString();
      current = (Microsoft.Office.Interop.Excel.Worksheet) 

                Globals.ThisWorkbook.ActiveSheet;
      current.Range["A1"].Value2 = val;
   }
}

Figure 4-13 shows the actions pane loaded to the right of the spreadsheet and the selected date inserted into cell A1.

Figure 4-13. The completed actions pane

Actions and task panes allow developers to inject their own interface and code into Office applications. The pane becomes like its own helper application to complete the document. PowerPoint 2010 is extended with a task pane in Chapter 8 so that slides can be built from SharePoint site content. The solution presented in Chapter 4 extends Microsoft Word 2010 so that a SharePoint people search can be performed to insert résumé requests into the document.

In addition to add-ins, ribbons, and task panes, Outlook 2007 introduced extensions through a new user interface element called form regions. Before Outlook 2007, developers who wanted to customize an Outlook form usually found themselves redesigning the entire form, even when all they wanted to do was add a few fields. In older versions of Outlook, these custom forms were deployed centrally on the Exchange server and often took too long to open. Developers also had a difficult time injecting their own code or script into the form. Enter Outlook form regions, which enable you to insert your own custom user interface into Outlook's inspectors. Do not confuse this approach with sending InfoPath forms through e-mail. Form regions are used to extend the user interface of Outlook.

In Visual Studio 2010, an Outlook form region is an item hosted by an Outlook add-in. Once you have an Outlook add-in project, you can create the form region in two different ways. As the dialog shows in Figure 4-14, when you add an Outlook form region item to the project, you can choose to design a new form in Visual Studio or import a form region that was designed in Outlook and exported as a Outlook Form Storage (OFS) file. Choosing a new form region lets you specify whether your region should appear separately from the original form or merge with it, what type of Outlook items should be matched with your form, and whether it should show up in read or compose scenarios.

Figure 4-14. Adding a new region in Visual Studio

Once the form region is created, you can write code in it much as you would for any user control. Figure 4-15 shows a the custom form region you will create in Chapter 10, which is configured to display a chart of sales data for customers and allows the user to submit follow-up tasks, all from an Outlook contact form. This form region is accessed using the Show group in the ribbon.

Figure 4-15. A custom form region displayed for customer contacts

Document-level projects are solutions in which the application is a Word document or Excel spreadsheet and the technique used is often referred to as a code-behind document or a smart document. The difference between this type of solution and an add-in is that you are not just extending an Office application, but rather you're creating a tool from a specific file. The document or spreadsheet starts out with all of the traditional Office functionality and the developer can then drag new controls onto its surface, add custom code, add user-interface elements to an actions pane, and even cache data for offline access. Usually this type of solution is implemented to build a tool that helps an information worker create a complex document. Sales proposals and contracts whose contents likely come from many corporate resources are typical candidates for smart documents.

As shown earlier in Figure 4-8, Visual Studio Tools for Office provides a unique development environment for building document-level projects. When you start such a project, you have the option of creating a new file or importing an existing Word document or Excel spreadsheet. This is a useful feature for organizations that already have such assets and are looking to enhance them. Once in the designer, Windows form controls can be dragged onto the document's surface, and event-handler code can be placed in the code-behind. These projects also support data-binding the controls and caching of datasets for offline access. The result is an Office document with a code-behind assembly. Do keep in mind that the assembly is not contained within the document even if it is one based on the Open XML file format. Instead, the document includes a manifest that contains a pointer to where the code-behind assembly can be retrieved. This assembly location must be trusted by the code access security policies of the client. We will see this technique in action in Chapter 7 where we build a proposal tool out of a Microsoft Word 2010 document.