Every list contains special fields named Title, LinkTitle, and LinkTitleNoMenu. The Title field contains text that represents the list item. The LinkTitle field and the LinkTitleNoMenu field are computed fields based on the value in the Title field and cannot be edited. The LinkTitle field is used to render the Edit Control Block (ECB) menu shown in Figure 9-1.

This field is special for lists because it contains the value that is rendered inside the Edit Control Block (ECB) menu shown in Figure 9-1. Though the Title field contains the value rendered inside the ECB menu, it is not actually the field that SharePoint uses to create the ECB menu within a view. Instead, SharePoint uses a related field named LinkTitle, which reads the value of the Title field and uses it to render the ECB menu. The LinkTitleNoMenu field can be used to display the value of the Title field in a hyperlink that can be used to view an item.

Figure 9-1. The Edit Control Block menu

Each field in a list has an internal name as well as a display name. After a field has been created, its internal name can never be modified. However, the display name can be modified. For example, the out-of-the-box Contacts list modifies the display name of the Title field to “Last Name”. As a result, the Last Name column provides the ECB menu in a Contacts list.

The display name of the Title field can be modified directly in the browser or through code. As an example, imagine that you need a list to track product categories, so you create a new list by using the custom list template. The SPList class provides a Fields collection, from which you can retrieve the Title field. After you have obtained an SPField reference to the Title field, you can modify its display name by using the Title property of the SPField class and then save your changes by calling the Update method. Example 9-3 shows how to accomplish this modification by using different approaches.

//Server-Side Object Model
SPList list = SPContext.Current.Web.Lists["ProductCategories"];
SPField fldTitle = list.Fields.GetFieldByInternalName("Title");
fldTitle.Title = "Category";
fldTitle.Update();

//Client-Side Object Model
var ctx = new SP.ClientContext.get_current();
var field = ctx.get_web().get_lists().getByTitle("ProductCategories")
           .get_fields().getByInternalNameOrTitle("Title");
ctx.load(field);
field.set_title("Category");
field.update();
ctx.executeQueryAsync(success, failure);

//REST Interface
$.ajax(
  {
    url: _spPageContextInfo.webServerRelativeUrl +
         "/_api/web/lists/getByTitle('ProductCategories')" +
         "/fields/getByInternalNameOrTitle('Title')",

    type: "POST",
    contentType: "application/json;odata=verbose",
    data: JSON.stringify(
      { '__metadata': { 'type': 'SP.Field' },
        'Title': "Category"
      }),
    headers: {
      "accept": "application/json;odata=verbose",
      "X-RequestDigest": $("#__REQUESTDIGEST").val(),
      "IF-MATCH": "*",
      "X-Http-Method": "PATCH"
    }
  });

Now that we have covered the basics of fields and field types, let’s write the code required to add a few custom fields to a new list. The Fields collection supports an Add method that can easily be used to add new fields. Additionally, the AddFieldAsXml method allows you to structure an XML chunk to define the new field. These different approaches are shown in Example 9-4 using the various APIs.

//Server-Side Object Model
list.Fields.Add("ProductDescription", SPFieldType.Text, false);

//JavaScript Client Object Model
var xmlDef = "<Field DisplayName='ProductDescription' Type='Text'/>";
var ctx = new SP.ClientContext.get_current();
var field = ctx.get_web().get_lists().getByTitle("Products")
            .get_fields().addFieldAsXml(xmlDef, false,
             SP.AddFieldOptions.addToNoContentType);
field.update();
ctx.load(field);
ctx.executeQueryAsync(success, failure);

//REST Interface
$.ajax(
    {
        url: _spPageContextInfo.webServerRelativeUrl +
            "/_api/web/lists/getByTitle('Products')/fields",
        type: "POST",
        contentType: "application/json;odata=verbose",
        data: JSON.stringify(
            { '__metadata': { 'type': 'SP.Field' },
              'Title': 'ProductDescription',
              'FieldTypeKind': SP.FieldType.text
            }),
        headers: {
            "accept": "application/json;odata=verbose",
            "X-RequestDigest": $("#__REQUESTDIGEST").val()
        }
    }
);

Note that the previous code avoids using spaces when creating the field names. If you create a field with a space in its name, SharePoint creates the internal name by replacing each space with _x0020_. This means that calling the Add method and passing a name of “List Price”, for example, would create a field with an internal name of List_x0020_Price. Most developers prefer to create fields by using a name without spaces.

The properties of the field, such as DefaultValue, represent common properties that are shared across all field types. However, specific field types have associated classes that inherit from SPField. Examples of these field type classes include SPFieldBoolean, SPFieldChoice, SPFieldCurrency, SPFieldDateTime, SPFieldDecimal, SPFieldNumber, SPFieldText, SPFieldUrl, and SPFieldUser. Although the REST API lets you use an endpoint against the underlying type, when you are using the server-side or client-side object model you must convert a field to one of these specific field types. The following code shows an example:

//Server-Side Object Model
SPList list = SPContext.Current.Web.Lists["Products"];
SPFieldCurrency fld =
  (SPFieldCurrency)list.Fields.GetFieldByInternalName("ListPrice");
fld.MinimumValue = 0;
fld.MaximumValue = 10000;
fld.Update();

//JavaScript Client Object Model
var ctx = new SP.ClientContext.get_current();
var field = ctx.get_web().get_lists().getByTitle("Products")
            .get_fields().getByInternalNameOrTitle("ListPrice");
var fieldAsNumber = ctx.castTo(field, SP.FieldNumber);
fieldAsNumber.set_minimumValue(0);
fieldAsNumber.set_maximumValue(10000);
fieldAsNumber.update();
ctx.load(field);
ctx.executeQueryAsync(success, failure);

SharePoint Foundation supports lookup fields, which make it possible for users to update a field value by using a drop-down menu populated by the field values in another list. For example, imagine that you want to add a field to a list of products that lets the user pick a product category. You can accomplish this by adding a new lookup field to the products list. Example 9-5 shows how to do this by using both the server-side and client-side object models.

//Server-Side Object Model
string LookupFieldDisplayName = "Category";
SPList LookupList = site.Lists["Product Categories"];
Guid LookupListID = LookupList.ID;
list.Fields.AddLookup(LookupFieldDisplayName, LookupListID, true);
SPFieldLookup fldLookup = (SPFieldLookup)list.Fields["Category"];
fldLookup.Indexed = true;
fldLookup.RelationshipDeleteBehavior = SPRelationshipDeleteBehavior.Restrict;
fldLookup.Update();

//JavaScript Client Object Model
var ctx = new SP.ClientContext.get_current();
var xmlDef = "<Field DisplayName='Category' Type='Lookup'/>";
var field = ctx.get_web().get_lists().getByTitle("Products").get_fields()
           .addFieldAsXml( xmlDef, false, SP.AddFieldOptions.addToNoContentType);
var lookupField = ctx.castTo(field, SP.FieldLookup);
lookupField.set_lookupList({GUID});
lookupField.set_lookupField("Title");
lookupField.set_relationshipDeleteBehavior(SP.RelationshipDeleteBehavior.restrict);
lookupField.update();
ctx.executeQueryAsync(success, failure)

When a list is used as a lookup, it can have a relationship that allows you to prevent users from deleting any item in the lookup list that is currently being used by any item in the master list. This is accomplished by setting the RelationshipDeleteBehavior property to restricted delete. Under this setting, attempting to delete an item from the lookup list that is in use will then result in an error. In some scenarios, however, it makes more sense to create a relationship with cascading delete behavior. When a user deletes an item in the lookup list under this setting, all the list items that have been assigned that lookup will be deleted as well.

Though there is definite value in the support for defining list relationships, it is important that you don’t overestimate what it really does. It cannot be used to create the same level of referential integrity that can be achieved between two tables in a Microsoft SQL Server database. That means that it does not really prevent the possibility of orphaned items, such as when you have products that are not assigned to an existing product category. The value of creating the relationship with delete behavior simply relieves you of additional development work, such as writing an event handler to achieve something such as cascade delete behavior.

The SPWeb class provides a ContentTypes collection that makes it possible to enumerate through the content types in the content types gallery of the current site. Following the same pattern as for site columns, you can also use a second collection named AvailableContentTypes, which allows you to enumerate the aggregation of content types in the content types gallery of the current site and all parent sites. Example 9-9 shows how to retrieve a content type when you know its name or ID.

//Server-Side Object Model
SPWeb site = SPContext.Current.Web;
SPContentType ctype = site.ContentTypes["WingtipToysProduct"];

//JavaScript Client Object Model
var ctx = new SP.ClientContext.get_current();
var ctype = ctx.get_web().get_contentTypes().getById("0x010100F51AEB6BBC8EA2469E1617071D9
FF658");
ctx.load(ctype);
ctx.executeQueryAsync(success, failure);

//REST Interface
$.ajax(
{
    url: _spPageContextInfo.webServerRelativeUrl +
         "/_api/web/contentTypes/getById('0x010100F51AEB6BBC8EA2469E1617071D9FF658')",
    type: "GET",
    headers: {
        "accept": "application/json;odata=verbose",
    }
});

When would you want to access a content type through the ContentTypes collection rather than the AvailableContentTypes collection? The first observation is that the line of code that accesses the content type through the ContentTypes collection will fail if it is ever executed within the context of a child site. Therefore, accessing content types with the AvailableContentTypes collection is more flexible if you need to write code that might execute in the context of a child site. This behavior is similar to the behavior of site columns.

However, another important point is that a content type that has been retrieved through the AvailableContentTypes collection is read-only and cannot be modified. Therefore, you must retrieve a content type by using the ContentTypes collection inside the context of the site where it exists if you need to modify it. Remember that all the standard SharePoint Foundation content types are added to the content types gallery of each top-level site.

Each content type is made up of a set of site columns. Although each content type defines a collection of site columns, it doesn’t just track a collection of site columns by using SPField objects as you might expect. Instead, it tracks each site column by using an SPFieldLink object. You can think of an SPFieldLink object as a layer on top of an SPField object that the content type can use to customize default property values of the underlying site column.

Consider a scenario in which you might want to add the standard site column named Company to an existing content type. This is accomplished by adding a new field to the collection of field links. Example 9-10 adds the site column to an existing content type, and then pushes the change to every list and document library that uses it.

//Server-Side Object Model
SPContentType ctype = site.ContentTypes["WingtipToysProduct"];
SPField companyField = site.Fields.GetFieldByInternalName("Company");
ctype.FieldLinks.Add(new SPFieldLink(companyField));
ctype.Update(true, false);

//JavaScript Client Object Model
var ctx = new SP.ClientContext.get_current();
var field = ctx.get_site().get_rootWeb().get_fields()
           .getByInternalNameOrTitle("Company");
var ctype = ctx.get_site().get_rootWeb().get_contentTypes()
           .getById("0x010100F51AEB6BBC8EA2469E1617071D9FF658");
ctx.load(ctype);
ctx.load(field);
var createInfo = new SP.FieldLinkCreationInformation();
createInfo.set_field(field);
var fieldLink = ctype.get_fieldLinks().add(createInfo);
ctype.update (true);
ctx.load(fieldLink);
ctx.executeQueryAsync(success, failure);

Note that the call to Update in the server-side code passes a second parameter with a value of false. This second parameter requires some explanation. There are some content types in the standard set of content types added by SharePoint Foundation that are defined as read-only content types. If you do not pass a value of false for the second parameter, the call to Update fails when it attempts to modify one of these read-only content types. When you pass a value of false, the call to Update succeeds because it ignores any failures due to the inability to update read-only content types.

You will never create a content type from scratch. Instead, you always select an existing content type to serve as the base content type for the new content types you are creating. For example, you can create the most straightforward content type by inheriting from the content type named Item. This automatically provides your new content type with the standard fields and behavior that are common across all content types. Alternatively, you can elect to inherit from another content type that inherits from the Item content type, such as Announcement, Contact, Task, or Document. Beyond creating a new content type manually by using the browser, you can also do it declaratively with Collaborative Application Markup Language (CAML), or programmatically.

Adding a new content type declaratively is straightforward when you are using Visual Studio 2012. Simply select the option to add a new item to your app or solution and select Content Type. A wizard then walks you through selecting from the available content types from which to inherit. You can then easily add existing site columns to the content type. If you need a custom site column, it can also be added as a new item. The end result is an Elements.xml file containing the declarative definition of the new content type, as shown in the following code:

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <!-- Parent ContentType: Document (0x0101) -->
  <ContentType ID="0x010100FD82D89E067A4B0C8D0E7474FA662E70"
               Name="FinancialDocument"
               Group="Financial"
               Description="Financial Content Type"
               Inherits="TRUE"
               Version="0">
    <FieldRefs>
      <FieldRef ID="{a93bbb3e-2a86-4763-bfb5-9c86849d1630}"
                DisplayName="Amount"
                Required="FALSE"
                Name="Amount" />
    </FieldRefs>
  </ContentType>
</Elements>

To create a new content type with server-side code, you must call the SPContentType class constructor, passing three parameters. The first parameter is the SPContentType object associated with the parent content type you want to inherit from. The second parameter is the target content types collection, which should typically be the ContentTypes collection of a top-level site. The third parameter is the string name of the content type to be created. After you have created an SPContentType object and initialized its properties, such as Description and Group, you must call the Add method on the ContentTypes property of the target site to actually save your work in the content database. The following code shows how this works:

SPWeb site = SPContext.Current.Site.RootWeb;
string ctypeName = "FinancialDocument";
SPContentType ctypeParent = site.ContentTypes["Document"];
SPContentType ctype = new SPContentType(ctypeParent, site.ContentTypes, ctypeName);
ctype.Description = "A new content type";
ctype.Group = "Financial Content Types";
site.ContentTypes.Add(ctype);

When creating a new content type by using CSOM, the process is similar to the server-side example. When creating a new content type by using REST, the process uses a POST method but requires the same basic information. Example 9-11 shows each approach.

//JavaScript Client Object Model
var ctx = new SP.ClientContext.get_current();
var parent = ctx.get_web().get_contentTypes().getById("0x0101");
ctx.load(parent);
var createInfo = new SP.ContentTypeCreationInformation();
createInfo.set_name("FinancialDocument");
createInfo.set_description("A new content type");
createInfo.set_parentContentType(parent);
var newCtype = ctx.get_site().get_rootWeb().get_contentTypes().add(createInfo);
ctx.load(newCtype);
ctx.executeQueryAsync(success, failure)

//REST Interface
$.ajax({
    url: _spPageContextInfo.webServerRelativeUrl +
         "/_api/site/rootWeb/contentTypes",
    type: "POST",
    contentType: "application/json;odata=verbose",
    data: JSON.stringify({
        '__metadata': { 'type': 'SP.ContentType' },
        'Name': contentTypeName,
        'Description': description,
        'Id': {
            '__metadata': { 'type': 'SP.ContentTypeId' },
            'StringValue': createCTypeId(parentId)
               }
    }),
    headers: {
        "accept": "application/json;odata=verbose",
        "X-RequestDigest": $("#__REQUESTDIGEST").val()
        }
});

You can create a document library declaratively by creating a ListInstance element with a TemplateType value of 101 and the GUID that identifies the DocumentLibrary feature provided by SharePoint. The following code shows a sample in CAML:

<ListInstance
  TemplateType="101"
  FeatureId="00bfea71-e717-4e80-aa17-d0c71b360101"
  Url="ProductProposals"
  Title="Product Proposals"
  Description=""
  OnQuickLaunch="TRUE" />

If you prefer to create document libraries by using code, you can do so just as you do when creating a standard list. The only difference is that you pass a list template parameter with an SPListTemplateType enumeration value of DocumentLibrary. Refer to the code in the section “Creating lists” earlier in this chapter for details.

One unique characteristic of document libraries is that they provide support for document templates. For example, you can create a document template by using Microsoft Word that allows users to create documents that already contain the company letterhead; or you can create a document template using Microsoft Excel that allows users to create documents for expense reports that have a structure predefined by the accounting department. A user can then create a new document from a document template by using the New Document menu on the Documents tab of a document library.

A document library is initially configured with a generic document template. However, you can update it to utilize a custom document template. One approach is to upload the custom document template file by using a Module element. SharePoint contains a special folder named Forms inside the root folder of each document library. The Forms folder is the proper location in which to upload the document template for a document library. The following code shows the declarative CAML necessary to define a document template within a Module element:

<Module Name="ProductProposalTemplates" Url="ProductProposals" List="101" >
  <File Path="ProductProposalTemplatesProposal.dotx"
        Url="Forms/Proposal.dotx"
        Type="GhostableInLibrary" />
</Module>

The Url attribute of the Module element contains the site-relative URL of the document library, which in this case is ProductProposals. This is required so that the files inside this Module element are provisioned relative to the root of the document library instead of the root of the site. You should also notice that the Module element includes a List attribute with a value of 101, which is required to indicate that the target location exists inside the scope of a document library. The Url attribute of the File element contains a path to provision the document template file in the Forms folder. The Type attribute is configured with a value of GhostableInLibrary, which tells SharePoint to provision the template within a document library.

As an alternative to using a declarative Module, you can upload the document template programmatically. In the server-side object model, the SPFolder object provides a Files collection property with an Add method, which makes it possible to upload a copy of the document template file. When you call the Add method on a Files collection object to create a new file such as a document template, you can pass the contents of the file by using either a byte array or an object based on the Stream class, as shown in the following code:

SPDocumentLibrary DocLib = (SPDocumentLibrary)site.Lists[DocLibID];
SPFolder formsFolder = DocLib.RootFolder.SubFolders["Forms"];
formsFolder.Files.Add("Specification.dotx", {Stream Object});
DocLib.DocumentTemplateUrl = @"ProductSpecifications/Forms/Specification.dotx";
DocLib.Update();

When using the client-side object model or REST interface through JavaScript, you must create a mechanism for selecting and reading the document template into a variable. This is accomplished by using an input element of type file and the FileReader object. The following code snippet shows some HTML and JavaScript for selecting files:

<input id="inputFile" type="file" />
<input id="uploadButton" type="button" value="Upload" class="app-button"/>

$("#uploadButton").click(function () {
    var buffer;
    var error;
    var file = document.getElementById("inputFile").files[0];
    var filename = document.getElementById("inputFile").value;

    var reader = new FileReader();
    reader.onload = function (e) {
        buffer = e.target.result;
    };
    reader.onerror = function (e) {
        error = e.target.error;
    };
    reader.readAsArrayBuffer(file);
});

The input element of type file creates a control that allows users to browse for and select files. The FileReader object can read the selected files into a buffer by referencing the files collection of the control. The FileReader reads the selected file asynchronously and fires the onload event when finished. The FileReader should use the readAsArrayBuffer method, which returns an array that is compatible with both the client-side object model and REST. At this point, the buffer containing the file can be retrieved from the result property. The contents of the buffer can be used with the client-side object model to upload the file into the Forms folder, as shown in the following code:

var ctx = new SP.ClientContext.get_current();
var createInfo = new SP.FileCreationInformation();
createInfo.set_content(buffer);
createInfo.set_overwrite(true);
createInfo.set_url("template.dotx");
var files =
ctx.get_web().getFolderByServerRelativeUrl("ProductSpecifications/Forms").get_files();
ctx.load(files);
files.add(createInfo);
ctx.executeQueryAsync(success, failure);

Note that the JavaScript client object model can only upload files up to 1.5 megabytes (MB) in size. If you need to upload larger files, then you must use the REST interface. The REST interface supports uploading documents as large as 2 gigabytes (GB). The following code shows the endpoint to use for uploading with the REST interface:

$.ajax({
    url: _spPageContextInfo.webServerRelativeUrl +
         "/_api/web/GetFolderByServerRelativeUrl(' ProductSpecifications/Forms ')/Files" +
         "/Add(url='template.dotx', overwrite=true)",
    type: "POST",
    data: buffer,
    processData: false,
    headers: {
        "accept": "application/json;odata=verbose",
        "X-RequestDigest": $("#__REQUESTDIGEST").val(),
        "content-length": content.length
    }
});

After the document template is uploaded, you must write code to configure the DocumentTemplateUrl property of the document library. This code ensures that the template is used for the creation of new documents. The following code shows how to do this with server-side and client-side code:

//Server-Side Object Model
SPWeb site = SPContext.Current.Web;
SPDocumentLibrary libProposals;
libProposals = (SPDocumentLibrary)site.Lists["Product Proposals"];
string templateUrl = @"ProductProposals/Forms/Proposal.dotx";
libProposals.DocumentTemplateUrl = templateUrl;
libProposals.Update();

//JavaScript Client-Side Object Model
var ctx = new SP.ClientContext.get_current();
var library = ctx.get_web().get_lists().getByTitle('ProductSpecifications')
ctx.load(library);
library.set_documentTemplateUrl('ProductSpecifications/Forms/Proposal.dotx'),
library.update();
ctx.executeQueryAsync(success, failure);

You can create custom content types for tracking documents. This design approach provides the same type of advantages as creating content types for standard lists. For example, you can create a content type that defines a custom set of site columns for tracking document metadata and then reuse that content type across many document libraries. If you design document libraries by using custom content types, you also have the ability to add new columns to a content type and push the changes to many document libraries at once. Custom content types for document libraries generally inherit from the Document content type. One unique aspect of content types that inherit from the Document content type is that they add support for document templates.

After you have created a new content type, your next step is to upload a copy of the document template file. Uploading a file for the content type document template is done in the same way as for a document library. However, the tricky part is knowing where to upload this file. SharePoint creates a dedicated folder for each site content type in the virtual file system of the hosting site within a special folder named _cts. When you create a new content type, SharePoint automatically creates a new folder for it at _cts/[Content Type Name]. This is the location where you should upload the document template.

After you have uploaded a copy of the document template file, you can then configure the content type to use it by modifying its DocumentTemplate property. In order to properly set the document template, you must know the content type ID and the name of the file to use. Example 9-12 shows how this is accomplished for a content type named “Invoice”.

//Server-Side Object Model
SPWeb site = SPContext.Current.Site.RootWeb;
SPContentType ctype = site.ContentTypes["Invoice"];
ctype.DocumentTemplate("Invoice.docx");

//JavaScript Client-Side Object Model
var ctx = new SP.ClientContext.get_current();
var ctype = ctx.get_web().get_contentTypes().getById("0x0101adbc123")
ctx.load(ctype);
ctype.set_documentTemplate("Invoice.docx");
ctype.update();
ctx.executeQueryAsync(success, failure);

//REST Interface
$.ajax({
    url: _spPageContextInfo.webServerRelativeUrl +
         "/_api/web/contentTypes/getById('0x0101adbc123')",
    type: "POST",
    contentType: "application/json;odata=verbose",
    data: JSON.stringify({
        '__metadata': { 'type': 'SP.ContentType' },
        'DocumentTemplate': 'Invoice.docx'
     }),
     headers: {
     "accept": "application/json;odata=verbose",
     "X-RequestDigest": $("#__REQUESTDIGEST").val(),
     "IF-MATCH": "*",
     "X-Http-Method": "PATCH"
     }
});

After the content types are created and the templates defined, the content types can be used in a document library. To add new content types to a document library, the ContentTypesEnabled property must be set to true. After that, new content types can be added to the collection as shown in Example 9-13.

//Server-Side Object Model
SPWeb site = SPContext.Current.Site.RootWeb;
SPDocumentLibrary DocLib = (SPDocumentLibrary)site.Lists["FinancialDocuments"];
DocLib.ContentTypes.Add(site.AvailableContentTypes["Invoice"]);

//JavaScript Client-Side Object Model
var ctx = new SP.ClientContext.get_current();
var library = ctx.get_web().get_lists().getByTitle("FinancialDocuments");
ctx.load(library);
var ctype = ctx.get_web().get_contentTypes().getById('0x0101adbc123'),
ctx.load(ctype);
library.get_contentTypes().addExistingContentType(ctype);
ctx.executeQueryAsync(success, failure);

//REST Interface
$.ajax(
{
    url: _spPageContextInfo.webServerRelativeUrl +
         "/_api/web/lists/getByTitle('FinancialDocuments')" +
         "/contentTypes/addAvailableContentType('0x0101adbc123')",
    type: "POST",
    contentType: "application/json;odata=verbose",
    headers: {
        "accept": "application/json;odata=verbose",
        "X-RequestDigest": $("#__REQUESTDIGEST").val(),
        }
});

There is a dual aspect to programming documents in a document library. Though you can program against a document library by using an SPList object, you can also program against a document by using an SPListItem object. However, each document is also represented in the server-side object with an SPFile object. That means you can program against a document in a document library as either an SPListItem or SPFile object. You can obtain the file associated with an item through the File property by using either server-side or client-side code.

The SPListItem object can be used to read or update fields just as you would read or update fields for an item in a standard list. The SPFile object, on the other hand, can be used to control other aspects of the document, such as versioning, check-in, and checkout, as well as reading from and writing to the document’s content.

When the server-side object model is used, the contents of the document can be accessed through the SPFile.OpenBinaryStream method. When the managed client object model is used, the contents can be accessed through the File.OpenBinaryDirect and File.SaveBinaryDirect methods. When the REST API is used, the contents are accessed by referencing the file followed by the $value method, as shown here:

http://site/_api/web/GetFileByServerRelativeUrl('filepath')/$value

Many document libraries contain folders in addition to documents. Folders, like files, are stored as items within a document library and show up as SPListItem objects in the Items collection. This can be confusing if you are expecting a classic treeview structure within the library. Instead of navigating a tree, you can inspect a SPListItem property named FileSystemObjectType before attempting to process an item as an SPFile object, as shown in Example 9-14.

//Server-Side Object Model
SPWeb site = SPContext.Current.Site.RootWeb;
SPDocumentLibrary DocLib = (SPDocumentLibrary)site.Lists["FinancialDocuments"];
foreach (SPListItem item in docLib.Items) {
  if (item.FileSystemObjectType == SPFileSystemObjectType.File) {
    // process item as document
    SPFile file = item.File;
  }
}

//JavaScript Client Object Model
var enumerator = listItems.getEnumerator(); //Returned from async call
while (enumerator.moveNext()) {
    var listItem = enumerator.get_current();
    if (listItem.get_fileSystemObjectType() === SP.FileSystemObjectType.file) {
        //process item as document
    }
}

//REST interface
var results = data.d.results; //Returned from async call
for (var i = 0; i < results.length; i++) {
    if (results[i].FileSystemObjectType === SP.FileSystemObjectType.file) {
        //process item as document
    }
}

One last point to keep in mind is that discovering documents by enumerating through the Items collection of a document library finds all documents without regard to whether they exist in the root folder or in folders nested below the root folder. If you would rather enumerate through only the documents in the root folder of a document library, you can use a different approach by using the SPFolder and SPFile classes. Example 9-15 shows how to access only documents located in the root folder of the library.

//Server-Side Object model
SPWeb site = SPContext.Current.Site.RootWeb;
SPDocumentLibrary DocLib = (SPDocumentLibrary)site.Lists["FinancialDocuments"];
foreach (SPFile file in docLib.RootFolder.Files) {
  // program against file using SPFile class
}

//JavaScript Client Object Model
var ctx = new SP.ClientContext.get_current();
var list = ctx.get_web().get_lists().getByTitle("FinancialDocuments");
ctx.load(list);
var files = list.get_rootFolder().get_Files();
ctx.load(files);
ctx.executeQueryAsync(success, failure);

//REST interface
$.ajax(
{
    url: _spPageContextInfo.webServerRelativeUrl +
         "/_api/web/lists/getByTitle ('FinancialDocuments')/rootFolder/files",
    type: "GET",
    headers: {
        "accept": "application/json;odata=verbose",
             }
});

Event handling with the server-side object model in SharePoint solutions is based on event receiver classes. You create a new event receiver class by inheriting from one of the following event receiver base classes that are defined inside the Microsoft.SharePoint assembly:

The SPItemEventReceiver class provides event handling support for when users add, modify, or delete items in a list or documents in a document library. The SPListEventReceiver class provides event handling support for when users create and delete lists, as well as when users modify a list’s fields collection. The SPEmailEventReceiver class provides event handling support for when users send email messages to an email-enabled list.

The SPWebEventReceiver class provides event handling support for when users create new child sites within a site collection. The SPWebEventReceiver class also provides event handling support for when users move or delete sites, including both child sites and top-level sites. The SPWorkflowEventReceiver class provides event handling support for when users start a workflow instance as well as event handling support to signal when a workflow instance has completed or has been postponed.

After you have created a class that inherits from one of the event receiver base classes, you implement the event receiver class by overriding methods that represent event handlers. As an example, Example 9-16 creates an event handler for an announcements list. The event handler adds a notice and tracking number to each announcement when it is added to the list. Furthermore, the handler prevents the deletion of any items from the list.

public class ItemReceiver : SPItemEventReceiver
{
   public override void ItemAdding(SPItemEventProperties properties)
   {
       properties.AfterProperties["Body"] += "
 ** For internal use only ** 
";
   }

   public override void ItemAdded(SPItemEventProperties properties)
   {
       properties.ListItem["Body"] +=
         "
 Tracking ID: " +  Guid.NewGuid().ToString() + " 
";
       properties.ListItem.Update();
   }


   public override void ItemDeleting(SPItemEventProperties properties)
   {
       properties.Cancel = true;
       properties.Status = SPEventReceiverStatus.CancelWithRedirectUrl;
       properties.ErrorMessage = "Items cannot be deleted from this list.";
   }
}

Event handling in apps is based upon a completely different architecture known as remote event receivers. Remote event receivers are similar in concept to the standard event handlers except that the receiver is a remote service endpoint instead of an assembly. Remote event receivers support events at the web, app, list, and list-item level, which can be both synchronous and asynchronous.

Remote event receivers can be added to an app through either the Add New Item dialog box or the Properties dialog box. If the remote event receiver is to handle anything other than app life cycle events, it should be added to the app by using the Add New Item dialog box. If the remote event receiver is to handle one of the app life cycle events, it is added by setting one of the event properties for the app, as shown in Figure 9-2 and detailed in Chapter 4.

Figure 9-2. Setting an app life cycle event

If the remote event receiver is added through the Add New Item dialog box, you will be further prompted to select the event scope and event types to handle. After the scope and type are defined, Visual Studio will automatically add a new web project to your app to handle the events. This web project is automatically set as the associated remote web for the app so that it will start during the debugging process.

Remote event receivers implement the IRemoteEventService interface. This interface consists of two methods: ProcessEvent and ProcessOneWayEvent. You use the ProcessEvent method to handle synchronous events and the ProcessOneWayEvent method to handle asynchronous events. The new web project comes with template code that implements the IRemoteEventService interface and uses the TokenHelper class to retrieve a CSOM ClientContext for calling back into SharePoint. Example 9-17 implements a remote event handler for an announcements list similar to what is shown in Example 9-16.

public class AnnouncementsReceiver : IRemoteEventService
{
    public SPRemoteEventResult ProcessEvent(RemoteEventProperties properties)
    {
        SPRemoteEventResult result = new SPRemoteEventResult();
        switch (properties.EventType)
        {
            case SPRemoteEventType.ItemAdding:
                result.ChangedItemProperties.Add("Body", "
 ** For internal use only **

");
                break;
            case SPRemoteEventType.ItemDeleting:
                result.ErrorMessage = "Items cannot be deleted from this list";
                result.Status = SPRemoteEventServiceStatus.CancelWithError;
                break;
        }
        return result;
    }
    public void ProcessOneWayEvent(RemoteEventProperties properties)
    {
        HttpRequestMessageProperty requestproperty =
        (HttpRequestMessageProperty)OperationContext.Current.
        IncomingMessageProperties[HttpRequestMessageProperty.Name];
        string contexttokenstring = requestproperty.Headers["x-sp-accesstoken"];
        if (contexttokenstring != null)
        {
            SharePointContextToken contexttoken =
            TokenHelper.ReadAndValidateContextToken(
            contexttokenstring, requestproperty.Headers[HttpRequestHeader.Host]);
            Uri sharepointurl = new Uri(properties.ItemEventProperties.WebUrl);
            string accesstoken = TokenHelper.GetAccessToken(
            contexttoken, sharepointurl.Authority).AccessToken;
            using (ClientContext clientcontext =
            TokenHelper.GetClientContextWithAccessToken(
            sharepointurl.ToString(), accesstoken))
            {
                if (properties.EventType == SPRemoteEventType.ItemAdded)
                {
                    List list =
                    clientcontext.Web.Lists.GetByTitle(
                    properties.ItemEventProperties.ListTitle);
                    clientcontext.Load(list);
                    ListItem item =
                    list.GetItemById(properties.ItemEventProperties.ListItemId);
                    clientcontext.Load(item);
                    clientcontext.ExecuteQuery();
                    item["Body"] += "
 Tracking ID: " +  Guid.NewGuid().ToString() + " 
";
                    item.Update();
                    clientcontext.ExecuteQuery();
                }
            }
        }
    }
}

After you create an event receiver, you must bind one or more of its event handler methods to a host object by using event registration. The types of objects that support event registration include site collections, sites, lists, content types, and documents. Note that only certain types of event handlers are supported by each type of host object. For example, you can register a ListAdded event handler with a site collection or a site, but that event type is not supported by host objects such as lists, content types, or documents. Likewise, you can register an ItemUpdating event handler with a list, content type, or document, but that event type is not supported by site collections or sites.

The simplest and most common technique for registering an event handler is to use a declarative Receivers element in a CAML file. Example 9-18 shows the CAML for registering the event receiver in Example 9-16.

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Receivers ListTemplateId="104" Scope="Web">
      <Receiver>
        <Name>ItemReceiverItemAdding</Name>
        <Type>ItemAdding</Type>
        <Assembly>$SharePoint.Project.AssemblyFullName$</Assembly>
        <Class>AnnouncementHandler.ItemReceiver.ItemReceiver</Class>
        <SequenceNumber>10000</SequenceNumber>
      </Receiver>
      <Receiver>
        <Name>ItemReceiverItemDeleting</Name>
        <Type>ItemDeleting</Type>
        <Assembly>$SharePoint.Project.AssemblyFullName$</Assembly>
        <Class>AnnouncementHandler.ItemReceiver.ItemReceiver</Class>
        <SequenceNumber>10000</SequenceNumber>
      </Receiver>
      <Receiver>
        <Name>ItemReceiverItemAdded</Name>
        <Type>ItemAdded</Type>
        <Assembly>$SharePoint.Project.AssemblyFullName$</Assembly>
        <Class>AnnouncementHandler.ItemReceiver.ItemReceiver</Class>
        <SequenceNumber>10000</SequenceNumber>
        <Synchronization>Synchronous</Synchronization>
      </Receiver>
  </Receivers>
</Elements>

The registration file for a remote event receiver is nearly identical to the one used for a server-side event receiver. The only difference is that the file adds a Url element that refers to the endpoint of the remote event receiver. This is the endpoint that is invoked when the event occurs. Example 9-19 shows the registration CAML for the remote event receiver described in Example 9-17.

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Receivers ListTemplateId="10000">
      <Receiver>
        <Name>AnnouncementsReceiverItemAdding</Name>
        <Type>ItemAdding</Type>
        <SequenceNumber>10000</SequenceNumber>
        <Url>http://webs.wingtiptoys.com/AnnouncementsReceiver.svc</Url>
      </Receiver>
      <Receiver>
        <Name>AnnouncementsReceiverItemDeleting</Name>
        <Type>ItemDeleting</Type>
        <SequenceNumber>10000</SequenceNumber>
        <Url>http://webs.wingtiptoys.com/AnnouncementsReceiver.svc</Url>
      </Receiver>
      <Receiver>
        <Name>AnnouncementsReceiverItemAdded</Name>
        <Type>ItemAdded</Type>
        <SequenceNumber>10000</SequenceNumber>
        <Url>http://webs.wingtiptoys.com/AnnouncementsReceiver.svc</Url>
      </Receiver>
  </Receivers>
</Elements>

Although registering event handlers declaratively by using a Receivers element works in the majority of scenarios, there are a few cases for which it doesn’t suffice. For example, you cannot register an event handler on a host object such as a content type or an individual document by using a declarative Receivers element. These types of event registration must be accomplished by using code.

Host object types that support events such as SPContentType expose a collection property named EventReceivers. The EventReceivers collection property exposes an Add method that has five different overloaded implementations. The most straightforward implementation of the Add method accepts three parameters: the event type, the four-part assembly name, and the namespace-qualified name of the event receiver class. Example 9-20 shows an example of implementing the FeatureActivated method for a site collection–scoped feature that registers an event handler on the Item content type by using an event receiver class named ItemContentTypeEvents.

public override void FeatureActivated(SPFeatureReceiverProperties properties) {
  SPSite siteCollection = (SPSite)properties.Feature.Parent;
  SPWeb site = siteCollection.RootWeb;

  // retrieve content type
  SPContentType ctypeItem = site.ContentTypes["Item"];

  // register event handler for content type
  string ReceiverAssemblyName = this.GetType().Assembly.FullName;
  string ReceiverClassName = typeof(ItemContentTypeEvents).FullName;
  ctypeItem.EventReceivers.Add(SPEventReceiverType.ItemDeleting,
                            ReceiverAssemblyName,
                            ReceiverClassName);

  // push updates to all lists and document libraries
  ctypeItem.Update(true, false);
}

Though calling the Add method with these three parameters provides the easiest approach for registering an event handler, it provides the least flexibility. For example, you cannot assign registration properties for receiver data or synchronization. To obtain more control, you can register the event handler by calling another implementation of the Add method, one that takes no parameters and returns an SPEventReceiverDefinition object, as shown in the following code:

SPContentType ctypeItem = site.ContentTypes["Item"];
string ReceiverAssemblyName = this.GetType().Assembly.FullName;
string ReceiverClassName = typeof(ItemContentTypeEvents).FullName;

// register event handler by creating SPEventReceiverDefinition object
SPEventReceiverDefinition def = ctypeItem.EventReceivers.Add();
def.Type = SPEventReceiverType.ItemDeleting;
def.Assembly = ReceiverAssemblyName;
def.Class = ReceiverClassName;
def.SequenceNumber = 100;
def.Data = "MyData";
def.Update();

// push updates to all lists and document libraries
ctypeItem.Update(true, false);

Remote event receivers support the same type of server-side registration by using a different overload of the Add method. In this case, an additional parameter is passed that specifies the Url of the remote event receiver endpoint. The general form of the registration is shown in the following code:

string serviceUrl= "http://webs.wingtiptoys.com/AnnouncementsReceiver.svc";
string siteUrl= "http://intranet.wingtiptoys.com";
using (SPSite site = new SPSite(siteUrl))
{
   using (SPWeb web = site.RootWeb)
   {
      SPList list = web.Lists["Announcements"];
      list.EventReceivers.Add(SPEventReceiverType.ItemAdded, serviceUrl);
   }
}

When you write the event handler implementation for a before event, you have the ability to cancel the user action that triggered the event. In server-side event handlers, you can accomplish this by assigning a value of true to the Cancel property of the event handler parameter named properties, as shown in Example 9-16. In remote event receivers, you can assign the value of CancelWithError to the SPRemoteEventResult object, as shown in Example 9-17. When canceling the event action, you can also use the ErrorMessage property to assign a custom error message that will be displayed to the user.

When you cancel a before event, SharePoint responds by short-circuiting the user’s request and canceling the event action. The key point here is that before events provide you with a layer of defense against unwanted modifications. Instead of deleting the list as the user requested, SharePoint displays the custom error message in an message box to the user.

In some scenarios, you might decide that the standard error dialog box displayed by SharePoint upon the cancellation of an event action is not sufficient. If you want to create a user experience that is more customized, you can modify the Status and RedirectUrl properties to redirect the user to a custom error page. These properties are available in both server-side and remote event receivers. The following code shows a sample for each:

//Server-Side Event Receiver
public override void ListDeleting(SPListEventProperties properties) {
  if (!properties.Web.UserIsSiteAdmin) {
    properties.Cancel = true;
    properties.Status = SPEventReceiverStatus.CancelWithRedirectUrl;
    properties.RedirectUrl = properties.Web.Site.Url + "/error.aspx";
  }
}

//Remote Event Receiver
public SPRemoteEventResult ProcessEvent(RemoteEventProperties properties)
{
    SPRemoteEventResult result = new SPRemoteEventResult();
    switch (properties.EventType)
    {
        case SPRemoteEventType.ItemDeleting:
            result.Status = SPRemoteEventServiceStatus.CancelWithError;
            result.RedirectUrl = "http://webs.wingtiptoys.com/error.aspx";
            break;
    }
    return result;
}

After events cannot be used to cancel an event action. Instead, after events provide the opportunity to execute code in response to an event action, such as a user successfully creating a new list. When programming after events, you must reach back into SharePoint and explicitly retrieve the item to work with. In Example 9-16, for example, the server-side object model is used to retrieve the recently added announcement.

In remote event handlers, you must take the same basic approach. However, calling back into SharePoint requires passing an OAuth token from the event handler, as shown in Example 9-17. This process is the same for remote event receivers as it is for any SharePoint app containing a remote web. Chapter 6, covers the OAuth security infrastructure and the use of the TokenHelper class in detail, so it will not be repeated here.

When programming after events, you must be aware of several situations that can result in unexpected behavior. The first situation concerns the manner in which items are updated. The second situation involves the accidental creation of cascading events. The third situation is the need to call after events synchronously.

When updating list items in an after event, you should use the UpdateOverwriteVersion method instead of the Update method. You should avoiding calling Update in an after event on a list where versioning is enabled because it will generate two versions each time a user adds or updates an item. The UpdateOverwriteVersion method is provided for this exact scenario because it updates the most current version instead of generating a new version.

Modifying an item in an after event can trigger another cascading event. Consider the scenario in which an after event updates an item, which triggers the after event, which updates the item that triggers the after event, and so on. If you are not careful, you can create a recursive loop that will run until an error occurs. Here is an example of a flawed implementation of ItemUpdated that will experience this problem:

public override void ItemUpdated(SPItemEventProperties properties) {
  properties.ListItem["Title"] = properties.ListItem["Title"].ToString();
  properties.ListItem.UpdateOverwriteVersion();
}

Whenever you are required to implement an after event in which you update the item that triggered the event, you must disable event firing by modifying the EventFiringEnabled property of the event receiver class:

public override void ItemUpdated(SPItemEventProperties properties) {
  this.EventFiringEnabled = false;
  properties.ListItem["Title"] = properties.ListItem["Title"].ToString();
  properties.ListItem.UpdateOverwriteVersion();
  this.EventFiringEnabled = true;
}

If you do not disable event firing in the ItemAdded event handler, it is not as critical because it will not cause a recursive loop. However, it is still recommended because you then avoid triggering an Update event and executing ItemUpdated unnecessarily when a user adds a new item:

public override void ItemAdded(SPItemEventProperties properties) {
  this.EventFiringEnabled = false;
  properties.ListItem["Title"] = properties.ListItem["Title"].ToString();
  properties.ListItem.UpdateOverwriteVersion();
  this.EventFiringEnabled = true;
}

The final situation of concern centers on the fact that SharePoint Foundation executes the event handlers for after events such as ItemAdded and ItemUpdated asynchronously by default. The problem with after events that execute asynchronously revolves around the user seeing inconsistent field values. When a user updates a field value in the browser and saves the changes with a postback, any updates made by an event handler running asynchronously are not guaranteed to be reflected in the page that is sent back to the user. When you configure the event handler for an after event to run synchronously, you guarantee that the event handler’s updates are reflected in the page returned to the user. To configure the event handler for an after event to run synchronously, you can add a Synchronization element with an inner value of Synchronous into the Receiver element, which is shown in Example 9-18.

Querying a list for specific items that meet a certain criteria can done by using the Microsoft.SharePoint.SPQuery object. The SPQuery object exposes a Query property that accepts a CAML fragment, which defines the query to perform. A ViewFields property defines the fields to return. The following code shows a simple query run against a list:

SPQuery query = new SPQuery();
query.Viewfields = @"<FieldRef Name='Title'/><FieldRef Name='Expires'/>";
query.Query =
@"<Where>
  <Lt>
    <FieldRef Name='Expires'/>
    <Value Type='DateTime'><Today/></Value>
  </Lt>
</Where>";
SPList list = SPContext.Current.Web.Lists.TryGetList("Announcements");
SPListItemCollections items = list.GetItems(query);

The ViewFields property accepts a CAML fragment containing a series of FieldRef elements. Each FieldRef element has a Name attribute that specifies the name of the list field to return from the query. Note that the Name attribute must contain the name of the field as it is defined in the schema.xml file for the list definition and not simply the display name of the field.

In order to create a query, you must properly construct a CAML fragment defining the items to return from the list. At the highest level, the CAML fragment may contain Where, OrderBy, and GroupBy elements. Inside of each of these elements, you can use additional CAML elements to specify conditions. Table 9-5 contains a complete list of CAML elements that can be used to create a query, and Example 9-21 shows the basic form of the CAML query.

<Where>
  <Lt>,<Gt>,<Eq>,<Leq>,<Geq>,<Neq>,<BeginsWith>,<Contains>,<IsNotNull>,<IsNull>
    <FieldRef/>
    <Value>[Test Value], Today</Value>
  </Lt>,</Gt>,</Eq>,</Leq>,</Geq>,</Neq>,</BeginsWith>,</Contains>,</IsNotNull>,</IsNull>
  <And>,<Or>
  <Lt>,<Gt>,<Eq>,<Leq>,<Geq>,<Neq>,<BeginsWith>,<Contains>,<IsNotNull>,<IsNull>
    <FieldRef/>
    <Value>[Test Value], Today</Value>
  </Lt>,</Gt>,</Eq>,</Leq>,</Geq>,</Neq>,</BeginsWith>,</Contains>,</IsNotNull>,</IsNull>
  </And>,</Or>
</Where>
<OrderBy>
  <FieldRef/>
</OrderBy>
<GroupBy>
  <FieldRef/>
<GroupBy>

In addition to querying single lists, the SPQuery object can be used to query across two lists that are joined by a Lookup field and surfacing projected fields. As an example, consider two lists named Instructors and Modules. The Instructors list is a simple list that contains contact information for classroom instructors. The Modules list is a custom list that contains information about training modules that will be taught in the classroom. The Modules list is joined to the Instructors list via a lookup field that shows the FullName of the instructor. Additionally, the instructor’s Email Address is available as a projected field. In this way, an instructor can be assigned a Module to teach. Using a SPQuery object and CAML, you can create a query that returns fields from both of these lists as shown in Example 9-22.

SPWeb site = SPContext.Current.Web;
SPList listInstructors = site.Lists["Instructors"];
SPList listModules = site.Lists["Modules"];

SPQuery query = new SPQuery();
query.Query = "<Where><Eq><FieldRef Name="Audience"/>" +
              "<Value Type="Text">Developer</Value></Eq></Where>";
query.Joins = "<Join Type="Inner" ListAlias="classInstructors">" +
              "<Eq><FieldRef Name="Instructor" RefType="Id" />" +
              "<FieldRef List="classInstructors" Name="Id" /></Eq></Join>";
query.ProjectedFields =
"<Field Name='Email' Type='Lookup' List='classInstructors' ShowField='Email'/>";
query.ViewFields = "<FieldRef Name="Title" /><FieldRef Name="Instructor" />" +
                   "<FieldRef Name="Email" />";

SPListItemCollection items = listModules.GetItems(query);

In Example 9-22, the Where clause is created to return training modules that are intended for a developer audience; this is similar to the simple example shown earlier. The Join property is new and defines the join between the two lists through the lookup field. Remember that the query is being run on the Modules list, so it must be joined to the Instructors list. The ListAlias attribute defines an alias for the Instructors list that can be used in the join clause. The first FieldRef element refers to the name of the lookup field in the Modules list and will always have a RefType equal to Id. The second FieldRef in the join clause uses the alias name for the Instructors list and will always have a Name equal to Id. The ProjectedFields property also uses the alias name for the Instructors list and refers to additional fields in the instructors list that should be returned with the query.

Although the SPQuery object is good for querying a single list or joined lists, if you want to query multiple lists within a site collection simultaneously, then you can make use of the Microsoft.SharePoint.SPSiteDataQuery object. Like the SPQuery object, the SPSiteDataQuery object has Query and ViewFields properties. In addition to these fields, the SPSiteDataQuery object also has Lists and Webs properties. The Lists property is used to specify the lists within the site collection that should be included in the query. The Webs property is used to determine the scope of the query. Example 9-23 shows a query that returns events from all calendars in the current site collection where the end date is later than today.

//Creates the query
SPSiteDataQuery query = new SPSiteDataQuery();

//Builds the query
query.Query = "<Where><Gt><FieldRef Name='EndDate'/>" +
              "<Value Type='DateTime'><Today OffsetDays="-1"/></Value></Gt></Where>";

//Sets the list types to search
query.Lists = "<Lists ServerTemplate='106' />";

//Sets the Fields to include in results
query.ViewFields = "<FieldRef Name='fAllDayEvent' />" +
                   "<FieldRef Name='Title' />" +
                   "<FieldRef Name='Location' />" +
                   "<FieldRef Name='EventDate' />" +
                   "<FieldRef Name='EndDate' />";

//Sets the scope of the query
query.Webs = @"<Webs Scope='SiteCollection' />";

//Execute the query
DataTable table = SPContext.Current.Site.RootWeb.GetSiteData(query);

The Lists property in Example 9-23 is a CAML fragment that can take several forms to specify the lists to include in the query. Setting the property to <Lists ServerTemplate=[value]/> limits the query to lists of a certain server template. For example, type 106 is a calendar. Table 9-6 shows all of the possible values for the ServerTemplate attribute. Setting the property to <Lists BaseType=[value]/> limits the query to lists of a certain BaseType. Table 9-7 lists the possible vales for the BaseType attribute. Setting the property to <Lists Hidden=’true’/> includes hidden lists in the query. Setting the property to <Lists MaxListLimit=[value]/> limits the query to considering no more than the specified number of lists.

The Webs property is a CAML fragment that must either be <Webs Scope=’SiteCollection’/> or <Webs Scope=’Recursive’/>. SiteCollection includes all lists in the site collection, whereas Recursive includes only those lists in the current site or subsites beneath the current site.

SharePoint 2013 provides special support for large lists. In particular, SharePoint allows administrators to throttle the number of items returned in a list view in order to prevent performance degradation caused by returning an excessive number of items. Though these throttle settings apply to views created by end users, they also apply to queries executed in custom code.

When you are executing queries, the number of results returned will be determined by the throttle settings for the specified list and the rights of the current user. Throttle settings are set for the web application in Central Administration. Rights for the current user that affect throttling include administration rights on the web front-end server, administration rights in the web application, and auditor rights in the web application.

In the context of list throttling, users who have server administration rights on the web front end where the query is run are known as server administrators. Users granted Full Read (auditors) or Full Control (administrators) through the web application policy in Central Administration are considered super users. Everyone else is simply termed a normal user.

The List View Threshold is set at the web application level and specifies the maximum number of items that can be involved in a database operation at a single time. The default value for this setting is 5,000, which means that results returned from a SPQuery or SPSiteDataQuery object will be generally limited to 5,000 items for both super users and normal users. Additionally, the List View Lookup Threshold specifies the maximum number of lookup, person/group, or workflow status fields that can be involved in the query. This value defaults to 6. Server administrators are normally not affected by the List View Threshold or List View Lookup Threshold settings.

Both the SPQuery and SPSiteDataQuery objects have a QueryThrottleMode property that can be set to one of the values in the Microsoft.SharePoint.SPQueryThrottleOption enumeration. The possible values for the property are Default, Override, and Strict. Setting the QueryThrottleMode property to Default causes query throttling to be implemented for both super users and normal users based on the List View Threshold and List View Lookup Threshold settings. Server administrators are not affected.

Setting the QueryThrottleMode property to Override allows super users to return items up to the limit specified in the List View Threshold for the Auditors and Administrators setting as long as the Object Model Override setting is set to “Yes” for the current web application. Normal users are still limited to returning the number of items specified in the List View Threshold and List View Lookup Threshold settings. Server administrators remain unaffected.

Setting the QueryThrottleMode property to Strict causes the limits specified by the List View Threshold and List View Lookup Threshold settings to apply to all users. In this case, it makes no difference what rights you have in the web application or server. Example 9-24 shows the RenderContents method from a Web Part with configurable throttling returning query results from a list and demonstrating the concepts discussed.

protected override void RenderContents(HtmlTextWriter writer)
{
    SPWeb site = SPContext.Current.Web;
    SPList list = site.Lists[listName];
    SPUser user = SPContext.Current.Web.CurrentUser;
    SPQuery query = new SPQuery();

    //Throttle settings
    if (overrideThrottling)
        query.QueryThrottleMode = SPQueryThrottleOption.Override;
    else
        query.QueryThrottleMode = SPQueryThrottleOption.Strict;

    //Execute query
    query.Query = "</OrderBy>";
    query.ViewFields = "<FieldRef Name="Title" />";
    SPListItemCollection items = list.GetItems(query);

    //Show user role
    if(user.IsSiteAdmin || user.IsSiteAuditor)
        writer.Write("<p>You are a 'Super User'</p>");
    else
        writer.Write("<p>You are a regular user</p>");

    //Is throttling enabled?
    if(list.EnableThrottling)
        writer.Write("<p>Throttling is enabled</p>");
    else
        writer.Write("<p>Throttling is not enabled</p>");

    //Show count of items returned
    writer.Write("<p>" + items.Count + " items returned.</p>");

}

Regardless of the value set for the QueryThrottleMode property, no results will be throttled if the query is run within the time specified in the Daily Time Window for Large Queries. During this time period, all queries are allowed to run to completion. Additionally, the EnableThrottling property of the list can be set to False to remove the list from any and all throttling restrictions. The EnableThrottling property can only be set by someone with Farm Administrator rights using a Windows PowerShell script similar to the following:

$site = Get-SPWeb -Identity "http://wingtip.com/products"
$list = $site.Lists["Toys"]
$list.EnableThrottling = $false

SharePoint list data is maintained in the content database. This means that the structure of the list and item data is based on relational tables. As a SharePoint developer, however, you do not need to understand the structure of these tables because the object model abstracts the structure into SPList and SPListItem objects. When you write a LINQ to SharePoint query, you should expect the same experience as when using the object model. List and item data should be abstracted so that you do not have to understand the content database schema.

LINQ to SharePoint provides an object layer abstraction on top of the content database through the use of entity classes. Entity classes are lightweight, object-relational interfaces to the list and item data in the content database. Additionally, entity classes are used to track changes and provide optimistic concurrency during updates.

Entity classes are created by using a command-line utility called SPMetal. SPMetal is located in theSharePoint system directory at C:Program FilesCommon FilesMicrosoft Sharedweb server extensions15in. As a best practice, you should update the PATH variable in your environment to include the path the SPMetal. This way, you can simply run the utility immediately after opening a command window.

Generating entity classes with SPMetal can be very simple. At a minimum, you must specify the site for which you want to generate entities and the name of the code file to create. After the code file is created, you can immediately add it to a project in Visual Studio and start writing LINQ queries. The following code shows an example that will generate entity classes for all of the lists and content types in a site:

SPMetal /web:http://wingtiptoys.com /code:Entities.cs

Though generating entity classes can be quite easy, you will likely want more control over which entities are created and how they are structured. SPMetal provides a number of additional arguments that you can use to alter code generation. Table 9-8 lists all of the possible arguments for SPMetal and describes them.

If you examine the code file generated by SPMetal, you will see that there are two kinds of classes created. First, a single class is created that inherits from Microsoft.SharePoint.Linq.DataContext. The DataContext class provides a connection to lists and change tracking for operations. You can think of the DataContext class as serving a purpose similar to the SqlConnection class in data access code. Second, multiple classes are generated that represent the various content types used by the lists in the site. Using the DataContext class together with the entity classes allows you to write LINQ queries. Example 9-25 shows a simple LINQ query written to return all training modules contained in the list named Modules by using a DataContext class named Entities.

using (Entities dc = new Entities(SPContext.Current.Web.Url))
{
    var q = from m in dc.Modules
            orderby m.Title
            select m;

    foreach (var module in q)
    {
        moduleList.Items.Add(module.Title);
    }
}

<?xml version="1.0" encoding="utf-8"?>
<Web Class="Entities" AccessModifier="Public"
  xmlns="http://schemas.microsoft.com/SharePoint/2009/spmetal" >
  <List Name="Instructors" Member="Instructors">
    <ContentType Name="Contact" Class="Instructor">
      <Column Name="FullName" Member="FullName"/>
      <ExcludeOtherColumns/>
    </ContentType>
  </List>
  <List Name="Modules" Member="Modules" />
  <ExcludeOtherLists/>
</Web>

After you have entities generated, then you can begin to write LINQ to SharePoint queries. Writing LINQ to SharePoint queries is very similar to writing LINQ queries for other data sources. You formulate a query by using query syntax, receive the results into an anonymous type, and then use the IEnumerable interface to iterate over the results.

LINQ to SharePoint also supports querying across lists that are joined by a lookup field. LINQ to SharePoint makes the syntax much simpler than with CAML. The following code shows the equivalent LINQ query for the CAML shown in Example 9-22. Note how the join is simply done by using the dot operator to move easily from the Modules list to the joined Instructors list:

var q = from m in dc.Modules
        orderby m.ModuleID
        select new { m.Title, Presenter = m.Instructor.FullName, Email = m.Instructor.Email};

Not only does the code join two lists together, but you can see that it is also using a projection. The new keyword is creating a new set of anonymous objects whose field names have been set to Title, Presenter, and Email.

LINQ to SharePoint also allows you to perform query composition. Query composition is the ability to run a LINQ query on the results of a LINQ query. For example, the following code shows how to run a new query specifically looking for a training module named Visual Studio 2012:

var q1 = from m1 in dc.Modules
        orderby m1.ModuleID
        select new { m1.Title, Presenter = m1.Instructor.FullName, Email = m1.Instructor.Email};
var q2 = from m2 in q1
        where m2.Title.Equals("Visual Studio 2012")
        select m2;

Finally, LINQ to SharePoint supports a number of extension methods that you can use for aggregation, grouping, and returning specific entities. The methods are often used on the results of the query. The following code, for example, shows how to return the total number of training modules in the query results. The most commonly used extension methods are listed in Table 9-10:

var t = (from m in dc.Modules
         select m).Count();

Along with querying, you can also add, delete, and update lists with LINQ to SharePoint. Adding and deleting items are accomplished by using methods associated with the EntityList<T> property of the DataContext. The InsertOnSubmit() method adds a single new item to a list. The InsertAllOnSubmit() method adds a collection of new items to a list. The DeleteOnSubmit() method deletes a single item from a list, and the DeleteAllOnSubmit() method deletes a collection of items from a list. The RecycleOnSubmit() method puts a single item into the recycle bin, and the RecycleAllOnSubmit() method puts a collection of items in the recycle bin. The following code shows an example of adding a new item to the Modules list:

using (Entities dc = new Entities(SPContext.Current.Web.Url))
{
    ModulesItem mi = new ModulesItem();
    mi.Title = "LINQ to SharePoint";
    mi.Id = 301;
    dc.Modules.InsertonSubmit(mi);
    dc.SubmitChanges();
}

Updating items in lists is done by simply changing the property values in the item and then calling the SubmitChanges() method of the DataContext. The following code shows a simple example of an update operation:

using (Entities dc = new Entities(SPContext.Current.Web.Url))
{
    var q = (from m in dc.Modules
            where m.Id==1
            select m).First();

    q.Title = "Revised Title for Module 1";
    dc.SubmitChanges();
}

When updating items, LINQ to SharePoint uses optimistic concurrency. The provider will check to see whether the items in the list have been changed since your LINQ query was run before it will attempt to update them. If a discrepancy is found for any of the submitted entities, then no changes are committed. All discrepancies must be resolved before any change in the current batch can be committed.

When discrepancies are found during the update process, LINQ to SharePoint throws a Microsoft.SharePoint.Linq.ChangeConflictException. Additionally, the ChangeConflicts collection of the DataContext is populated with ObjectChangeConflict objects that contain data about fields in the item causing conflicts. The SubmitChanges() method supports overloads that allow you to specify whether update attempts should continue after the first conflict or whether update attempts should stop. The ChangeConflicts collection will only be populated with information about failed attempts, so electing to stop after the first failure will not provide complete data on all conflicts. Regardless of whether or not you continue update attempts, remember that no changes will be saved if any conflict occurs. The purpose of continuing update attempts is to completely populate the ChangeConflicts collection.

The ChangeConflicts collection contains a MemberConflicts collection that has the detailed information about the actual values causing the conflict. In particular, the MemberConflicts collection is populated with MemberChangeConflict objects. These objects each have OriginalValue, CurrentValue, and DatabaseValue properties. The OriginalValue is the value of the column when the LINQ query was run. The CurrentValue is the value that SubmitChanges() is attempting to write to the content database. The DatabaseValue is the current value of the column in the database. Trapping the ChangeConflictException and using the MemberChangeConflict objects allows you to display the conflicts to the end user. The code in Example 9-26 shows how to iterate the collection, build a list, and bind the list to a grid for display.

Try
{
    //Update code
}
catch (Microsoft.SharePoint.Linq.ChangeConflictException x)
{
    conflicts = new List<Conflict>();
    foreach (ObjectChangeConflict cc in dc.ChangeConflicts)
    {
        foreach (MemberChangeConflict mc in cc.MemberConflicts)
        {
            Conflict conflict = new Conflict();
            conflict.OriginalValue = mc.OriginalValue.ToString();
            conflict.CurrentValue = mc.CurrentValue.ToString();
            conflict.DatabaseValue = mc.DatabaseValue.ToString();
            conflicts.Add(conflict);
        }

    }
conflictGrid.DataSource = conflicts;
conflictGrid.DataBind();
}

Along with displaying the results, you can also resolve conflicts in code. After displaying the results to the end user in a grid, you can allow the user to select whether the pending changes should be forced or lost. The Resolve() method of the MemberChangeConflict class accepts a Microsoft.SharePoint.Linq.RefreshMode enumeration that can have a value of KeepChanges, KeepCurrentValues, or OverwriteCurrentValues. KeepChanges accepts every pending change, but gives the highest priority to the current user. KeepCurrentValues keeps only the changes made by the current user and loses all other changes. OverwriteCurrentValues loses the current changes and sets the values to what is in the database. After calling the Resolve() method, you must SubmitChanges() again to complete the operation. The following code shows an example of keeping the current changes and losing all other changes:

foreach (ObjectChangeConflict cc in dc.ChangeConflicts)
{
    foreach (MemberChangeConflict mc in cc.MemberConflicts)
    {
        mc.Resolve(RefreshMode.KeepCurrentValues);
    }
}
dc.SubmitChanges();