Chapter 5. Manage search capabilities

Search in SharePoint 2016 maintains its position as a key feature of a SharePoint deployment. The configuration of this feature varies in complexity and scale, and can be deployed on a single server, multiple servers, or even a separate search-centric farm.

In this chapter, the changes present in the search feature of SharePoint 2016 will be discussed, and you will have a chance to become acquainted (or perhaps, reacquainted) with SharePoint Search from both a configuration and administrative standpoint.

Traditional topology A series of servers, generally distributed in three tiers: Web, App, and Data.

Streamlined topology A concept introduced with SharePoint 2013, this is specialized configuration for SharePoint servers, intended to optimize system resources and maximize performance to the end user.

Understanding which of these two selections has been chosen for your SharePoint 2016 farm directly influences the topology actions that will be taken during the configuration process.

Crawl component This component performs crawls of the content sources you specify, retrieving both metadata and crawled properties. This information is then propagated to the content processing component.

Content processing component This component transforms the crawled items before sending them on to the index component. Crawled properties are mapped to managed properties by this component; other information is also provided to the analytics processing component.

Analytics processing component This component has two major functions: analysis and reporting. It improves search relevance by analyzing the crawled items and how users interact with search results; it also creates search reports and recommendations.

Index component This component has two major functions. This component receives items from the content processing component before writing them to the search index; it also handles both incoming and outgoing queries from the query processing component, retrieving information and returning result sets, respectively, to and from the search index.

Query processing component This component analyzes queries before passing them to the index component for result set retrieval.

Search administration component This component does not have an active role in handling incoming or outgoing search requests; it only runs system processes pertaining to Search and adds and initializes new search component instances.

Administration database This database stores all the Search service application settings; for example, the topology, rules, and mappings between crawled and managed properties.

Crawl database This database stores tracking information and crawled items details, as well as crawl metrics (such as the last crawl time and ID).

Link database This database stores information about search clicks as well as information extracted by the content processing component; both types of information are analyzed by the analytics processing component.

Analytics reporting database Reports are generated from the contents of this database, which include analysis statistics as well as the results of usage analysis.

Decision points for designing your SharePoint Search topology should include the following:

Whether or not your SharePoint 2016 farm topology is to be MinRole compliant

Resiliency requirements for your farm

Overall volume of content to be searched

Client load on the farm, in terms of queries and page views per second

If the farm design will require search resiliency in a MinRole scenario, there are two available choices:

Add additional servers in the Search role Having more than two servers hosting the Search role in a MinRole-compliant SharePoint farm meets the requirements for resiliency.

Provide search features via a dedicated Search farm In enterprise scenarios, the decision is often made to create a farm responsible solely for search requests. In such a configuration, all the servers in that farm host the Search role.

The sheer size of the content corpus (body of content) could be an issue: If you are in a larger environment with thousands of people generating information to be included in the search, you might need to consider the addition of a dedicated Search farm and alter the topology to make these servers focus the processing and delivery of search information.

Finally, client load (particularly on Internet-facing sites) could be an issue; if your SharePoint site hosts a large amount of content, you must deliver an outstanding search experience to provide your customers with the information they need. To do so, you might decide to break the index into multiple partitions and add servers that are specifically focused on providing query functionality.

Selecting the Search service application from within Central Administration and then choosing Manage allows you to review the Search topology (Figure 5-2).

The color and icon beneath each component indicates its status on each server in the farm:

Green check A green check mark indicates that the search component is running correctly.

Yellow triangle A yellow triangle indicates that the search component cannot perform all operations correctly.

Red cross A red cross indicates that the search component is not running or that there are errors that prevent the component from running correctly.

If you have a single server hosting the Search role within your MinRole farm and choose to use the Farm Configuration Wizard (not the SharePoint 2016 Products Configuration Wizard), each search component will be deployed to only that server within the farm. This configuration is fine for an initial deployment, but does not address long-term growth or tuning needs.

At some point, you might encounter a requirement to change the Search topology in a SharePoint farm. Reasons for this change could include performance and scaling, or simply adding resiliency to the search layer of a MinRole farm.

There are two options for changing the Search topology of a farm:

If you have a new farm or one that has an empty index, you can directly change the topology in Windows PowerShell.

If you have a farm with an existing index that you want to keep, you must do the following:

Clone the active Search topology.

Make changes to the cloned topology.

Make the clone the active Search topology for the farm.

If you have a farm with an empty index, you can use the following steps to alter the topology of your farm.

1. Assign the host name of each server to a variable using the Get-SPEnterpriseSearchServiceInstance cmdlet (the example server name shown is wtsearch01):

Click here to view code image

$hostA = Get-SPEnterpriseSearchServiceInstance -Identity "wtsearch01"

2. If you are configuring your farm using a traditional topology (not MinRole-compliant), then you will need to start the search instance on each Search host server using the Start-SPEnterpriseSearchServiceInstance cmdlet. If not, skip to step 4:

Click here to view code image

Start-SPEnterpriseSearchServiceInstance -Identity $hostA

3. Verify the status of Search on each host using the Get-SPEnterpriseSearchServiceInstance cmdlet:

Click here to view code image

Get-SPEnterpriseSearchServiceInstance -Identity $hostA

4. Create a new Search topology and set a reference using the Get-SPEnterpriseSearchServiceApplication and New-SPEnterpriseSearchTopology cmdlets, respectively:

Click here to view code image

$ssa = Get-SPEnterpriseSearchServiceApplication

$newTopology = New-SPEnterpriseSearchTopology -SearchApplication $ssa

Add the appropriate search components to the new topology using the appropriate cmdlets (you will be distributing them across multiple hosts, replacing $hostA with the appropriate server variable, as shown in Table 5-1).

SharePoint 2016 enables content sources to be generated for any of eight possible content source types (see Table 5-2).

Each content source type displayed in the Search service application uses an indexing connector when crawling content. The BCS Connector Framework handles connections to the Lotus Notes and Exchange Public Folder content source types; a standard series of protocol handlers handles connections to SharePoint Sites, Web Sites, and File Shares.

There are three types of crawls available in SharePoint Server 2016: full, incremental, and continuous.

A full crawl can be scheduled to run monthly, weekly, or daily at a given time of day. It is also possible (although not recommended) to run the full crawl multiple times within a given day. You can also stagger the schedule using the Run Every/On option to crawl every X number of days (Daily setting), by choosing the days of the week in which to run the crawl, or by selecting a particular day to run the crawl each month (Monthly setting).

There are several reasons for running a full crawl on a regular basis, including the following:

Creating or re-creating the search index

Correcting issues or corruption within the search index

Detecting security changes made on a file share

Capturing changes made by crawl rules (additions, deletions, modifications)

Capturing new metadata mappings in Search

Capturing new server name mappings in Search

Credential changes for the crawl account

Software changes (hot fixes, cumulative updates, service packs) applied to the farm

An incremental crawl can be scheduled to run monthly, weekly, or daily at a given time of the day. Most often, an incremental crawl is scheduled a few times per day on every day of the week (again depending on the volume of content being crawled). The scheduling options for incremental crawls are identical to those of full crawls.

Continuous crawls are unique among the crawl types, in that more than one can be running at a time. While a crawl of a content source is running (but in an incomplete state), the next scheduled crawl can begin on the same content source (thus the name continuous). By default, continuous crawls kick off every 15 minutes; there is no mechanism for scheduling these crawls in Central Administration.

PowerShell cmdlets exist that can be used to either shorten or lengthen the crawl interval. In the following example, the Local SharePoint Sites content source is set to crawl continuously, and then the crawl interval is reset to 1 minute:

Click here to view code image

$searchapp = Get-SPEnterpriseSearchServiceApplication "Search Service Application"
$contentsource = Get-SPEnterpriseSearchCrawlContentSource -SearchApplication $searchapp
-Identity "Local SharePoint Sites" $contentsource | Set-SPEnterpriseSearchCrawlContentSo
urce -EnableContinuousCrawls $true $searchapp.SetProperty("ContinuousCrawlInterval", 1)

Exclude content from being crawled.

Crawl only a portion of content from a site that is otherwise excluded from crawls.

Specify authentication credentials.

Crawl rules are added on the Add Crawl Rule page within the Search service application (Figure 5-5).

Each new crawl rule requires that three values be chosen: Path, Crawl Configuration, and Specify Authentication.

The Use Regular Expression Syntax For Matching This Rule check box can be used to craft a regular expression that would omit potentially sensitive items such as credit card or Social Security numbers across an assortment of SharePoint sites.

Exclude All Items In This Path If you choose to exclude a path, you can optionally choose to only exclude complex URLs.

Include All Items In This Path If you choose to include a path, you can optionally select the following check boxes:

Crawl Complex URLs (URLs that contain a question mark - ?)

Crawl SharePoint Content As HTTP Pages

Use The Default Content Access Account Choosing to use the default content access account requires no extra settings, and content will be crawled using the search crawl account specified in Central Administration.

Specify A Different Content Access Account If you choose to specify a different content access account, you will be prompted for the account name and password. You can optionally choose to not allow basic authentication, which will prevent your password from being transmitted without encryption during the crawl.

Specify Client Certificate If you choose to specify a certificate, a drop-down list will appear with a list of available certificates.

Specify Form Credentials If you choose to specify form credentials (Forms-based authentication on the site being crawled), you will need to specify the logon address and enter credentials (user name and password).

Use Cookie For Crawling If you choose to use a cookie for crawling content, you can either obtain a cookie from a URL or specify one from your system.

Anonymous Access If you choose to access the site anonymously, no extra settings are required. This rule crawls the content without attempting to authenticate.

There are three configuration options for optimizing performance of the crawl component within SharePoint Server 2016:

Determining how much crawl interaction the destination content sources can handle

Deciding how best to prepare and tune the servers that host the crawl component

Understanding how to monitor the crawl component

Some of these sources might be located across a wide area network (WAN) segment; aggressively crawling these sources could cause diminished performance across the WAN segment, affecting the business.

Performance levels for content sources not located on the SharePoint farm can be negatively affected by a well-provisioned crawl component. If the SharePoint farm is crawling a content source so aggressively that it winds up diminishing performance on the content source, there is a very good chance that the crawl account will be temporarily (or perhaps permanently) disabled from accessing the content source.

Consider coordinating crawls with the administrator(s) of the “crawled” system for potential configuration changes like these:

Time crawled What time of the day is considered “nonbusiness” but also does not interfere with regular processes such as backup jobs?

Crawl duration How long will the crawl take?

Items that are crawled Perhaps it is only important to crawl certain folders or sites, thus reducing the overall load on the content source and improving crawl metrics.

Site The name of the site (not including the protocol).

Request Frequency Indicates how the crawler will request documents from the site. From here you can choose to do either of the following:

Request up to the specified number of documents at a time and do not wait between requests (also specifying the number of simultaneous requests).

Request one document at a time and wait the specified time between requests (also specifying a time to wait in seconds).

1. A content source is crawled (including the permissions applied) and then processed into an index.

2. A user issues a search query, which returns a result set.

3. The result set is security trimmed before being returned to the user.

If a user is granted permission to access an item, the item then appears in the result set. This does not mean that the information wasn’t there all along (the index certainly contained the data), but that it was simply “held back” from view at query time.

Content access accounts should be ordinary Active Directory accounts; in fact, these accounts should specifically have no special privileges at all.

They should not be the same account as that used for SharePoint setup or farm administration.

They should not be the same account as those that run any of the application pools or services.

They should not have local administrative access granted to any of the SharePoint or SQL member servers in the farm.

The reason we should care about the status of these accounts is their effect on security trimming and the Publishing model. One of the most important components of SharePoint 2016, Publishing can be used for the life cycle of items contained within a SharePoint site:

1. An item is initially created and saved in draft mode.

2. After the document is deemed to be ready for publication by its creator, it is submitted.

3. If approvals are enabled, the item can be approved and then scheduled for publication.

4. After the item is approved, it can be published immediately or scheduled for publication at a later date.

The default content access account is used to access most of the content in the SharePoint installation. If you want, this account can also be assigned Read permissions to other content sources; items crawled within these sources are also security trimmed as queries are issued by users. This account is shown (and can be assigned) in the System Status section of the Search Administration page.

Changing the default content access account is a very basic process:

1. Click the content access account name (in this case, RCsp_svcfarm).

2. Type a new user name and password (you have to type the password twice).

3. Click OK.

After this account is designated as being the default content access account, it is assigned Read permissions on all published content in a SharePoint farm on each web application in the farm.

If these sources are not under your control, you can be issued credentials to access these sources for crawling. The content source denotes what type of content is to be crawled, but does not assign specific access account permissions.

From a technical standpoint, cloud hybrid search removes technical obstacles present in the older federated search model, as neither a reverse proxy configuration nor a public Secure Sockets Layer (SSL) certificate is required for configuring cloud hybrid search. Cloud hybrid search also introduces a side benefit, namely the idea that the search index is no longer maintained on premises, but is instead stored in Office 365. Parsed content transferred from on premises to the cloud index is encrypted while in transit between the environments.

The configuration of a cloud Search service application involves a series of configuration actions. If you’ve already configured a hybrid relationship with your SharePoint farm, for OneDrive or Sites, you might have already completed some of these actions.

Search servers Normally, Search servers can host one of six possible search components: crawl, admin, query processing, index, analytics, and content processing. When the farm is configured for cloud hybrid search, the index, analytics, and content processing components still remain on the Search server but are inactive.

Database servers Servers hosting the search databases usually have four databases in use: Search admin, Crawl, Link, and Analytics. In a cloud hybrid search configuration, the Link and Analytics databases are rendered inactive, although they must still exist on the database server.

Global Administrator (Office 365 and Azure AD) Use an Office 365 work account that has been assigned to the Global Administrator role for Office 365 configuration tasks such as configuring SharePoint Online features, running Azure AD and SharePoint Online Windows PowerShell commands, and testing SharePoint Online.

Active Directory Domain Administrator (On-premises Active Directory) Use an Active Directory account in the Domain Admins group to configure and test Active Directory, Active Directory Federation Services (AD FS), Domain Name System (DNS), and certificates, and to do other tasks that require elevation.

SharePoint Farm Administrator (On-premises Active Directory) Use an Active Directory account in the Farm Administrators SharePoint group for SharePoint Server configuration tasks.

Federated Users (On-premises Active Directory) Use Active Directory accounts that have been synchronized with Office 365 to test access to specific resources in both SharePoint Server and SharePoint Online.

1. Configure Office 365 for SharePoint hybrid Configure your Office 365 tenant for a hybrid environment, including registering your domain, configuring User Principal Name (UPN) suffixes, and synchronizing your user accounts.

2. Create a cloud Search service application in SharePoint on premises Use the CreateCloudSSA.ps1 PowerShell script to create the cloud Search service application on the search farm that’s running SharePoint Server 2016.

3. (Optional) Alter the default Search architecture in SharePoint 2016 for cloud hybrid search If you planned a Search architecture that’s different from the default one, set up the planned Search architecture.

4. Connect your cloud Search service application to your Office 365 tenant Use the Onboard-CloudHybridSearch.ps1 PowerShell script to onboard your cloud Search service application and Office 365 tenant to cloud hybrid search. The script sets up the cloud Search service application to interact with the Office 365 tenant and also sets up server-to-server authentication.

5. Create a test content source to crawl for cloud hybrid search Add a small file share first for testing the crawl, as you can add more on-premises content later.

6. (Optional) Set up a separate Search Center to validate hybrid search results in Office 365 This recommended configuration allows you to validate and tune a new search experience in a separate Search Center, while keeping the original search experience entirely unchanged.

7. Start a full crawl of on-premises content for cloud hybrid search When the crawl completes, your on-premises content shows up in the search results in your validation Search Center in Office 365 and in Office Delve.

8. Verify that cloud hybrid search works Go to your Search Center in SharePoint Online in Office 365 and enter this query: IsExternalContent:true. The results you get should show content from the on-premises content source that you’ve crawled.

9. Tune cloud hybrid search Set up and tune the search experiences you’ve planned for your users.

10. Remove the validation Search Center and expose all users to hybrid search results Set your Search Center and any site search in Office 365 to use the default result source and set up the default result source with the search experiences that you’ve tuned. Your on-premises content shows up in the search results in your Search Center in Office 365, site search in Office 365, and in Office Delve.

Consider a larger enterprise installation that wishes to prevent the availability of a particular service application (for example, Access Services) to a specific web application.

From a functional point of view, this effort requires three steps:

1. Create a new proxy group.

2. Associate all service application proxies to this group, except Access Services and Access Services 2010.

3. Associate the web application to the newly created proxy group.

To create a service application proxy group, you would then issue the following PowerShell commands, first creating and then verifying the existence of a new proxy group, as shown in Figure 5-6.

Two PowerShell cmdlets are required to associate a proxy with a proxy group. First, the service application proxy is assigned to a variable representing the proxy group member using the Get-SPServiceApplicationProxy cmdlet, and then the variable is assigned to the proxy group via the use of the Add-SPServiceApplicationProxyGroupMember cmdlet, as shown in Figure 5-7.

1. On the Content Type Hub page, select Site Settings (gear icon).

2. In the Web Designer Galleries section, select Site Content Types.

3. Select the link for your content type.

4. Select Manage Publishing For This Content Type.

5. The Content Type Publishing page opens (see Figure 5-9).

6. Choose from the following:

Unpublish makes the content type unavailable; it also “unseals” each local copy of the content type.

Republish publishes the newly updated changes to your content type.

7. Click OK to close this dialog box.

The first two check boxes are the focus here; they determine the configuration of the MMS from a keyword or term set standpoint:

Selecting the first check box (default keyword storage location) enables users to store new keywords into this term store.

Selecting the second check box (default term set location) enables site admins to create new term sets.

Web applications are often associated with more than one MMS application; they can be used to create separate content type hubs. If you elect to have multiple MMS connections provided to a web application, leave these first two check boxes cleared on all but the primary connection.

Content Type Hub This is a singular job that controls the content type log maintenance and manages unpublished content types.

Content Type Subscriber If there are several web applications that are connected or proxied to the MMS, each is assigned a Content Type Subscriber job. This job retrieves content type packages from the hub and applies them to a local content type gallery.

1. Open Central Administration and then select Monitoring.

2. On the Monitoring page, in the Timer Jobs section, select Review Job Definitions.

3. On the Job Definitions page, scroll down and select the Content Type Subscriber job for the intended web application.

4. Alter the Recurring Schedule settings for your job (see Figure 5-11).

5. At the bottom of this page, you can choose one of the following options:

Run Now starts the propagation cycle immediately.

Disable disables the job (not recommended).

OK accepts the changes you made to the propagation schedule.

Cancel discards all the changes made to the propagation schedule.

There are two distinct property types: shared and local. Local properties are available only for a particular term in a term set, whereas shared properties are useful on all reused or pinned instances of the term anywhere within the term store.

1. Open the Term Store Management Tool for your MMS application.

2. Expand a Term Set Group.

3. Select a Term or Term Set.

4. Select the Custom Properties tab.

5. Add a new Shared Property Name and Value.

6. If you are adding a custom property for a term, you can instead choose to use a Local Property Name and Value.

7. Select Add to complete the addition and then click Save.

Term Store Administrators Members of this role can create new term set groups and assign users to the Group Manager or Term Store Manager roles.

Group Managers Members of this role have contributor access and also can add users to the Contributor role.

Contributors Members of this role have full permissions to edit terms and term set hierarchies within their term group.

1. Select Manage Service Applications, then the link of your intended MMS application.

2. The General tab will appear for your MMS application. Add the new user in the Term Store Administrators text box (see Figure 5-12).

1. Select Manage Service Applications, and then the link of your intended MMS application.

2. Choose the link for the term set group to which you want to assign people.

3. On the General Tab, you can now add Group Managers and Contributors (see Figure 5-13).

1. From Site Settings, Site Administration, choose Term Store Management.

2. Select the MMS application you intend to alter.

3. On the General tab, choose View A Sample Import File to download the ImportTermSet.csv file, and alter it to suit (Figure 5-14).

4. Once you’ve completed your alterations to the import file, select the Import Term Set drop-down next to the group that will receive the uploaded terms.

5. Select the import file and then click OK to complete the upload.

They are used to scope search results to a certain type of content or subset of search results.

They are also used to federate queries with the local SharePoint index, a remote SharePoint index, OpenSearch, or a Microsoft Exchange Server index.

If you created search scopes in prior versions of SharePoint, result sources will likely seem familiar. Search scopes have, in fact, been deprecated since SharePoint 2013, and have been replaced outright by result sources. If you upgraded from SharePoint 2010, you might find search scopes still exist and can be used in queries, but they will no longer be editable.

In SharePoint Server 2016, site collection administrators, site owners, and site designers can also create and configure result sources for use at the site collection and site levels. Search service administration privileges are required for creating farm-scoped result sources.

The creation of a new result source involves five major groupings of tasks: providing general information, specifying a protocol, selecting a search result type, developing a query transform, and specifying credential information.

Providing general information Entering a Name and Description for the result source.

Specifying a protocol Allows the selection of the search protocol to be used.

Local SharePoint provides results from the index of this Search service.

Remote SharePoint provides results from the index of a Search service located in another farm.

OpenSearch 1.0/1.1 provides results from a search engine using this protocol.

Exchange provides results from an Exchange source.

Selecting a search result type Depending on the protocol chosen, different options are presented.

For local SharePoint, either SharePoint or People search results can be searched.

For remote SharePoint, the Remote Service URL must be entered for the remote farm, then SharePoint or People search results can be searched.

No options are presented for OpenSearch 1.0/1.1.

Exchange sources request that you specify the Source URL (or optionally, use AutoDiscover).

Developing a query transform By using the query builder, transforms can be entered to focus on a particular type of content. For instance, a search for OneNote files might use a transform such as “{searchTerms} fileextension=one”.

Specifying credential information Each protocol type requires a different set of credentials for authentication.

Local SharePoint can select either Default Authentication (using their default SharePoint authentication credentials) or Basic Authentication (the same user name and password are provided regardless of user).

Remote SharePoint can select either Default Authentication or Single Sign-On ID credentials.

OpenSearch can either select Anonymous (no authentication required) or choose from Basic, Digest, NTLM/App Pool Identity, NTLM/user name and password, Form Authentication, or Cookie Authentication.

Query rules can be defined at one of three possible levels:

Search service application Applies to all site collections in the web applications that consume the Search service application

Site collection Applies to all sites within a particular site collection

Site Applies to a particular site

Creating a new query rule at the Search application level requires only a few steps. In this example, you will build a new query rule that acts much like a best bet would have in prior versions of SharePoint. This rule will fire when the user queries for the following words: 401K, Hiring, Interviewing, Retirement, or Termination.

To create this example query rule at the Search application level, follow these steps.

1. Select Manage Service Applications, then the link of your intended Search service application.

2. In the Queries And Results section of the Search Administration page, select Query Rules.

3. On the Manage Query Rules page, choose the For What Context Do You Want To Configure Rules drop-down list and select Local SharePoint Results (System). Next, select the New Query Rule link (see Figure 5-16).

1. In the General Information Rule Name text box, type Human Resources.

2. In the Query Conditions section, choose the Query Matches Keyword Exactly drop-down value. In the text box beneath the drop-down list, enter the following string of terms in the field: 401K;hiring;interviewing;retirement;termination.

3. In the Actions section, click the Add Promoted Result link, then enter the following values for Title and URL:

Title: Human Resources

URL: http://hr.wingtiptoys.com

You can optionally choose to render the URL as a banner instead of a plain hyperlink.

Users who search for a term present in the Query Conditions (Retirement) now see the promoted site (Human Resources—hr.wingtiptoys.com).

There are two major groupings of display templates:

Control templates These control the organization and layout of search results as well as the overall look of the Web Part.

Item templates These provide the display characteristics of individual items displayed in search results within the Web Part.

Editing a search Web Part (the Content Search Web Part is shown in Figure 5-17) enables you to configure its properties. Within the Display Templates section, you see the Control and Item selection drop-down boxes.

Display templates are housed within the Master Page Gallery of a given SharePoint site. Probably the easiest way to document this location is to use Design Manager to tell you the location.

1. On the Settings menu (click the gear icon) in your site collection, select Design Manager.

2. After the Design Manager page appears, select item 3, Upload Design Files. The URL shown (for example, http://intranet.wingtiptoys.com/_catalogs/masterpage/) is the location of the Master Page Gallery. Copy this URL to your clipboard.

1. Map a network drive from your client machine using the URL you previously captured.

2. In the newly mapped drive, open the Display Templates folder (Figure 5-18). Note that the Content Web Parts, Filters, Search, and System folders each maintain both control and item templates (not shown).

3. Open the Content Web Parts folder to display a series of control and item templates. Among them are the Control_List and the Item_Picture3Lines templates, which correspond to the List and Picture On Left, 3 Lines On Right item templates you saw in the Search Web Part.

There are three distinct levels of SEO configuration within SharePoint 2016: publishing pages, site collection, and XML site mapping.

Name A name for the page that could appear in the URL.

Title A phrase that can be used in search for keyword-based queries.

Browser Title A friendly title for the page that can appear in the title bar of web browsers viewing the page (must be between 5 and 25 characters).

Meta Description A description for the page that would appear on a search results page.

Keywords Individual words that describe the intent of the page. These words will be used during keyword searches, influencing the ranking of the page in Search. This field allows between 25 and 1,024 characters.

Exclude From Internet Search Engines This option enables you to include or exclude this page from the sitemap (if used) and search engines.

Most of these properties are stored as columns within the Pages library, although some are mapped a bit differently (for instance, Exclude From Internet Search Engines becomes Hide From Internet Search Engines in the page’s properties).

Additionally, you can use the last property, Exclude From Internet Search Engines, to both hide the page from an XML-generated sitemap and hide it from search engines.

To configure SEO properties for a publishing page, follow these steps.

1. Check the page out.

2. On the ribbon, select the Page tab and then select the Edit Properties drop-down list. Select the Edit SEO Properties menu item.

3. Change the values for the SEO Property fields and click OK to commit your changes. Note that changes made in this menu appear instantly on the site (such as changing the title of the page from Home to Intranet Home).

4. Check your page back in and publish it for the change to take effect.

Verify Ownership Of This Site With Search Engines You can go to a search engine’s webmaster tools page and request the appropriate <meta> tag that is specific to your URL.

Consolidate Link Popularity With Canonical URLs Allows you to specify filter link parameters that can help search engines combine the appropriate URLs on your site from a search metric point of view.

To configure SEO properties for a site collection, follow these steps.

1. Navigate to the root site of your site collection.

2. On the Settings menu (gear icon), select Site Settings.

3. In the Site Collection Administration section, select Search Engine Optimization Settings.

4. On the Search Engine Optimization Settings page, select the appropriate values for the two optimization settings sections and then click OK.

Enabling this feature adds two additional fields to the SEO Properties page, Sitemap Priority and Sitemap Change Frequency (Figure 5-19).

Sitemap Priority This value indicates the weighting (between 0.0 and 1.0) of the importance of one page over another within your site collection; the higher the number, the more important your page.

Sitemap Change Frequency This selection enables you to tell the search engine how regularly you expect the page to change to help improve search engine efficiency.

When a query is made via Search (for instance, a series of Microsoft Word documents), the result set that is returned is evaluated for the way the returned information should be presented. The Result Type defines the fields for evaluating the result set in terms of conditions and actions.

Conditions For the Microsoft Word result set, two conditions are evaluated:

Which result source should the item match? (All sources)

What types of content should match? (Microsoft Word)

Actions One action is applied, consisting of two parts:

What should these results look like? (Word item)

What is the appropriate display template URL? (for example, ~sitecollection/_catalogs/masterpage/Display Templates/Search/Item_Word.js)

For a user to be able to search on a piece of metadata, this crawled property must be mapped to a managed property. After this task is complete, queries can be performed against this metadata (such as looking up an item’s order number, stored in a list column).

To view the properties that have been crawled, select Search Schema from within the Queries And Results section of your Search service application. The resulting page shows a series of three links: Managed Properties, Crawled Properties, and Categories.

Allow access to the Office Graph (default).

Don’t allow access to the Office Graph.

A person’s page in OneDrive for Business will only show user profile information.

Users can search for other people in the organization, but have no ability to search for documents or boards in a Delve-like fashion.

You might have a team that generates and maintains its own term set in one site collection, and then wants to provide that term set to several other site collections. This cross-site collection term set access is Read-only, meaning that the sites allowed access are not themselves allowed to configure any new term sets or terms; however, the consuming site collections can bind and pin term sets from this group into their own term sets.

Configuring term set access in SharePoint 2016 can be done in a few steps.

1. Select Term Store Management.

2. Select the term set group for your site collection.

3. On the General tab for your term set group, scroll down to the Site Collection Access section.

4. Within the Site Collection Access section, enter the names of each site collection that should have access to this term set group (one per line, shown in Figure 5-24).

Choosing to have a term set be used for site navigation activates the Navigation and Term-Driven Pages tabs on individual navigation terms. After a term set is configured to be used as site navigation, its individual terms can function as navigation nodes.

Navigation Node Title You can choose to use the title as given by the term set or select the Customize check box and enter a more appealing name.

Navigation Hover Text Editing this text enables more helpful information to be displayed when the user moves a mouse over the link.

Visibility In Menus This setting enables you to determine whether the link will be available to global navigation, current navigation, or both.

Also located on the Navigation tab is the Navigation Node Type and the Associated Folder. The Navigation Node Type enables you to specify whether the node is simply a link or header, or whether the node sends users to a term-driven page. The Associated Folder allows you to specify a folder of pages that will be associated with this node (see Figure 5-27).

If you choose the Simple Link Or Header option, you can either choose to enter the URL of a link or simply a header that can have links beneath it.

If you choose the Term-Driven Page With Friendly URL option, this node will be a friendly URL to a term-driven page (configured on the Term-Driven Pages tab).

Configure Friendly URL For This Term Defaults to a friendly URL that matches the name of the individual term. Selecting the optional Customize check box allows you to change the friendly URL from its default.

Target Page Settings Enables you to choose both the target page for a term as well as specifying another target page for children of the term.

Category Image Allows you to specify a display image meant to represent this term in the Term Property Web Part.

Catalog Item Page Settings If you are using the term as a catalog category (covered in the next section), this setting enables you to choose both the Catalog Item Page for the category and another Catalog Item Page for children of the category.

Most often, these pages are used within the confines of a site collection that has been enabled with the cross-site collection publishing feature. There are two distinct page types available:

Category pages Used specifically to display a series of catalog items.

Item pages (also called catalog item pages in some documentation) Used to display an item in detail.

To specify the catalog pages’ master page as part of the connect-to-catalog process (Figure 5-29), follow these steps.

1. Navigate to the root site of your Publishing site collection, and in Site Settings, Site Administration, choose Manage Catalog Connections.

2. On the Manage Catalog Connections page, select Connect To A Catalog.

3. On the Connect To Catalog page, click Connect for your product catalog.

4. In Catalog Source Settings, scroll to the bottom of the page.

5. The last section of the page enables you to choose from the two out-of-the-box (OOB) master page types (or use an existing one if you have it created; see Figure 5-29). Category and item pages can also be configured within this section.

The number of options available has to do with whether you are changing this setting at the term set or term level:

If you are making this change at the term set level, your only option is to specify a custom page for the catalog item.

If you are making this change at the term level, you have two options: change the catalog item page for a category, and also change the catalog item for children of the category.

These custom properties have two distinct scopes, shared and local, that define at which levels a custom property can be applied.

Shared properties Properties that can be used on all reuses of pinned instances of the term anywhere in the term store.

Local properties Properties that can be used only for a term contained within the term set being configured.

Configuration of a custom property is done within the Term Store Management Tool. To build a new custom property, follow these steps.

1. In the Site Administration section of your site collection, select Term Store Management.

2. Expand your term set until you get to the desired location for your custom property, then select the term or term set and click the Custom Properties tab (see Figure 5-30).

3. Click the Add link for either Shared Properties or Local Properties and then enter values for the property name and a matching value.

Search refiners are represented within the Refiner Web Part, which is exposed by editing the search results page from within the Enterprise Search Center site.

Easily the more reusable of the two refinement options, using managed navigation for refinement enables you to set up refiners at the term or term set level. If you need more granular controls over refiners, you can configure each and every term; however, most refiners are applied at the term set level and then inherited by the individual terms.

1. In the Site Administration section of your site collection, select Term Store Management.

2. Expand your term set until you get to the desired location for your refiner to be applied and then select the term or term set.

3. Select the Faceted Navigation tab, shown in Figure 5-32. In this instance, the term selected already has custom refiners, but you can add refiners as well.

4. Clicking Customize Refiners causes refiner tags to be added to the Selected Refiners section, shown in Figure 5-33. Near the bottom of the screen, you can also see the sample values for the selected refiner. If you scroll down, you will see a Preview Refiners button; clicking this button enables you to see a preview of the Refinement Web Part output using this refiner.

5. Completed refiners for a term appear on the Faceted Navigation tab. Returning to the site where the Refinement Web Part was configured and selecting the Use The Refinement Configuration Defined In The Managed Navigation Term Set option completes the action (Figure 5-34).

Enabling list refinement is done via the list’s Settings page, from the Metadata Navigation Settings link (Figure 5-35). Adding a Key Filter component enables you to refine the contents of the library based on Selected Key Filter Fields.

Configuring a search workload consists of the steps necessary to initially gauge the desired relevancy within a SharePoint 2016 farm as well as the balancing act between overall performance and keeping Search up to date. Taking the long view of optimization, search analytics reports can be used to track trends in customer Search usage, providing feedback on the optimization changes that have already taken place.

General purpose Compute the relevance rank for most types of search results.

People search People search ranking models compute the relevance rank for search results that are related to people. They calculate, among other things, how relevant search results are based on social distance and expertise.

Special purpose Special purpose ranking models compute the relevance rank for search results related to various specific ranking scenarios. For example, there is a ranking model to calculate the ranking score for recommendations, and there are ranking models to calculate relevance ranks for cross-site Publishing sites that have an associated product catalog.

Query rules Define which actions to take when a query matches a condition.

Use promoted results.

Add a results block for promoted results.

Change the query itself.

Search schema Configure the context of a managed property.

Change the context of a managed property in Advanced Searchable Settings.

Create custom ranking model As you have already seen, there are several ranking models already in use; before attempting to create a new one, use query rules with an existing one to try and meet your goals.

There are three major factors in a SharePoint farm that can negatively affect index freshness:

Crawl frequency Search indexes that are not updated frequently ultimately cannot be fresh. Although full and incremental crawls work together to index content, the minimum window of freshness can be no smaller than the incremental crawl time. Moving instead to a continuous crawl mechanism by default causes the index freshness to be a minimum of 15 minutes, although this interval can be altered in PowerShell.

Crawler impact rules for external servers Similarly, the rate and frequency with which external servers are crawled has a direct bearing on index freshness for those content sources. Where possible, configure the crawler impact rules to give the absolute best possible crawl performance without causing undue load to the external system.

Permissions assignment Assigning users individual permissions on a site can negatively influence crawl performance. Where possible, combine these users into SharePoint groups and assign the permissions to the groups. For the absolute best crawl performance (and thus, index freshness), assign permissions via Active Directory group where reasonable and possible.

Search provider or source URL Search results are retrieved from a location, such as the search index of the local SharePoint Search service.

Protocol used for search results The search crawler can use many distinct protocols (shown earlier in this chapter: Local SharePoint, Remote SharePoint, OpenSearch, or Exchange) to parse content into an index.

Query transform This is used to narrow results to a specified subset of content.

Search experiences in the Enterprise Search Center each line up to a respective result source. Four result sources are configured in an OOB SharePoint 2016 implementation:

Local SharePoint Results Provides the Everything Search experience, retrieving any and every type of matching content in the SharePoint farm.

Local People Results Narrows the items queried to information concerning people in the organization.

Conversations Narrows the items queried to simply conversations in microblogs, newsfeed posts, and community sites.

Videos Local videos in the SharePoint farm.

List level Within a list or document library, the option is given to Find A File, thus limiting a search query to the contents of the current list or library (Figure 5-36).

Site level Site-level searches are performed from the standard Search box, near the upper right corner of a site. Search This Site is the default search experience in this box, although selecting the drop-down list allows the user to select from the Everything, People, Conversations, or This Site search experiences (Figure 5-37).

Search Center An Enterprise Search Center expands on the abilities of Search, providing search experiences in combination with refiner information to further tune search results (Figure 5-38).

Query rules Used to promote specific search results, show blocks of additional results, or influence the ranking of search results.

Result types Use a combination of keywords and synonyms, managed properties, search experiences, and result sources to narrow the search result.

Display templates Used in search-specific Web Parts, these templates are based on a combination of HTML and JavaScript to control the appearance and formatting of the search results.

1. The Search service application must be created, and content sources must have an initial, full crawl run on them.

2. The results of a crawl are crawled properties; certain crawled properties can then be selected and then mapped to managed properties, which can be used to narrow search results.

3. Changing certain items within the search schema requires a reindex of the content, including crawled to managed property mapping, token normalization, complete matching, company name extraction, custom entity extraction, and enabling the queryable, retrievable, refinable, and sortable property settings.

Analytics processing component This component runs the different analytics jobs used to extract information as it’s being crawled.

Analytics Reporting database This database stores statistical information from the different analyses.

Link database This database stores extracted information from the analytics processing component along with information about clicks in search results.

SharePoint 2016 Search requires four application databases: Administration, Crawl, Link, and Analytics Reporting.

SharePoint 2016 allows 25 index partitions, each with 20 million indexed items per partition for a total of 500 million items.

In a MinRole-compliant farm, all search components exist on servers hosting the Search role.

Changing a Search topology with an active index requires that you clone the existing topology, alter the clone, and then activate the cloned topology as the active topology.

Search content sources can include SharePoint sites, websites, file shares, Exchange public folders, Lotus Notes (connector install required), Documentum (connector install required), LOB data (using BCS), and custom repositories (requires custom development).

The three search crawl types are full, incremental, and continuous (which defaults to running at 15-minute intervals).

Crawler impact rules can be used to minimize the effect of a SharePoint crawl on external systems.

Search results are security trimmed by default.

A SharePoint 2016 hybrid search environment requires the same six components and four databases, though the index, analytics, and content processing components and Link and Analytics databases are all rendered inactive.

The managed metadata service provides a mechanism for defining taxonomical structures throughout your SharePoint 2016 farm.

Several service application proxies can be assigned to a single proxy group. Each service application proxy can also be assigned to multiple proxy groups.

The content type hub relies on two timer jobs to function correctly: the Content Type Hub job and the Content Type Subscriber job.

Terms in a term set can be entered manually or added to an import file that can configure multiple terms at one time.

Result sources can use one of four possible protocols: Local SharePoint, Remote SharePoint, OpenSearch, and Exchange.

Display templates control the formatting and presentation of search results.

Crawled properties can be mapped to managed properties for use within Search.

Topic catalog pages include category pages and item pages.

User search experience is shaped by query rules, result types, and display templates.

You are designing a hybrid search configuration between SharePoint 2016 and Office 365. As Office 365 interacts with multiple technologies, you have been asked several questions, intended to help shape how SharePoint will interact in the Office 365 space.

1. What sort of PowerShell scripts must be run to connect your SharePoint 2016 cloud Search service application to the Office 365 tenant?

2. What components might you use that tie the on-premises environment of SharePoint to SharePoint Online?

3. As your organization has previously implemented SharePoint, management would like to know what benefits hybrid search offers.

1. Two PowerShell scripts are required for this configuration, CreateCloudSSA.ps1 (used to create the cloud Search service application) and Onboard-CloudHybridSearch.ps1 (used to onboard your cloud Search service application).

2. Your organization could use OneDrive for Business, Hybrid Sites, and cloud hybrid search to unify the on-premises and cloud deployments of SharePoint.

3. Hybrid search offers the opportunity to provide a fully secure search index in Office 365. This index will be used for both on-premises and cloud search results, unifying the search experience for users. Search also provides the benefit of Delve and Office Graph, allowing users to find people both on their team and in others that work on similar projects within the organization.