Deploying a hybrid SharePoint infrastructure can provide great benefits to your enterprise and enable your business users to be more productive, while using the latest and greatest technologies that Microsoft offers.
In this chapter, we will look at how to deploy a hybrid SharePoint Server 2019 infrastructure from the requirements to the Hybrid Sites, OneDrive for Business, Hybrid Taxonomy, and the Cloud Search Service Application.
What Is a Hybrid Deployment?
Before going into technical details, let’s first understand what a SharePoint hybrid deployment is. A hybrid SharePoint deployment is a link between a SharePoint Server farm and Office 365. The SharePoint Server farm can be hosted in our own datacenter, in a private cloud, or in a public cloud such as Azure or AWS.
There are multiple reasons to deploy a hybrid SharePoint Server 2019 Infrastructure. As you probably heard countless times already, Microsoft’s vision is Cloud-First, Mobile-First, meaning that all the newest features come in the cloud first and then make their way in the next On-Premises release. Furthermore, some features such as Delve, Office 365 Groups, Flow, PowerApps, and Stream will not be available as purely on-premises servers.
At the same time, there are multiple reasons to keep using SharePoint On-Premises. Companies can easily customize SharePoint 2019 to fit their business requirements with farm solutions and timer jobs, as well as keeping control of all the SharePoint settings and configurations. Furthermore, due to legal, compliance, or security reasons, some companies simply cannot store some of their data in Office 365 as the data must be protected in certain ways.
This is why a Hybrid deployment is the best of both worlds. By using the right system for the right business need, your business users will be able to have the custom SharePoint solutions and control they need On-Premises, as well as the latest and greatest features in the cloud.
Authentication and Authorization
A big part of setting up a hybrid Microsoft environment is setting up the user synchronization between the On-Premises Active Directory and Azure Active Directory, which is the directory used by Office 365. Microsoft provides DirSync, now deprecated, Azure AD Connect, and a Microsoft Identity Manager Management Agent. Third parties also provide synchronization agents.
The next topic is providing Single Sign on (SSO). An SSO solution will make sure that your users only log in once on the local Active Directory and don’t have to re-enter their credentials when trying to use a cloud service. Microsoft offers Active Directory Federation Services (ADFS) as well as Azure AD Connect SSO to achieve SSO functionality, which is the preferred solution. Providing an SSO solution is optional (but highly recommended) for your user experience and is not mandatory for setting up a hybrid SharePoint 2019 Infrastructure.
In this book, we will not cover the differences between User Synchronization solutions and how to configure them since this action is typically implemented by a Domain Administrator and not by the SharePoint Administrator. In the context of this book we have used Azure AD Connect to sync our On-Premises users to Office 365. Let’s look at the high-level architecture of a hybrid SharePoint deployment.
Architecture Overview
In a Hybrid Infrastructure we have both a SharePoint 2019 On-Premises Farm as well as a SharePoint Online tenant. There is a Server-To-Server Authentication (S2S Trust) setup between the two different systems so that they could authenticate to each other and communicate securely.
Hybrid SharePoint 2019 Infrastructure high-level overview
Inbound features such as Hybrid Business Connectivity Services and inbound federated search are not part of a classic hybrid deployment so we will not cover those in detail in this book.
Hybrid Features Overview
Before starting the configuration, we will do an overview of what features are available in hybrid and what each one offers!
Hybrid App Launcher
The Hybrid App Launcher
Hybrid Sites
Hybrid Sites
Hybrid OneDrive for Business
Hybrid OneDrive for Business
Hybrid Business to Business (B2B) Sites
While you will see this feature in the hybrid configuration wizard, which we will talk about later, this feature does not really create any integrations between your SharePoint On-Premises farm and Office 365 tenant. It’s only there as a reminder of the extranet features in SharePoint Online and how you can benefit from hosting your external collaboration sites in Office 365 rather than On-Premises.
Note
You can learn more about using SharePoint Online as a business-to-business (B2B) extranet solution on Microsoft Docs at the following link: https://docs.microsoft.com/en-us/sharepoint/create-b2b-extranet .
Hybrid Self-Service Site Creation
Hybrid self-service site creation allows you to redirect the default self-service site creation page in SharePoint Server (if you have it enabled) to SharePoint Online. By enabling this feature, you can make sure all newly created sites are in SharePoint Online, therefore having less content to migrate in an eventual migration to Office 365.
Hybrid Taxonomy and Content Types
The hybrid taxonomy and content types feature allow you to have a shared taxonomy and set of Content Types between your SharePoint Online tenant and SharePoint On-premises farm. After the initial term store migration is done via PowerShell, managed metadata terms and Content Types will be synced to SharePoint On-Premises via a daily timer job.
Hybrid Business Connectivity Services
Hybrid Business Connectivity Services
Hybrid Search
SharePoint Server 2019 offers us two options to integrate search between SharePoint On-Premises and SharePoint Online. The first option is called Federated Search. In a Federated Search setup, SharePoint Server 2019 can show results from SharePoint Online by making a Remote SharePoint query, and users can also search SharePoint On-Premises directly from SharePoint Online. What is important to understand is that in a Federated Search scenario, the index stays on the same system as the data. The SharePoint Server 2019 index remains On-Premises while the SharePoint Online index remains in the cloud.
The second option is called Cloud Hybrid Search. This option requires a different type of Search Service Application called the Cloud Search Service Application, and the main difference between Federated Search and Cloud Hybrid Search is that in a Cloud Hybrid Search scenario, SharePoint Server 2019 pushes the index of On-Premises items and documents to Office 365, where it’s merged with the SharePoint Online index. By having the index of both On-Premises and Cloud documents merged in the cloud, your users will have access to Office 365–only features such as Delve and the Office Graph.
Hybrid Federated Search Overview
In a Hybrid Federated Search setup, the index of SharePoint On-Premises documents remains On-Premises, and all the SharePoint Online index remains in Office 365. When configuring Hybrid Federated Search, we have three possible topologies we can choose from.
One-Way Outbound Topology
In a One-Way Outbound Topology, SharePoint Server can query SharePoint Online; however, SharePoint Online cannot query SharePoint Server. Therefore, a user who logs on to SharePoint On-Premises and performs a search query will be able to retrieve both SharePoint On-Premises and SharePoint Online results. However, a user performing a query on SharePoint Online will not be able to get results from SharePoint On-Premises.
One-Way Inbound Topology
In a One-Way Inbound Topology, SharePoint Online can query SharePoint Server 2019; however, SharePoint On-Premises cannot query SharePoint Online. Therefore, a user that logs on to SharePoint Online and performs a query will be able to see results from both SharePoint Online and SharePoint On-Premises. However, a user performing a query in SharePoint On-Premises will only see results from SharePoint On-Premises and not SharePoint Online.
Two-Way (Bidirectional) Topology
In a Two-Way (Bidirectional) topology, we basically configure both the One-Way Inbound and One-Way Outbound topologies. In this topology, both systems can query each other and therefore return results from the other system.
Hybrid Cloud Search Overview
The main difference in the Hybrid Cloud Search topology is that the Cloud Search Service Application does not store the index on the SharePoint On-Premises; instead, it pushes it to Office 365. Out of the six Search components in the Search Service Application, only the Admin, Crawl and Query components are active. The Index, Content Processing and Analytics components do need to exist, but they are not used in a Hybrid Cloud Search scenario. All the Content Processing and Analytics are done in Office 365, where the Index is stored.
The Cloud Search Service Application can crawl the same type of Content Sources as a normal Search Service Application; therefore, you can push items from Remote SharePoint Sites, File Shares, BCS, and more in the SharePoint Online Index.
One of the disadvantages of the Hybrid Cloud Search topology is that you are limited to the Search customization options of SharePoint Online, since that is where the content processing is done and Index is stored. Therefore, some options like Custom Entity Extraction and Content Enrichment Web Service are not available. The big advantage of the Hybrid Cloud Search is having homogeneous results when doing a query, whether those results come from SharePoint Online or SharePoint On-Premises.
Which Option Should You Choose?
The choice between Federated Search and Hybrid Cloud Search will ultimately depend on your business requirements and on the regulation applicable to your data. In a Federated Search scenario, the index of your On-Premises documents remains On-Premises. In a Cloud Hybrid Search scenario, your index, and therefore the content of all your documents, will be in Office 365. Some regulations about the data and the documents might not allow your business to put the content of your documents in Office 365.
Furthermore, in a Cloud Hybrid Search topology, since the index is stored in the SharePoint Online, all your SharePoint users will have to be licensed in Office 365 even if they only want to search SharePoint On-Premises and never use SharePoint Online. With Hybrid Federated Search, users who are only licensed On-Premises can still search all the SharePoint On-Premises items.
Microsoft recommends using the Cloud Hybrid Search whenever possible since it will provide a better experience for your users, enable cloud-only features on On-Premises content, and save disk space and maybe even SharePoint Server 2019 licenses On-Premises, since you need a small search footprint in your On-Premises SharePoint Server 2019 infrastructure. This is the option that will be covered in this chapter.
Prerequisites
Before starting to configure different Hybrid features, there are a few requirements that you need to have.
SharePoint Server Prerequisites
- 1.
Managed Metadata Service Application
- 2.
User Profile Service Application
- 3.
App Management Service
- 4.
Subscription Settings Service Application
Furthermore, you also need to have MySites configured inside your User Profile Service Application. As we already covered how to create those service applications, we will not cover that again in this chapter. One important thing to note is that the Work Email user property needs to contain the e-mail address that you configured for the user in Active Directory, and the User Principal Name property must be mapped to the userPrincipalName attributed in Active Directory.
Licensing Prerequisites
In order to be able to use hybrid functionalities your users must have licenses assigned in Office 365. By default, when Azure AD Connect synchronizes new users in Office 365, the users are synchronized but no licenses are assigned automatically. You can use group-based licensing in Azure Active Directory to make sure your users are automatically licensed!
Tip
Learn more about group-based licensing in Azure Active Directory over here: https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-licensing-whatis-azure-portal .
Note
If you plan to use the SharePoint 2019 Data Loss Prevention feature and want it to work in a Cloud Hybrid Search mode, you will need to synchronize and assign a license to the SharePoint Farm account as well.
Reverse Proxy Requirements
Windows Server 2012 R2 with Web Application Proxy
Forefront Threat Management Gateway (TMG) 2010
F5 BIG-IP
Citrix NetScaler
Note
To view the up-to-date list of supported Reverse Proxies for a hybrid SharePoint infrastructure, visit the following TechNet article. https://docs.microsoft.com/en-us/SharePoint/hybrid/configure-a-reverse-proxy-device-for-sharepoint-server-hybrid#supported-reverse-proxy-devices .
Support client certificate authentication with a wildcard or SAN SSL certificate.
Support pass-through authentication for OAuth 2.0, including unlimited OAuth bearer token transactions.
Accept unsolicited inbound traffic on TCP port 443 (HTTPS).
Bind a wildcard or SAN SSL certificate to a published endpoint.
Relay traffic to an On-Premises SharePoint Server 2019 farm or load balancer without rewriting any packet headers.
Accounts Needed for Hybrid Configuration and Testing
Office 365 Global Administrator
SharePoint Farm Admin
For the Office 365 Global Administrator account, we recommend having a cloud-only account that does not have multifactor authentication enabled during the setup of your hybrid environment. In this book, we will use a dedicated service account named [email protected].
If you don’t plan to deploy Hybrid SharePoint features to all the users in your organization, consider creating a Security Group in Active Directory with all the accounts that will have access to Hybrid Features. This group should also be replicated to Azure Active Directory. By creating a Security Group, we will easily be able to create an audience in the SharePoint User Profile Service Application and only offer Hybrid Features to those users, while the rest of the users will still be able to use features such as OneDrive for Business Fully On-Premises.
Domain User Requirements
User Logon Name is the same as our Office 365 domain
Certificate Requirements
To configure secure communication between your SharePoint 2019 On-Premises farm and SharePoint Online from Office 365, you will need to update your SharePoint Server Security Token Service (STS) certificate. This certificate is only used to configure Server-to-Server trust between SharePoint Online and your SharePoint 2019 Server farm. Following Microsoft’s best practices, in this book we will use the Hybrid Picker , a Graphical User Interface tool that makes it easier to setup a hybrid environment. This tool will create a self-signed certificate for the Server-To-Server trust. The expiration of the certificate is in year 9999, so you won’t have to worry about replacing it anytime soon.
Software
Microsoft Online Services Sign-In Assistant ( www.microsoft.com/en-us/download/details.aspx?id=39267 )
MSOnline PowerShell for Azure Active Directory ( www.powershellgallery.com/packages/MSOnline/ )
Migrate Your Taxonomy and Content Types to SharePoint Online
If you plan to use the Hybrid Taxonomy and Content Types feature as part of your hybrid deployment, it’s recommended to migrate your Terms and Content Types to SharePoint Online before starting to configuration process. This can be done later, after the feature has been implemented, but it would require you to re-run the configuration wizard.
The copy process will preserve most information about the term sets such as owner and stakeholders, however only users can be copied over; Active Directory groups assigned as owners or stakeholders will not be copied over to Office 365.
LocalSiteUrl: The URL of the site where you want to migrate Content Types from. This could be your local Content Type Hub.
LocalTermStoreName: The name of the Term Store from your local Managed Metadata Service Application.
RemoteSiteUrl: The URL of your SharePoint Online tenant (https://<tenant>.sharepoint.com).
GroupNames: The name of the Term Groups you want to migrate to SharePoint Online.
Credentials: A credential object with the credentials of your Office 365 Global Administrator.
LocalSiteUrl: The URL of the site where you want to migrate Content Types from. This could be your local Content Type Hub.
LocalTermStoreName: The name of the Term Store from your local Managed Metadata Service Application.
RemoteSiteUrl: The URL of your SharePoint Online tenant (http://<tenant>.sharepoint.com).
ContentTypeNames: The name of the Content Types you want to migrate to SharePoint Online.
Credentials: A credential object with the credentials of your Office 365 Global Administrator.
Configuring Hybrid SharePoint
The Hybrid Picker Download Location
Installing the SharePoint Hybrid Configuration Wizard
Credential Validation in the SharePoint Hybrid Picker
SharePoint Hybrid Prerequisite Checker
Hybrid Feature Selection
Local Site URL: URL of a Root Site Collection inside your on-premises SharePoint Farm.
Local Term Store Name: The name of the Term Store from your local Managed Metadata Service Application. This might be different than the name of your Service Application!
Remote Group Names: The names of the taxonomy groups that you want to replicate from SharePoint Online to On-Premises. If you don’t specify any names, it will replicate all groups except system ones.
Remote Content Type Names: The names of the content types that you want to replicate from SharePoint Online to On-Premises. If you don’t specify any names, it will replicate all Content Types.
Configuring Hybrid Taxonomy and Content Types
Hybrid Search Parameters in the SharePoint Hybrid Configuration Wizard
Configuration Summary of the SharePoint Hybrid Configuration Wizard
After the Wizard finishes running, an IISReset must be performed on all SharePoint servers of the farm.
Now that we have configured all Hybrid OneDrive for Business, Hybrid Sites, the Hybrid App Launcher as well as Hybrid Taxonomy and Content Types, let’s verify that they work properly and view other configuration options that we have for those features. Afterwards, we will configure the Cloud Search Service Application via PowerShell.
Hybrid OneDrive for Business
Configure Hybrid OneDrive and Sites features in Central Administration
In the My Site URL section, the SharePoint Hybrid Picker has automatically entered the My Site URL of your SharePoint Online Tenant. This can be found in the SharePoint Online Admin Center and is the one under the format https://tenantName-my.sharepoint.com .
Configuring Hybrid OneDrive for Business Settings
OneDrive and Sites
OneDrive only
None
Hybrid Feature Selection Radio Boxes
Since we have decided to activate both features in the hybrid picker, the OneDrive and Sites feature is selected. If you want to disable the Hybrid Sites feature and only keep OneDrive for Business, you could select OneDrive Only.
Hybrid-Enabled App Launcher and Hybrid OneDrive for Business
Out-of-the-box SharePoint 2019 App Launcher
An important aspect to remember when configuring Hybrid OneDrive for Business is that if you had implemented OneDrive for Business On-Premises before and your users had already put documents in their SharePoint 2019 OneDrive for Business, those files will not be automatically migrated to Office 365. You will need to migrate those files manually, by using PowerShell or a third-party product. The original My Site and OneDrive are still accessible for your users if they access it by entering the URL in the browser. Make sure to educate your users to use the version in Office 365 for business after you have successfully migrated their content.
Tip
If you want to avoid users having the “Welcome” screen and wait for their OneDrive for Business to be created in Office 365 on their first use, you can preprovision them by following this TechNet article: https://docs.microsoft.com/en-us/onedrive/pre-provision-accounts .
Hybrid Sites
The Hybrid Sites configuration is really tied with OneDrive for Business as you have seen from the preceding screenshots. The Configuration is done via the SharePoint Server 2019 Central Administration in the Office 365 ➤ Configure hybrid OneDrive and Sites features section. You cannot enable Hybrid Sites without enabling hybrid OneDrive for business, and both Hybrid Sites and Hybrid OneDrive for Business share the same audience
The Follow Site button
Afterward , when you go in the Ribbon and click the SharePoint icon, you should be redirected to the SharePoint Home in Office 365.
On the Sites Page seen in Figure 14-21, you should see the site you just followed from your On-Premises SharePoint Server 2019 Farm. In our case, that site is called “IT Team Site” and you can see it in Office 365.
Tip
It can take a few minutes until the site shows up in the SharePoint Home.
On-Premises and SharePoint Online sites in the Office 365 Sites Page
Hybrid Taxonomy and Content Types
The Taxonomy Groups Replication Timer Job
Synchronized Terms
It is very important to train your delegated taxonomy administrators (anyone who is allowed to modify terms, Term Group or Term Set) to use SharePoint Online to create Term Groups, Term Sets and terms and not to create, or modify those On-Premises. If terms are created, modified, or deleted On-Premises, they will not be replicated to SharePoint Online. Furthermore, if the same term is added On-Premises first and in SharePoint Online After, it will be synced with a GUID inside the name on the next run of the timer job.
Tip
You can set the Term Set On-Premises to “Closed” while leaving it “Open” in SharePoint Online. When a term set is closed, only metadata managers can add terms to this term set. This will block users on-premises from adding terms to an existing Term Set.
Hybrid Cloud Search
The Hybrid Cloud Search is a feature that we recommend you manually do, as it will allow you to better control the parameters of your new Search Service Application!
Setting Up the Cloud Search Service Application
The Cloud Search Service Application checkbox
However, creating it via the Central Administration will cause your database name to have GUIDs, which we do not want. The other way is of course PowerShell. In this chapter, we will reuse the PowerShell code we saw in Chapters 3 and 6; the only difference is that you need to add the -CloudIndex $true Parameter when running the New-SPEnterpriseSearchServiceApplication Cmdlet.
Note
If you are running on a SharePoint server running the Custom MinRole, make sure you start the Search Service Instance before creating the Cloud Search Service Application. Starting the Search Service Instance was covered in Chapter 6.
The Cloud Search Service Application information box
You then need to create your Content Sources, but do not start the crawl yet as we need to set up the connection between our Cloud Search Service Application and SharePoint Online. This Process is called On-Boarding.
On-Boarding Process
The On-Boarding Process is done by a script provided by Microsoft in the Download Center. This download called “Windows PowerShell scripts to configure cloud hybrid search for SharePoint” can be found at www.microsoft.com/en-us/download/details.aspx?id=51490 .
Note
The OnBoarding process is only needed if you configured the Cloud Search Service Application via PowerShell or the Central Administration. You do not need to do this if you configured it with the Hybrid Configuration Wizard.
The zip file contains two scripts. The first script called CreateCloudSSA.ps1 is used to create the Cloud Search Service Application, but we already created it so you can discard it. The important script that we will use is called Onboard-CloudHybridSearch.ps1.
To run the Onboard-CloudHybridSearch.ps1 you need to have the Microsoft Online Services Sign-In Assistant as well as the Microsoft Azure AD PowerShell installed on the server on which you wish to run it. We have previously installed those prerequisites when we set up the Server-to-Server Authentication earlier in the chapter, so we will use the same server to run this Onboard script.
- 1.
PortalURL: The Root Site Collection of your SharePoint Online Tenant
- 2.
CloudSsaId: The GUID or Name of your Cloud Search Service Application
- 3.
Credential: A PSCredential object containing the credential of a Global Office 365 Administrator
- 1.
Get-HybridSSA
This section validates that the CloudSsaId you gave is a valid Cloud Search Service Application.
- 2.
Prepare-Environment
This section validates you have the Microsoft Online Services Sign-In Assistant and Microsoft Azure AD PowerShell installed on the server.
- 3.
Connect-SPFarmToAAD
This section checks if you already have a Server-to-Server Authentication setup and use it, or else it will create one for you. Since we have created one previously in this chapter, it just used the existing one. This section also creates a new SharePoint Online connection proxy to allow the farm to communication with the external endpoint of the cloud Search Service.
- 4.
Add-ServicePrincipal
This section will add the Office 365 Service Principal ID as well as create four new service Principals.
The script should finish without any errors and without breaking any existing Hybrid Configurations.
Note
The script might throw an error on “Restarting SharePoint Server Search…” if you did not run it from a Search MinRole server or a server running the Search Services. Make sure to run the Restart-Service OSearch16 cmdlet or manually restart the Search service on all the Search Servers in your Cloud SSA topology.
When the On-Boarding Process is done, it’s time to crawl our content and see if everything worked.
Crawling and Testing
In order to get the data in Office 365, we must first crawl it. Navigate to the Central Administration and to your Cloud Search Service Application. Afterward, go in Content Sources and start a Full Crawl on one of your content sources with SharePoint Content.
Wait for the Crawl to finish and verify the Crawl Log to make sure that there is a high Success Rate and no Top-Level Errors.
Tip
With the Cloud Search Service Application, the Searchable items row in the Search Administration screen will always show 0. You need to check the Crawl Log to see how many items have been marked as successfully crawled.
Testing the Cloud Search Service Application
You can easily verify the content is from On-Premises by looking at the URL. SharePoint Online content always ends with “.sharepoint.com”.
Results from both SharePoint Online and SharePoint On-Premises are shown
Once you get results from both systems it means that the Cloud Search Service Application is functioning correctly, and you can set the schedule and crawl your remaining Content Sources. However, since we don’t have an index On-Premises anymore, all your On-Premises Search boxes and Search Center will not work at this point. Let’s fix them so our users have a great experience and are able to search content from both SharePoint Online and Office 365.
Searching from SharePoint On-Premises
The default result source for Cloud Search Service Application remains the local SharePoint index, the same as a regular Search Service Application. However, since the Cloud Search Service application pushes all the crawled items in the SharePoint Online index, all our On-Premises Queries stop returning results, including the Search Boxes in document Libraries.
What we will need to do is change the default Result Source of the Cloud Search Service Application to use a new Result Source that queries SharePoint Online for the results. This is using the same technique as the Outgoing Federated Search.
To create the result source, open Central Administration, navigate to the Cloud Search Service Application, and go to the Result Sources Page. Click the “New Result Source” button to create a new Result Source.
Creating a Combined Results result source
Creating a Combined Results result source
Setting the Default Result Source
Your On-Premises Search Boxes as well as Search Center will now show results from both On-Premises and SharePoint Online, exactly as if a user searched from the SharePoint Online Search Center.
While having results from both systems combined is an amazing feature, your users might want to target only certain systems. In the next section, we will learn how to customize our Search Results.
Customizing Your Search Results
By creating Result Sources both in SharePoint Online and SharePoint Server 2019, you can create different pages in your Search Centers to show for example only On-Premises results, or only SharePoint Online results.
Something that will also need to be planned is People Search. By default, all the users in the SharePoint Online User Profile Service Application will be indexed by the Office 365 crawler. If you also crawl On-Premises users using your Cloud Search Service Application, your search results will return duplicate data. There are two options you can go with to go around this challenge.
The first option is to use the Office 365 User Profile service as the primary source of user information and let Office 365 search take care of crawling it. This way, you will not need to crawl people data On-Premises.
The second option if you wish to keep the On-Premises User Profile service as primary source of user information, you will need to crawl that On-Premises people data, and then use the Query transformation rules to only display results from On-Premises. This could be done by adding a new result source for People Search and using the isexternalcontent managed property to filter results.
Next Steps
In this chapter, we have learned how to configure a Hybrid SharePoint 2019 Infrastructure and offer additional features to your users. In the next chapter, we will learn how we can use other cloud-only features such as PowerApps and Microsoft Flow with your On-Premises content!