Appendix B

Microsoft Sentinel for managed security service providers

BY JAVIER SORIANO,
SENIOR PROGRAM MANAGER
MICROSOFT CxE SECURITY

Managed security service providers (MSSPs) play a key role in monitoring and managing security devices and systems for their customers. As part of their services, MSSPs might include multiple tasks related to Microsoft Sentinel, like architecture and design, implementation, management, or security incident handling.

Automation is another important pillar that MSSPs must put special effort into. MSSPs must operate at great scales, and therefore streamlining things like customer onboarding is critical to their success.

In this appendix, we focus on how MSSPs can manage and operate multiple Microsoft Sentinel customers, with a focus on automation and efficiency.

Accessing the customer environment

Before the MSSP can start managing and operating a customer environment, they need to have access to it. As you have seen earlier in this book, Microsoft Sentinel is a resource type inside Azure. Therefore, it lives within an Azure Active Directory (AAD) tenant, which belongs to the customer. On the other side, MSSP identities live in a separate AAD tenant, so there must be a way to connect those two identity providers. There are actually two methods: Azure Lighthouse and Azure AD B2B.

Azure Lighthouse

Azure Lighthouse enables cross-tenant management, allowing for higher automation, scalability, and enhanced governance across resources and tenants. This is the preferred method to access your customer environment because it allows you to manage customer resources as if they were in your own Azure AD tenant.

Azure Lighthouse is based on delegations. Each delegation contains three components: Identities, Roles, and Scope.

• Identities These are the identities (normally from the MSSP tenant) that will have access to customer resources. You can specify users, groups, or service principals as the recipients of a delegation.

• Roles These are the permissions that the identities will have when accessing customer resources. The roles that can be used here are all Azure built-in roles, with three exceptions. Custom roles are currently not supported. Also, you cannot grant Azure AD roles.

  Note

  See the differences between Azure and Azure AD roles at http://aka.ms/azureadazurerbacroles.

• Scope This indicates where the delegation will apply; valid scopes are subscription and resource group.

In the context of Microsoft Sentinel, Azure Lighthouse can be used to manage the service across multiple customers. Figure B-1 shows a high-level view of the setup.

As you can see, in this case, the MSSP (Fabrikam) has two delegations for each customer—one for engineers with the Microsoft Sentinel Contributor role and one for analysts with the Microsoft Sentinel Responder role—and all have delegated access at the resource group level where Microsoft Sentinel is located. This will effectively provide them with access to the whole resource group with the permissions included in the granted role.

What can’t you do through Lighthouse?

As we have explained before, the MSSP access to the customer’s Microsoft Sentinel environment utilizes Azure Lighthouse. However, there are things that you won’t be able to do with just Azure Lighthouse:

• You won’t be able to onboard some connectors that require Security Admin or Global Admin permissions in the customer Azure AD tenant. Several Microsoft first-party connectors, like Office 365, Azure AD, or Microsoft 365 Defender, require one of these permissions to be enabled, and these roles can’t be granted through Azure Lighthouse.

• You cannot assign incidents to a user in the customer’s Azure AD tenant. Therefore, as you manage incidents in the customer workspaces, you will only be able to assign them to users in your own tenant.

Later in this appendix, we will review Azure AD B2B invites, which enable these scenarios.

Azure Lighthouse onboarding

As already mentioned, there are two options—an ARM template or an Azure Marketplace offer—the latter being preferred because it provides a very easy experience for customers.

Marketplace offers have an additional concept called a plan. A plan defines the service that you will provide to your customer. For example, you can have a marketplace offer to provide managed services for your customers, and within that offer, several plans with different flavors, including monitoring, backup and recovery, compliance, and fully-managed service.

In the context of Microsoft Sentinel, you could have an Azure Marketplace offer like the one shown in Figure B-2.

As you can see, inside each plan, you define the groups of users from your tenant who will have access to the customer environment and the permissions that will apply. The plan also includes the scope (resource group or subscription), although we’ve omitted it in Figure B-2 for simplicity. Contoso, Wingtip, and Northwind Traders are customers that “purchase” specific plans from the Fabrikam offer.

You can make these plans public, so everyone in Azure can see them, or you can make them private if you only want a subset of customers to have access. This would allow you to create plans targeted just to specific audiences, such as a particular customer or a vertical.

Azure Lighthouse integration with Azure AD Privileged Identity Management (PIM)

Azure Lighthouse also can integrate with Azure AD Privileged Identity Management (PIM). This lets you grant delegated permissions to customer tenants on a just-in-time basis so that users only have those permissions for a set duration.

This can greatly reduce risks because it allows you to limit the number of permanent assignments of users to privileged roles. Because this feature relies on Azure AD PIM, it requires the MSSP Azure AD tenant to have licenses (such as Azure AD Premium P2) for Lighthouse. Also, Lighthouse can assign approvers who will be responsible for granting the requested permissions by the analysts.

Azure Lighthouse is a very important service for getting access to customer Azure resources, but it doesn’t work for other workloads outside Azure, such as Office 365 or Microsoft 365 security services.

Azure Active Directory B2B

Azure Active Directory (Azure AD) business-to-business (B2B) collaboration is a feature within External Identities that lets you invite guest users to collaborate with your organization. MSSP users can be “invited” to the customer tenant to perform management activities in that tenant. MSSP users will appear as guest users in the customer tenant and can then be granted roles within. The main difference with Lighthouse is that the guest users can be granted any Azure (even custom ones) or Azure AD roles. (Remember, Azure Lighthouse can only grant Azure built-in roles.)

The ability to grant Azure AD roles opens new possibilities, such as managing Office 365 and Microsoft 365 services. However, you still need Azure Lighthouse because it provides two important capabilities not available with Azure B2B:

• No cross-tenant management or visibility As you are invited into a customer tenant, you must log in to the customer tenant in order to see its resources. This blocks your cross-tenant visibility because you cannot query multiple tenants simultaneously.

• No ability to invite groups Azure B2B is done on a user-by-user basis, meaning you cannot invite an entire group. This is challenging because you need to manage the lifecycle of each account in multiple places. (This limitation can be removed by using Azure Entitlement Management, which we review below.)

Taking these disadvantages into account, if you, as an MSSP, need to manage both Azure and Office 365/Microsoft 365 workloads, the best approach is to use a combination of Azure Lighthouse and Azure AD B2B invites. Figure B-3 depicts the combination.

In Figure B-3, a new user from Fabrikam (MSSP) has been invited to Wingtip’s Azure AD and is now a guest user in that AAD. Also, this user has been granted the Security Admin role. Notice that Security Admin is an Azure AD role, so it can be granted to guest users, but it cannot be granted to users who have delegated access via Azure Lighthouse. (Remember, Azure Lighthouse can only grant Azure roles.) Although not shown in Figure B-3, the same user can have Azure roles delegated through Azure Lighthouse and can be invited as a guest and be granted Azure AD roles like Security Admin or Global Admin.

Azure B2B invites provide a solution for the need to manage Office 365 or Microsoft 365 environments, but they can be difficult to automate.

Azure AD entitlement management

Azure Active Directory (Azure AD) entitlement management is an identity governance feature that enables organizations to manage identity and access lifecycle at scale by automating access request workflows, access assignments, reviews, and expiration. This feature requires an Azure AD P2 license.

This feature can also be used to manage access from external Azure AD organizations, so it’s a perfect fit for the MSSP access needs when it comes to Microsoft 365 Defender workloads. Using this feature, MSSP users can be automatically invited into the customer tenant (after the appropriate approvals) to manage customer services. You can also assign which specific roles will be granted to those users; these roles should be specially crafted to manage Microsoft 365 Defender workloads.

Remember, this is only needed to manage the Microsoft 365 Defender part of the customer environment. As explained above, the Azure Sentinel part will be managed through Azure Lighthouse.

Cross-workspace features

Now that we have reviewed how to access the customer environment, let’s see how we can manage multiple Microsoft Sentinel customers in parallel.

At a high level, we will use the MSSP tenant as the single pane to look at multiple Microsoft Sentinel workspaces across Azure AD tenants. Figure B-4 shows this setup.

Although this is the generic architecture, each Microsoft Sentinel feature has different characteristics and implementation details.

KQL Queries

Microsoft Sentinel supports querying multiple workspaces in a single query, allowing you to search and correlate data from multiple workspaces in a single query. This is done by leveraging the workspace() operator, which allows you to reference a table in a different workspace.

This can be very useful for SOC analysts when trying to analyze data from multiple customers in parallel. You can even create an alias, so your commands are easier to write and use. For example, if you want to look for failed AAD logins across two workspaces (A and B), you could create a function (alias) called FailedLoginsAB that contains the following code:

Click here to view code image

union isfuzzy=true workspace("workspaceA").SigninLogs, workspace("workspaceB").
SigninLogs
| where ResultType !in ("0", "50125", "50140", "70044", "70043")

Instead of having to write the KQL code above, an analyst can just use the FailedLoginsAB alias, which will return the aggregated results from both workspaces.

Analytics rules

Scheduled Analytics rules also support the use of the workspace operator to reference tables in workspaces other than where the rule is being created. Following are important things to remember about cross-workspace rules:

• All workspaces referenced in the query must be onboarded into Microsoft Sentinel.

• A maximum of 20 workspaces can be used in a single analytic rule.

• Alerts and incidents will only be created in the workspace where the cross-workspace rule is created.

• There might be a performance impact if the same query contains workspaces in different regions.

• Investigation graph functionality for incidents and alerts coming from cross-workspace rules is limited. For example, expansion queries that can be executed on entities won’t work properly.

• Cross-workspace rules are only possible with scheduled analytics rules. Other types of rules are not supported in this mode.

Because of these limitations, the use of cross-workspace analytics rules is only recommended in two scenarios:

• The intellectual property of the MSSP needs to be protected, and therefore the rule must be created in the MSSP tenant. (Artifacts created in the customer tenant will always be visible to the customer.)

• There is a need to correlate data coming from multiple customers (rarely needed in the MSSP scenario).

Figure B-5 shows how an MSSP can protect the intellectual property in an analytic rule by creating it in its tenant but pointing to the customer workspace.

If cross-workspace rules are needed, the following best practices are recommended:

• Unless correlation is needed, don’t mix customer workspaces into a single rule to avoid performance issues and poor manageability. Create one rule per customer.

• If you are managing many customers, consider whether you might hit the current limit of rules per workspace (512). If that’s a possibility, create one workspace per customer in the MSSP tenant and place them in separate resource groups. Figure B-6 shows this scenario.

  

Hunting

Threat hunting is a key function for MSSPs because it allows you to sweep through multiple customer environments in parallel to look for evidence of malicious activity. Microsoft Sentinel has several features related to threat hunting: hunting queries, Notebooks, and Watchlists.

Hunting queries

Like Analytics Rules, hunting queries can also use cross-workspace queries in KQL by utilizing the workspace() operator. As with analytics rules, all workspaces that are part of the query need to be onboarded into Microsoft Sentinel.

Although not strictly related to MSSPs, hunting queries also permit the use of the adx() operator, which can be used to reference data sitting on an Azure Data Explorer (ADX) cluster. In some scenarios, this can be useful for correlating with other data sitting in ADX.

Notebooks

In Chapter 6, you saw many of the great things you can do with Notebooks. In the context of MSSPs, Notebooks can be a very versatile investigation and threat hunting tool. For example, you can have a Notebook that looks for evidence of Log4j exploitation that looks at all your customer workspaces in parallel without looking at each one of them individually.

The first thing you should do to use Notebooks in a multi-tenant setup is to add all your workspaces to your msticpyconfig.yaml file. This allows you to reference whichever workspace you need, depending on the query. Following is an example of what a msticpyconfig.yaml file with three workspaces would look like:

Click here to view code image

AzureSentinel:
  Workspaces:
    Default:
      ResourceGroup: mssp_sentinel
      SubscriptionId: xxx-xxx-xxx-xxx
      TenantId: xxx-xxx-xxx-xxx
      WorkspaceId: xxx-xxx-xxx-xxx
    customerA:
      ResourceGroup: customerA
      SubscriptionId: yyy-yyy-yyy-yyy
      TenantId: yyy-yyy-yyy-yyy
      WorkspaceId: yyy-yyy-yyy-yyy
    customerB:
      ResourceGroup: customerB
      SubscriptionId: zzz-zzz-zzz-zzz
      TenantId: zzz-zzz-zzz-zzz
      WorkspaceId: zzz-zzz-zzz-zzz

Once you’ve done that, there are several ways in which you can utilize these workspaces within your Notebooks:

• Use a cross-workspace query (utilizing the workspace() operator) that will result in a table that will include records from all the specified workspaces. You can then split it into multiple data frames. (This option doesn’t use the definitions in the msticpyconfig.yaml file, but it is a good practice to add them there.)

• Create multiple connections and query each, one by one. You can have multiple queries in one %kql cell. Separate each query with an empty line and assign the result of each query to a different Python variable. You can then aggregate results using the append() function.

• Write Python code that iterates over the different workspaces in your msticpyconfig.yaml file and use %kql for each.

All the options above are valid, and choosing the right one will greatly depend on the specific needs of the MSSP and the type of Notebook you need to create.

Watchlists

Watchlists are another important tool for MSSPs. Besides delivering all the normal functions, they can be used to provide additional context to the MSSP. For example, if the customer is using cross-tenant content like a Workbook, it might be difficult for a customer to identify from which customer the data is coming because the data doesn’t contain the customer’s name per se. By using a Watchlist, we can build a mapping table that correlates the workspace ID (stored in the TenantID field present in every table) with a friendly customer name.

It's important to remember that Watchlists only work in the context of the workspace where they are defined, and we can’t reference a Watchlist in a remote workspace in a KQL query—not even by using the workspace() operator.

Incident management

Microsoft Sentinel allows you to view incidents coming from multiple workspaces in a single consolidated view. For better navigation, this cross-workspace incident view includes new columns indicating the workspace and the directory (Azure AD tenant) the incident is coming from. This view is extremely useful for MSSPs because it allows an analyst to oversee multiple customers from a single pane.

At the time this book was written, the cross-workspace incident view has a limit of 100 workspaces that can be monitored in parallel. Figure B-7 shows the Incidents view with incidents coming from multiple workspaces and tenants.

Once the user drills down into an incident, the user will be redirected to the appropriate workspace where that incident was created.

As we mentioned previously in this appendix, MSSP users managing incidents will only be able to assign them to other users in the MSSP Azure AD tenant. If there’s a requirement to assign to users in the customer tenant, Azure B2B must be used in addition to Azure Lighthouse.

Automation/SOAR

As explained in other chapters in this book, there are two main SOAR components in Microsoft Sentinel: automation rules and Playbooks. In general, if there is no need to protect MSSP intellectual property, we recommend that you create both artifact types in the context of the customer’s workspace. This simplifies the management of credentials used inside Playbooks and allows for the use of managed identities where possible.

However, if protection of the MSSP’s intellectual property is a requirement, Playbooks can be saved in the MSSP tenant, and the automation rules in the customer workspace can use them. This is the same if the Playbook is referenced directly from an analytics rule (see Figure B-8).

There are a couple of additional considerations for this model:

• The cost of the Logic App execution is charged to the MSSP tenant.

• If the Playbook needs to perform some sort of remediation activity in the customer environment, it will need the appropriate credentials. For example, if the Playbook needs to block Azure AD accounts, we will need to provision a service principal in the customer tenant that has the relevant permissions and then use those credentials in the Logic App.

Workbooks

Workbooks can also be modified to query data from multiple workspaces. This can be very useful if you need to see an aggregated view of data coming from multiple customers (see Figure B-9).

Like with other artifact types, hosting your Workbooks in the MSSP tenant can also be used as a way to protect intellectual property inside that Workbook. However, there are situations where the customer also needs to see and interact with that Workbook.

Additional benefits of PowerBI include the following:

• Easier to share You can just send a link to the PowerBI dashboard, and the user will be able to see the report. There is no need to have Azure access permissions.

• Scheduling You can configure PowerBI to send an email on a given schedule that will contain a snapshot of the dashboard.

Security content management

Automation and DevOps practices are crucial components for a successful managed security practice. These are some of the key benefits:

• Reduction of human error

• Much faster deployment and configurations

• Improved change management because changes are tracked in source code control

• Enhanced security as consistency is guaranteed

• Time savings to allow employees to focus on adding value to our customers

When multiple customers are managed in parallel, there will always be content that is deployed to many or even all your customers. If a modification is needed for that content, we must have a way to make that modification only once and then have an automatic process that verifies if the change is valid and that updates any copies of that same content across all our customers. This is achieved via continuous integration and continuous deployment, or CI/CD.

How to adopt CI/CD?

There are several steps you should follow to adopt and implement CI/CD in your content management process:

• Turn your security content (detections, dashboards, Playbooks, queries, and so on) into code that can be interpreted by a machine. The most common formats are YAML and JSON.

• Host your code in a source code repository, such as Git, GitHub, Azure DevOps Repos, or BitBucket.

• Build continuous integration (content validation) and continuous deployment (content implementation) pipelines.

• Choose and configure a DevOps tool that orchestrates it all (such as Azure DevOps, or GitHub).

Luckily for us, Azure is very much built with DevOps in mind and already offers a great way to codify and automatically validate and deploy our content: Azure Resource Manager (ARM) templates.

As with any other Azure resource, Microsoft Sentinel and its associated content can benefit from using ARM templates, so we already have a great way to codify our security content that comes with tools to check the validity of the content and deploy it to Azure environment.

Microsoft Sentinel repositories

Once you have your security content in ARM template format and hosted in a source code repository, you still need to configure a DevOps tool to create your CI/CD pipelines. To make this process even easier, Microsoft Sentinel offers Repositories, a feature that seamlessly integrates with GitHub and Azure DevOps, automating the following steps:

• Automatically creates a service principal (SPN) in Azure AD

• Grants that SPN permissions to deploy content to the Microsoft Sentinel workspace

• Creates a connection from either GitHub or Azure DevOps to the Azure environment

• Places a PowerShell script in the source code repository that can deploy the ARM content in the repository to Microsoft Sentinel

• Creates a CD pipeline that uses the PowerShell script

All the steps above are completely transparent to the user, who just needs to authenticate to the DevOps platform for the setup to be successful.

The repositories feature also allows you to select which content types should be deployed with this feature. Your choices are Analytics Rules, Hunting Queries, Workbooks, Playbooks, Parsers, and Automation Rules.

In the context of MSSPs, repositories can be a very useful tool to automate content deployment to customer workspaces. For example, you can set up a code repository in Azure DevOps where you store your security content. Then you create connections from each of the customer’s workspaces to that repository. Whenever you make a change to your code, the updates will be automatically deployed to all your customers in parallel. If there’s specific content that you only want to deploy to a subset of your customers, you can organize your content in branches or folders, as shown in Figure B-10.

As you have seen in this appendix, Microsoft Sentinel is fully adapted to work in an environment where there are multiple Azure AD tenants involved, as is the case for MSSPs. Additionally, Sentinel uses a cloud-native and API-driven approach, making it ideal for automating at scale so MSSPs can speed up their operations and focus on serving their customers.