Plan an Azure Sentinel workspace

Azure Sentinel is an enrichment layer that sits on top of a Log Analytics workspace. You cannot use Azure Sentinel without first having a Log Analytics workspace created in your Azure tenant. Log Analytics is where all logs that are ingested into Azure Sentinel are stored. There are several aspects of design and architecture to consider before creating your Log Analytics workspace(s) for Azure Sentinel.

First, you must consider the number of workspaces. Where possible, it is recommended that you use one central security workspace. However, there are times when this might not be possible. The main reasons for requiring a multi-workspace deployment are as follows:

• If logs need to be kept in a certain jurisdiction for compliance or regulatory requirements for a global organization

• To reduce Azure region networking egress costs

• For subsidiary organizations that run their own security operations

If you require multiple workspaces, this will take one of two forms:

• Cross-tenant scenario Where multiple Azure tenants each have Azure Sentinel workspaces that need to be centrally managed

• Cross-workspace scenario Where there are multiple workspaces in one Azure tenant that need to be centrally managed

Azure Sentinel can support scenarios where multiple Azure tenants are involved by using Azure Lighthouse, which is an Azure service that allows cross-tenant management, with the MSSP managing the customer’s Azure Sentinel workspace. This architecture can also be used for organizations that have subsidiaries who have their own separate Azure tenants. Figure 3-1 shows how Azure Lighthouse can be used to manage Azure Sentinel workspaces in different Azure tenancies.

There are many advantages to using Azure Lighthouse if your organization chooses to outsource their security operations. Here are a few:

• All data stays in the end customer’s Azure tenant Data is not stored in your MSSP’s Azure tenant and is not mixed with other customer data. This preserves data sovereignty and allows for straightforward offboarding should the need arise.

• MSSPs only have access to the Azure resource(s) that the end customer grants them This is unlike traditional delegated access in on-premises environments where a third-party service provider might have had access to the whole IT environment, even when only a specific application is needed.

• MSSPs get consolidated views of all the customer workspaces they manage They don’t have to log in to each Azure Sentinel workspace separately, which is inefficient and not scalable.

Aside from Azure Lighthouse, there are several features in Log Analytics and Azure Sentinel that allow for investigation of incidents across workspaces in the same Azure tenant, as shown in Figure 3-2 and discussed in detail later in this chapter:

• Cross-workspace queries

• Cross-workspace analytics rules

• Cross-workspace hunting queries

• Cross-workspace workbooks

  

Other aspects of Azure Sentinel design to consider are as follows:

• Azure tenant placement A Log Analytics workspace is an Azure resource, and like with any Azure resource, consideration needs to take place for where this will sit. You will need to define a subscription, resource group, and region. It is recommended that the Log Analytics workspace that you are going to set up Azure Sentinel on top of is placed in a separate resource group for simplicity in configuring RBAC (more detail later). You may choose to place the workspace in an existing Azure Subscription or in a separate one.

• Commitment tiers If you plan to ingest more than 100GB per day into your Azure Sentinel workspace, it is worthwhile looking at commitment tiers that give you a discount on ingestion compared to pay-as-you-go pricing. Note that even if you don’t ingest data up to your commitment tier allotment, you will be charged for it anyway if you change commitment tiers. Commitment tiers need to be configured at both the Log Analytics and Azure Sentinel level.

After you have decided on the number of workspaces and how they will fit into your Azure tenancy, you can enable Log Analytics workspace(s) and subsequently Azure Sentinel by performing the following steps:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel Workspace. The Azure Sentinel page appears, as shown in Figure 3-3.

   

3. Click Create. The Add Azure Sentinel To A Workspace page appears.

4. Click Create A New Workspace.

5. The Create Log Analytics Workspace page appears, as shown in Figure 3-4.

   

6. Configure the Subscription, Resource Group, and Region for your Log Analytics workspace.

7. Select Pricing Tier, add Tags (if required), and then select Create. Wait for your Log Analytics workspace to be provisioned.

8. In the Azure portal, search for Azure Sentinel and select Add on the Azure Sentinel page. The Add Azure Sentinel To A Workspace page appears, as shown in Figure 3-5.

   

9. Select the Log Analytics workspace on which you want to activate Azure Sentinel.

Configure Azure Sentinel roles

As with other Azure resources, Azure Sentinel comes with several built-in Azure roles that you can assign to users who need to access your workspace. Remember to adhere to the principle of least privilege and always assign the absolute lowest level of privilege that a user requires to complete their role. Following are the built-in rules:

• Azure Sentinel Reader Can view data, incidents, workbooks, and other Azure Sentinel resources.

• Azure Sentinel Responder In addition to the permissions granted by an Azure Sentinel Reader role, the Azure Sentinel Responder role allows for managing of incidents.

• Azure Sentinel Contributor In addition to the permissions that Reader and Responder roles have, Azure Sentinel Contributor can create and edit workbooks, analytics rules, and other Azure Sentinel resources.

To assign an Azure Sentinel role to a user, perform the following steps:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Resource Groups, and under Services, click Resource Groups. The Resource Groups page appears, as shown in Figure 3-6.

   

3. Select the resource group that your Azure Sentinel workspace is associated with. The resource group’s overview page appears.

4. From the resource group overview page, select the Access Control (IAM) page, which is shown in Figure 3-7.

   

5. On the Access Control (IAM) page, select Add role assignment, as shown in Figure 3-8.

   

6. The Add Role Assignment blade appears, as shown in Figure 3-9.

   

7. On the Add Role Assignment blade, search for the Azure Sentinel role you want to assign, and in Azure AD, search for the user who you want to assign the role to.

8. Select Save to add this assignment.

Design Azure Sentinel data storage

Earlier in this chapter we explained that Azure Sentinel’s data store is Log Analytics, which itself is a part of the wider Azure Monitor platform. As the name suggests, Azure Monitor is a suite of services in the Azure platform that assist with monitoring.

Log Analytics is an immutable log store that uses different tables to store the logs that it ingests in rows. What this means is that after data has been ingested into a Log Analytics workspace, it cannot be changed or amended and will only be removed from the workspace when the log reaches its retention period and is aged out of the workspace.

It is possible to retain data in a Log Analytics workspace for up to 730 days (2 years), but the default data retention is set to 30 days. When you activate Azure Sentinel on a Log Analytics workspace, you receive up to 90 days of free data retention. Azure Sentinel is priced on ingestion and log retention, so if you choose to retain logs in your workspace for more than 90 days, fees will be assessed. It is recommended that you choose a retention period in your workspace that balances how far back you are likely to actively query your security logs against the cost of retention.

Remember that you can set retention periods in Log Analytics on a per-table basis, so you can opt to retain certain tables for longer than others to reduce the cost of retaining an entire workspace for a longer period. A table that is often kept for longer than others is the SecurityIncident table, as this stores details of the security incidents that have been raised by Azure Sentinel and querying this table allows for SOC managers to see their SOC’s performance metrics, number of incidents raised over time, and so on. Ultimately, a cost/benefit analysis will have to be performed to decide what works best for your implementation of Azure Sentinel; there are tools such as the Azure Sentinel pricing calculator that can help you with this.

To change the data retention settings in your workspace perform the following steps:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Log Analytics, and under Services, click Log Analytics Workspace. The Log Analytics Workspace page appears.

3. Select your workspace, and the Usage And Estimated Costs page appears, as shown in Figure 3-10.

   

4. Select the Usage And Estimated Costs blade.

5. Select Data Retention and move the slider on the Data Retention blade to your desired retention period for your workspace (shown in Figure 3-11) and select OK.

   

Configure Azure Sentinel service security

Azure Sentinel relies on other services for some of its functionality. This means that to use these features, additional permissions other than the built-in Azure Sentinel roles need to be used:

• Using Playbooks for automation In order to use Playbooks in Azure Sentinel, you will also need to assign the Logic App Contributor built-in role because Playbooks are part of Azure Logic Apps, which is considered to be a separate Azure resource and thus, has its own set of permissions.

• Connecting data sources to Azure Sentinel A user must have write permissions on the Azure Sentinel workspace to be able to add a data source. They might also need additional permissions specific to each data source; these are listed on the connector’s page.

• Guest users assigning incidents To assign incidents in Azure Sentinel, guest users require the Directory Reader permissions to be assigned to them. Note that this role is not an Azure role but an Azure Active Directory role and that regular (non-guest) users have this role assigned by default.

• Creating and deleting workbooks In order to create and delete workbooks in Azure Sentinel, a user will need to be assigned the Azure Monitor role of Monitoring Contributor. This is not required for using workbooks. It’s only necessary for creating and deleting workbooks.

Identify data sources to be ingested into Azure Sentinel

Identifying which data sources to ingest into Azure Sentinel is a critical activity that should ideally be decided upon before you begin your implementation. Azure Sentinel makes it easy to identify data sources that can be connected to the product via the built-in data connectors in the data connector gallery.

As a starting point, we recommend that you review the data connector gallery and identify which of these data sources you have in your environment and which ones you want to connect. Azure Sentinel has an extensive collection of built-in data connectors for both Microsoft and third-party products that can be utilized.

Follow these steps to review the data connector gallery:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel workspace page appears, as shown in Figure 3-12.

   

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears, as shown in Figure 3-13.

   

4. Click Data Connectors, which opens the Data Connectors page, as shown in Figure 3-14.

   

5. Scroll through the gallery and note the data sources that are in your IT environment and that you want to ingest.

6. Add these data sources to your design document.

Although Azure Sentinel has many built-in data connectors, there might be data sources that you need to connect that are not in the gallery. We will cover how you can configure a custom connector to ingest these data sources into Sentinel later in the chapter. Meanwhile, we will continue to focus on identification of data sources. Many organizations have older, on-premises SIEMs that they are migrating away from, and you should review whether the data sources ingested by these SIEMs should be redirected to be ingested by Azure Sentinel.

After reviewing the data connector gallery and the current data sources that are being monitored by an organization’s incumbent SIEM solution, the last stage of the process is to verify whether there are other data sources that need to be connected to Azure Sentinel from the IT environment. When setting up or migrating to a new SIEM, it is worthwhile to verify there aren’t any blind spots in your monitoring setup and that there aren’t any assets that have not been monitored previously. This can occur for various reasons, including:

• Difficulty in integrating the data source

• Volume or noisiness of data source

• Human error/oversight

If a data source was previously unable to be integrated into a SIEM, the data source should be reassessed to see whether it is more viable to connect it to the SIEM to reduce as many blind spots as possible in the environment when moving to a modern solution.

Identify the prerequisites for a data connector

Azure Sentinel makes it easy to understand what prerequisites you need before you can use a data connector: they are all listed on each data connector’s page. Some data connectors require Syslog/CEF connectors or Windows Event collectors to be set up as a prerequisite for use, but we’ll dive into that in more detail later in this chapter.

Follow these steps to view the prerequisites for using a data connector:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Data Connectors, which opens the Data Connectors page.

5. Select the data connector for which you want to view the prerequisites and click the Open Connector page button.

6. The connector’s overview page appears, as shown in Figure 3-15.

   

7. The prerequisites to be able to use the selected data connector can be found in the top-right part of the connector’s page.

Configure and use Azure Sentinel data connectors

Now that you’ve identified which data connectors you want to use and have all the prerequisites in hand, it’s time to start configuring your Azure Sentinel data connectors. As always, Azure Sentinel tries to make this process as easy and painless as possible for you (you’re probably noticing a theme here!).

Following are some examples of the Azure Sentinel data connectors you might connect in this manner:

• Microsoft Cloud App Security (MCAS)

• Azure Defender

• Azure Defender for IoT

• Microsoft Defender for Endpoint

• Microsoft Defender for Identity

• Microsoft Defender for Office 365

• Azure Active Directory Identity Protection

To configure a data connector:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Data Connectors. The Data Connectors page appears.

5. Select the data connector that you want to configure and click the Open Connector page button.

6. The configuration steps to activate the selected data connector can be found on the bottom-right of the connector’s page. Configuration steps vary from data connector to data connector, as shown in Figures 3-16 and 3-17.

   

   

7. Follow the configuration steps detailed to connect your data source to your workspace.

8. After you’ve connected the data source to your workspace, you can use the data connector’s page to check the status of the connector, when the last log was received, and the like, as shown in Figure 3-18.

   

Design and configure Syslog and CEF event collections

Syslog and CEF formats are used by a huge range of systems for logging. You’ll notice that if you review the prerequisites for the built-in Azure Sentinel data connectors that several of them use Syslog or CEF collection to ingest logs.

Before we dive in to how to design a collector for these types of logs, let’s quickly step back and understand what Syslog and CEF are.

CEF:Version|Device Vendor|Device Product|Device Version|Signature
ID|Name|Severity|Extension

Syslog/CEF collector architecture options

The great news is that it doesn’t matter whether you’re collecting Syslog or CEF, the agent used and architectural considerations are exactly the same. A Syslog/CEF collector for Azure Sentinel uses the Linux version of the Log Analytics agent, also known as the OMS agent.

Tip Installation Script

Installation of this agent is very simple, and Microsoft provides an installation script that you can learn more about at https://docs.microsoft.com/en-us/azure/azure-monitor/agents/agent-linux.

With regard to the architecture choices for your Syslog/CEF collector, you have two choices: Deploy on-premises or deploy in the cloud. Both architectures are supported, so ultimately, you will need to decide what works best for your environment. A cloud-based collector offers the elasticity and resiliency of cloud resources but might require more on-premises configuration. For example, if there is a firewall between the on-premises environment and the network connection to the Azure cloud (such as via Express Route, VPN, and so on), then ports would need to be opened for every Syslog/CEF source to be allowed through to reach the cloud. This might not be feasible or acceptable for security teams. In this case, an on-premises collector could better serve the implementation. As with so many things in IT implementations, there are pros and cons to each choice. These architecture options are shown below in Figures 3-19 and 3-20.

Tip Events per Second (EPS)

Remember to size your collectors appropriately depending on the number of events per second (EPS) that you expect to collect from your environment. A single log forwarder machine using the rsyslog daemon has a supported capacity of up to 8500 EPS.

Design and configure Windows Events collections

Windows security events are events logged by devices using the Windows OS (servers and endpoints, physical and virtual) and can be sent to an Azure Sentinel workspace for analysis and for correlation with other events in your environment. Azure Sentinel provides a built-in connector where you can stream Windows security events to your workspace, as shown in Figure 3-21. Logs collected using this data source go into the SecurityEvents table.

As we did with collecting Syslog and CEF events, we will be using the Log Analytics agent to collect Windows security events, but we will be using the Windows version (unsurprisingly!). Unlike Syslog/CEF collection, we won’t be creating a centralized collector. When using the Windows version of the Log Analytics agent, each agent streams directly to the Azure Sentinel workspace. There is no intermediary device. In Figure 3-22, you can see how Windows systems stream security events to a Sentinel workspace.

Choosing Windows security events to stream to an Azure Sentinel workspace

There are thousands of Windows Events, and it can be hard to choose which ones you need to ingest, so Microsoft provides preset options in the connector itself (shown in Figure 3-31):

• All Events

• Common

• Minimal

• None

More Info Windows Security Events Included in Each Preset Option

You can learn more about the Event IDs included in each preset streaming option at https://docs.microsoft.com/en-us/azure/sentinel/connect-windows-security-events.

Configure custom threat intelligence connectors

Using threat intelligence (TI) feeds enriches the information a SIEM collects and thus enhances your security operations. TI feeds contain information about cyberthreats. Most typically, we see these expressed as indicators of compromise (IOCs). An IOC could be a known malicious IP, URL, hash value, and the like. If this IOC is matched to values in the data from your IT environment, it could indicate that an attacker is (or has been) in your environment. This is why IOCs from threat intelligence are often used for proactive hunting. From an incident perspective, if an incident is raised where some entities have matches to your threat intelligence, an SOC might choose to raise the severity of the incident because there is a higher likelihood that known attackers are part of the incident.

IOCs often have an expiration date attached to them. We know that attackers will change how they present their attacks frequently to avoid detection, which is why IOCs need frequent updating via a TI feed. TI feeds can be purchased from a vendor, but there are also open-source and free-to-use TI feeds available in the community.

In Azure Sentinel, when TI feeds are connected, the IOCs from the feed will be stored in the ThreatIntelligenceIndicator table, and you’ll be able to review them in a more user-friendly format on the Threat intelligence page in the Sentinel UI. You can also add IOCs manually on the New Indicator blade, as shown in Figure 3-32.

If you are using a threat intelligence feed, it is likely to be sent from a STIX/TAXII setup:

• STIX (Structured Threat Information eXpression) STIX is a standardized language that has been developed by MITRE in a collaborative way to represent structured information about cyber threats.

• TAXII (Trusted Automated eXchange of Indicator Information) TAXII is a transport vehicle for STIX-structured threat information that allows STIX information to be exchanged between parties.

STIX and TAXII were created to allow easy and consistent sharing of threat information between individual people or organizations worldwide.

Azure Sentinel has a built-in threat Intelligence data connector that can be used to connect to TAXII servers and import IOCs into the ThreatIntelligenceIndicator table.

Follow these steps to configure the Threat Intelligence Connector in Azure Sentinel:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Data Connectors. The Data Connectors page appears.

5. Select the Threat intelligence (TAXII) Data Connector and click the Open Connector button. The Threat Intelligence (TAXII) Data Connector appears, as shown in Figure 3-33.

   

6. Under Configuration, complete the details required to connect your TAXII server to Azure Sentinel. (This is standard information and should be provided by the threat intelligence feed provider.) Click Add.

7. You can check which TAXII servers you have connected to your Sentinel workspace by scrolling down to the bottom of the Threat Intelligence Connector page and checking the List Of Configured TAXII Servers, as shown in Figure 3-34.

   

8. You can view IOCs on the Threat intelligence page in the Sentinel UI, as shown in Figure 3-35.

   

Create custom logs in Azure Log Analytics to store custom data

Sometimes, it won’t be possible to use any of the native methods described in the previous sections to connect your data source…so what then? The main method by which you can ingest custom logs into Azure Sentinel is to use the HTTP Data Collector API, but there are different ways to interact with it—direct via the API or via Azure Logic Apps. Depending on the use case, volume, and type of data you need to ingest, it will likely become obvious which method is better to use.

Custom log ingestion via the Azure Monitor HTTP Data Collector API

Azure Monitor provides the HTTP Data Collector API that can ingest data from a REST API client. Remember, Log Analytics is part of the wider Azure Monitor platform, so don’t be put off by the name! Data must be sent to the HTTP Data Collector API in JSON format, and from there, it will be parsed into a custom table. When you submit the data, an individual record is created in the repository for each record in the request payload, as shown in Figure 3-36.

You have many options when choosing how to interact with this API; typically this would be via an existing REST API client or a serverless function written in Powershell, Python, C#, and so on. There really are no limits to how you interact and send data as long as you stick to the rules and formats required by the API.

Custom log ingestion via Azure Logic Apps

We’ll be covering more about Azure Logic Apps and automation later in this chapter, but in this section we’ll talk about how you can configure a Logic App to ingest custom data into your workspace. If you’re unfamiliar with this product, Logic Apps provides a GUI-based interface to write automation scripts called Playbooks. If you’ve ever used Microsoft Flow, you’ll have a good idea of what you’ll be doing in Azure Logic Apps.

Let’s walk through an example of pulling data from an external API to store in a custom table in Log Analytics. In my example, we’re going to be pulling weather data, but in real life security operations this is more likely to be threat intel or something else that enriches the data in your workspace.

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Automation. The Automation page appears, as shown in Figure 3-37.

   

5. Click Create > Add New Playbook. The Create A Logic App page appears, as shown in Figure 3-38.

6. Fill out the Subscription, Resource Group, Region, and Name of the Playbook and click Review + Create to validate your template.

7. When your template has validated, click Create.

   

8. Open the blank Playbook you have created. You will be given the option to choose from various predefined templates. This time, select Blank Logic App. The Logic Apps Designer page appears.

9. We can now search for a trigger to kick-off the Playbook in the Logic App Connector Gallery. In Figure 3-39, you can see that we are searching for the Schedule trigger.

   

   Note Don’t Confuse Connectors

   Don’t confuse a Sentinel data connector with a Logic App connector. They are two very different things! Logic App connectors are a collection of triggers and actions that you can add to a Playbook to make your automation flow.

10. Select the Schedule connector and choose the Recurrence trigger.

11. Configure how frequently you want this Playbook to run. In my example, I’m going to configure it to run every 5 minutes, as you can see in Figure 3-40.

    

12. For the next step in the Playbook, select HTTP Logic App Connector > HTTP Action. This is where we will configure the call to the external API for the custom data to be ingested.

13. Select GET for Method and add the URI for the API, plus any other necessary parts of the request such as authentication, headers, and so on. You can see the completed one for my weather API in Figure 3-41.

    

14. The final step in the Playbook will be to send this data into Log Analytics. Search for and select the Azure Log Analytics Data Collector and then select the Send Data action.

15. You will be asked to complete the details of the workspace to which you want to send the data (see Figure 3-42). Give the connection a memorable name and provide your workspace ID and key. (Earlier in this chapter, we explained how to obtain these.)

16. Click Create.

    

17. As shown in Figure 3-43, you will need to provide the JSON Request Body details that will be received by the Playbook. This allows the Playbook to parse the data correctly when it is received. You will also need to specify the name of the custom table that the data will be sent to in the Custom Log Name field.

    

18. Click Save at the top left of the Logic App Designer page.

19. Navigate to the Playbook’s Overview page and select Run Trigger > Recurrence to test your Playbook (see Figure 3-44). You will be able to see whether the Playbook ran successfully by looking at the bottom-right of the page.

    

20. Now it’s time to check to see whether our logs were received properly into our custom table: Navigate to the Logs page. As you can see in Figure 3-45, on the Tables tab, we now have an extra drop-down menu, Custom Logs. Whatever name you specified in the Custom Log Name will be appended with _CL. This prevents overlaps with the naming of built-in-in tables in Log Analytics.

    

Design and configure analytics rules

As with the rest of the product, you’ll find that Azure Sentinel has many out-of-the-box analytics rules to start you off in your implementation. If you’ve worked with other SIEMs before, you might know them as “detection rules.” These rules have been written by Microsoft security experts and, while they are fully customizable to your environment, they are a great way to get started with your base analytics rules in your Azure Sentinel implementation. It’s worth reviewing the out-of-the-box templates regularly to check if there are new ones; Microsoft adds more on a regular basis. There are five types of analytics rules in Azure Sentinel:

• Scheduled query These queries run on a fixed schedule (every 5 minutes, every hour, and the like), and you can see the query logic and can make changes to it. We will discuss how to do this later in this section.

• Microsoft security These rules automatically create incidents based on alerts from other Microsoft security products. These rules are a great way to get your Azure Sentinel deployment up and running quickly.

• Fusion Fusion uses scalable machine learning algorithms that can correlate many low-fidelity alerts and events across multiple products into high-fidelity and actionable incidents. Fusion is enabled by default. Because the logic is hidden and therefore not customizable, you can only create one rule with this template.

• Machine learning (ML) behavioral analytics These templates are based on proprietary Microsoft machine learning algorithms, so you cannot see the internal logic of how they work and when they run. Because the logic is hidden and therefore not customizable, you can only create one rule with each template of this type.

• Anomaly These rules use SOC-ML (machine learning) to detect specific types of anomalous behavior. Each rule has its own unique parameters and thresholds appropriate to the behavior being analyzed, and while its configuration can’t be changed or fine-tuned, you can duplicate the rule and change and fine-tune the duplicate.

First, let’s look at the Analytics page in the portal, which can be seen in Figures 3-46 and 3-47.

We recommend that you enable all out-of-the-box templates that use data sources that you ingest into your workspace. It’s very easy to do: You can use the filters to filter rule templates by a specific data source, or you can just look at the rule template summary (see Figure 3-47) and look at Data Sources.

Pay attention to the color of the connector icon next to the name of the data source. If it’s green, then Sentinel has found that type of logs in the workspace; if it’s gray, then you need to add that log type for the rule to work properly. Figure 3-48 shows examples of both green (AzureActivity) and gray (Amazon Web Services) icons, with the green icon shown first.

To activate an analytics rule template:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the desired workspace. The Azure Sentinel | Overview page appears.

4. Click Analytics. The Analytics page appears, as shown in Figure 3-49.

   

5. Select the Rule Templates tab.

6. Select the rule template you want to activate and at the bottom of the rule template summary, click Create Rule.

7. You will be taken to the Analytics Rule Wizard. Work your way through the tabs of the wizard. (These tabs are all prefilled when using a rule template.) After the validation check, click Create.

   Note Customizing Analytics

   All the tabs in the Analytics Rule Wizard are customizable. We will cover this in more detail later in this chapter.

8. You will return to the main Analytics page. Under the Active Rules tab, you should now see your newly created rule, as shown in Figure 3-50.

   

Create custom analytics rules to detect threats

Although the out-of-the-box analytics rules are a great way to start off your implementation, they aren’t optimized for a specific environment and therefore, you might want to customize these rules to make them more specific to your thresholds, operational procedures, and so on. There are many reasons for customizing query logic, but the most common reason is to reduce false positives (such as when a rule triggers an incident but further investigation indicates there wasn’t a security issue). SOCs are always trying to reduce false positives and noise because it wastes SOC analyst time. It is common for SOCs to not have enough analysts to look at alerts as it is, so they certainly don’t want them looking at false positives!

In this section, we will look at how you can customize analytics rules to optimize them. Let’s go back to the Analytics page and the analytics rule templates:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Analytics. The Analytics page appears.

5. Select the Rule Templates tab.

6. Select the rule template you want to use and click Create Rule at the bottom of the rule template summary at the bottom-right of the page. For our example, I’m going to use the Failed AWS Console Logons But Success Logon To AzureAD Rule template. You will be taken to the Analytics Rule Wizard, as shown in Figure 3-51.

   

7. On the General tab, you can customize the Name, Description, and the MITRE Tactics that this rule detects on, as well as the Severity of the rule. In this example, I’m planning to change the failed login attempts in the rule’s logic, so I’m going to update the description of the number of failed logins from 5 to 10.

8. On the Set Rule Logic tab, you will see various aspects of the rule logic can be customized.

   • Rule Query The Kusto Query Language (KQL) of the query. (See the “Define incident creation logic” section later in this chapter for further details on this.) This is fully editable, and in this example, I’m going to change the variable named signin_threshold, which can be seen in Figure 3-52. This threshold is currently set to 5 failed logins, but I’m going to change it to 10. What this means is that until there have been 10 failed logins for a user in AWS followed by a successful Azure login by the same user, this rule will not trigger and create an alert.

   • Alert Enrichment This is where you can define entities that can be classified for further analysis by Azure Sentinel.

   • Query Scheduling In this section, you can configure how frequently your query runs and how far back in the logs the rule will look for matches (known as the lookback window). You can run queries in Azure Sentinel as frequently as every minute.

   • Threshold This allows you to define how many rule “hits” need to occur before an alert is raised. Generally, this can almost always be left on the default setting, Is Greater Than 0.

   • Event Grouping where you can configure how rule query results are grouped into alerts.

   Note Keep Lookback Settings Consistent

   Lookback windows can also be configured in the rule query logic itself. Remember to keep the lookback window consistent between the KQL logic and the rule settings in the portal.

   

9. Let’s move to the Incident Settings tab, as shown in Figure 3-53.

   

10. The Incident Settings tab has many options that can be customized, and how these are configured is largely going to come down to the individual SOC and operational processes in an organization:

    • Incident Settings Here, you can enable/disable whether an incident in Azure Sentinel is triggered by an analytics rule. Often, people will say: “Of course I need an incident triggered from an analytics rule. That’s the whole point of the rules!” However, sometimes an alert is sufficient enough for an SOC to take note, but there is no need for a full-scale incident. A Playbook can be run on an alert, and automation might be able to take care of the actions that need to take place without having any human intervention. (More on that later in this chapter.) The SOC might decide that an incident is triggered only when a collection or threshold of multiple alerts occur. Again, the aim here is to maximize the efficiency of the SOC and to ensure that SOC analysts spend their time on the right events that haven’t been seen before and that require human intervention.

    • Alert Grouping If you’ve worked in security operations (or any other kind of monitoring, for that matter), you’ll likely have seen when a single event raises multiple incidents, overwhelming the analysts and your monitoring panel. Alert grouping can prevent this in an SOC by telling Azure Sentinel to group identical alerts into the same incident in a specified timeframe and thereby reducing noise.

    • Re-open closed matching incidents As the name suggests, this setting will allow Azure Sentinel to re-open a closed incident if an alert matching the alert grouping configured on the rule matches.

11. The Automated Response tab is where either automation rules or Playbooks can be attached to a rule. We discuss this in more detail later in this chapter.

12. After the validation check, click Create. You will return to the main Analytics page. On the Active Rules tab, you should now see your newly created rule.

Activate Microsoft security analytics rules

Microsoft security services perform in-depth analysis of the logs they process and generate high-fidelity alerts. The services in this suite are:

• Microsoft Cloud App Security (MCAS)

• Azure Defender

• Azure Defender for IoT

• Microsoft Defender for Endpoint

• Microsoft Defender for Identity

• Microsoft Defender for Office 365

• Azure Active Directory Identity Protection

As we learned earlier in this section, these products’ alerts can be connected to Azure Sentinel using built-in data connectors. Instead of performing further analysis on these alerts, as they have already had a significant amount of analysis done in the service, you might want to create an Azure Sentinel incident right away. This can be achieved quickly and simply by using Microsoft security analytics rules.

Let’s learn how to activate these rules:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel Overview page appears.

4. Click Analytics. The Analytics page appears.

5. Click Create and select Microsoft Incident Creation Rule, as displayed in Figure 3-54.

   

6. The Microsoft Incident Creation Rule Wizard appears, as shown in Figure 3-55.

   

7. The wizard has several fields that need completing, but you will notice that it has far fewer configurables than a scheduled query rule:

   • Name The name of the rule.

   • Description The description of the rule.

   • Status Set to Enabled or Disabled.

   • Microsoft Security Service This is where you select the type of Microsoft security service alerts that the rule will listen for. You must have one rule per Microsoft security service; they cannot be combined into one.

   • Filter By Severity You can choose to only create incidents for alerts of a certain severity. For example, you might only choose to create incidents for high- and medium-severity alerts using the rule.

   • Include/Exclude Specific Alerts This is where you can explicitly include or exclude certain alerts. This can be useful if a security service generates a noisy alert that does not require an incident to be raised.

   Tip Include/Exclude Specific Alerts Feature

   Be careful when using the Include/Exclude Specific Alerts feature. While this is an effective way to reduce noise coming from your environment, it means that every instance of the alert specified will not raise an incident, so it is important to be sure that by using this feature, you won’t inadvertently miss a real incident.

8. The Automated Response tab is where either automation rules or Playbooks can be attached to a rule. We will cover this in more detail later in this chapter.

9. After the validation check, click Create.

10. You will return to the main Analytics page. On the Active Rules tab, you should now see your newly created rule.

Configure connector-provided scheduled queries

We do often seem to start a section with a statement like this, but once again, I’ll be referring to Azure Sentinel’s out-of-the-box capabilities and how Microsoft makes it straightforward to configure relevant analytics rules. Earlier in the chapter, we already looked at the data source connector page and analytics rule templates, and in this section, we’ll again be looking at these parts of Azure Sentinel.

If you open a Data Connector page, you can see the instructions that tell you how to connect that data source to the workspace. You might have noticed the Next Steps tab, which contains links to workbook templates, query samples, and Relevant Analytics Templates (see Figure 3-56).

Here, you can click Create Rule, and you will be taken to the Analytics Rule Wizard—Create New Rule From Template page. From there, you can create and customize this rule. (We walked through using this wizard earlier in this chapter in “Create custom analytics rules to detect threats.”)

Although this isn’t showcasing functionality that can’t be found elsewhere in Azure Sentinel—especially on the Analytics page—it is strongly recommended that you activate all rule templates that use the data sources you are choosing to connect to your workspace, so having another method to verify this has been done correctly is never a bad thing!

Configure custom scheduled queries

Earlier in this skill, we discussed how to customize analytics rule templates. Now let’s discuss how to create a custom scheduled query from scratch. If you’ve been following along using Azure Sentinel and testing out the steps earlier in this section, you’ve probably got a good idea what’s coming in this section. We’ll be using the Analytics Rule Wizard on the Analytics page to create our brand-new rule. This time—rather than deploying or amending an existing rule template—we will be completing the entire rule ourselves.

So why do we need these rules? Aren’t the rule templates enough to cover most likely security events? Although the analytics rule templates are written by experts and cover a wide range of scenarios, it is likely that a large, complex IT environment will need to have custom analytics rules for detections that are very specific to that environment or that are for data sources that don’t have any rule templates.

Let’s step through creating an analytics rule from scratch:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel Overview page appears.

4. Click Analytics. The Analytics page appears.

5. Click Create > Scheduled Query Rule. The Analytics Rule Wizard—Create page New Rule page appears, as shown in Figure 3-57.

   

6. The wizard is the same as the Analytics Rule Wizard used for templates, except this time, it is completely blank for you to fill in as you please.

7. We covered the different fields and how to complete them in the wizard in the “Create custom analytics rules to detect threats” section, earlier in this chapter.

8. After the validation check, click Create.

9. You will return to the main Analytics page. On the Active Rules tab, you should now see your newly created rule.

Define incident creation logic

For the final part of this section, we’re going to address how to create incident creation logic, which is something we’ve skipped over to leave for the grand finale of this section. It’s important to remember that although the exam outline (and hence, the title of this section) calls for learning how to define incident creation logic, what we’re going to discuss in this section is the use of the KQL query language, which is used to define queries, events, alerts, and incidents.

Kusto Query Language (KQL)

Kusto Query Language (KQL) is a query language used in several Microsoft products, so if you’re learning about Microsoft security services, then it’s worthwhile investing some time to become familiar with the language. A not-so-well-known fact is that Kusto Query Language is named after Jacques Cousteau, the famous underwater explorer. KQL is a high-level language (meaning that it is closer to human language), but it has the flexibility and power to perform complex queries. Although not identical, if you’ve spent time using SQL, then learning KQL should be a straightforward task.

KQL is a complex language with many operators, so in this section, we will cover the basics of KQL and some of the operators commonly used to form incident creation logic.

Let’s start with the basics. In KQL, you must define the table that you are searching in. In this example, I’m searching the OfficeActivity table (shown in Table 3-1) for Exchange workloads. The where operator is used to filter the table to a subset of rows. In this case, that means any rows that have Exchange in the OfficeWorkload column.

Click here to view code image

OfficeActivity
| where OfficeWorkload == "Exchange"

A very commonly used operator (and one of my personal favorites) is take. This operator returns the number of rows you specify:

OfficeActivity
| take 10

In this case, the query would return 10 rows from the OfficeActivity table. This operator is especially useful for testing queries when you’re not sure how many results it might bring back. Also, it’s helpful for looking at the structure and a few examples of rows from the specified table.

Sort is a powerful operator that does exactly as the name suggests: It sorts rows of the table by one or more specified columns in ascending or descending order:

Click here to view code image

OfficeActivity
| sort by OfficeWorkload, UserId asc

In this example, we’re sorting the OfficeActivity table (see Table 3-1) in ascending order by the OfficeWorkload and UserId columns. A close relative of this operator is top, which will return the top values after the sort (for example, the top 10 or top 100). This can be very useful for making queries more efficient if you don’t require the entire contents of the table to be sorted, which is often the case in security queries. Often, searches will be looking for things such as the top number of failed logins in an environment and the users associated with them.

Let’s break down one of the query templates to better understand how KQL works in practice. In Figure 3-58, the Failed AWS Console Logons But Success Logon To AzureAD rule is shown.

The KQL in the rule is shown in Listing 3-1.

Listing 3-1 Failed AWS Console Logons But Success Login to AzureAD

Click here to view code image

//Adjust this threshold to fit environment
let signin_threshold = 5;
//Make a list of IPs with failed AWS console logins
let aws_fails = AWSCloudTrail
| where EventName == "ConsoleLogin"
| extend LoginResult = tostring(parse_json(ResponseElements).ConsoleLogin)
| where LoginResult != "Success"
| where SourceIpAddress != "127:0:0:1"
| summarize count() by SourceIpAddress
| where count_ > signin_threshold
| summarize make_list(SourceIpAddress);
//See if any of those IPs have sucessfully logged into Azure AD.
SigninLogs
| where ResultType !in ("0", "50125", "50140")
| where IPAddress in (aws_fails)
| extend Reason = "Multiple failed AWS Console logins from IP address"
| extend timestamp = TimeGenerated, AccountCustomEntity = UserPrincipalName,
IPCustomEntity = IPAddress

Let’s break this rule down step-by-step:

1. The signin threshold variable is declared as 5 and is named signin_threshold.

2. A second variable is declared as aws_fails, but this variable is a list of IP addresses, so there are more lines of KQL to filter out these IP addresses.

3. Note that comments can be added to the query and prepended with //, and they will be ignored by the query parser.

4. To find a list of IP addresses from AWS Cloudtrail, the query first searches for ConsoleLogin events.

5. A new column is created using the extend operator called LoginResult. As part of the creation of this column, the query is using the parse_json operator is used to parse the embedded JSON in the ResponseElements column, so the query can search for records that do not contain Success.

6. Local logins are removed (127.0.0.1).

7. The number of unsuccessful logins are summarized by the SourceIpAddress column.

8. A list is created of any IP addresses that have been counted by the query as having more than the threshold (5) unsuccessful login attempts.

9. This list of IP addresses is now our aws_fails variable.

10. Moving to the second part of the query, it searches Azure AD logs to see if there have been any successful logins to Azure from the list of IP addresses we made in the first part of the query.

11. Finally, any matching results will be presented in a user-friendly manner and mapped to entities using the extend operator and custom entities.

Create Azure Sentinel Playbooks

If you’ve used other Azure products, you might already be familiar with Azure Logic Apps, which is the main “engine” that drives automation in Azure Sentinel. Azure Logic Apps is a GUI-based tool that can create complicated automation Playbooks with little-to-no programming and coding knowledge required. This is great for SOC analysts and SOC engineers who might have little previous knowledge of how to make automation scripts. If you’ve used Microsoft Flow before, you’ll also have a good idea of what to expect when it comes to creating automation in Azure Sentinel.

Before we go any further, let’s have a terminology check to clarify our understanding:

• Azure Logic Apps This is the name of the Azure service that provides automation throughout the Azure cloud, including for Azure Sentinel. Because Azure Logic Apps is a separate service to Azure Sentinel, it requires separate permissions for a user to create and run Playbooks (which are discussed earlier in the chapter).

• Playbook A Playbook is a collection of automated actions in a workflow.

• Logic App connector This is not the same as an Azure Sentinel data connector. Instead, a Logic App Connector is a predefined trigger or action that can be added into a Playbook. At the time this book was written, there were more than 300 Logic App connectors.

There are three main scenarios for which automation is used in Azure Sentinel:

• Alerting This is the most used type of automation and the most straightforward to configure. When an incident or alert is triggered, Playbooks can be configured to send emails, Teams messages, and the like to alert the on-call team that an incident has been raised.

• Remediation This is where automation takes remedial action in the IT environment to contain or even stop a security incident. Examples here could be isolating a virtual machine that has an Azure Defender alert raised against it, blocking the account of a user in Azure AD when their activity indicates the account has been compromised, or taking a malicious IP address from an incident in Azure Sentinel and writing a block rule back to the firewall to stop traffic from that IP address.

  Tip Alerting and Remediation

  Although they can be configured independently of each other, alerting and remediation can—and usually should be—contained in one Playbook.

• Enrichment This is where supplementary data is used to “enrich” raw logs and results. This can be done as an alert, when an incident is triggered, or during an investigation. For example, after an incident has been raised, a SOC analyst could run an enrichment Playbook to check if the IP address entities in the incident match third-party threat intel feeds (for example, VirusTotal).

Let’s look at how to create a simple email alert Playbook in Azure Sentinel:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Automation. The Automation page appears, as shown in Figure 3-59.

   

5. Click Create > Add New Playbook. The Create A Logic App page appears, as shown in Figure 3-60.

   

6. Fill out the Subscription, Resource Group, Region, and Name of the Playbook and click Review + Create, as shown previously in Figure 3-60.

7. When your template has validated, click Create.

8. Open the blank Playbook you have created; you will be given the option to choose from various predefined templates. This time, we will select Blank Logic App. The Logic Apps Designer page appears.

9. We can now search for a trigger to kick-off the Playbook in the Logic App Connector gallery. In Figure 3-61, you can see that we are searching for an Azure Sentinel trigger.

   

10. Select the When A Response To An Azure Sentinel Alert Is Triggered trigger.

11. Sign in to create a connection to your Azure Sentinel workspace from the Playbook, as shown in Figure 3-62. You can also use a service principal or managed identity if you would prefer.

    

12. For the next step in the Playbook, I’m going to select the Outlook.com connector and under Actions, I will choose Send An Email (V2). This is where we will configure the email to be sent when an alert is raised in Azure Sentinel (see Figure 3-63).

    

    Note Email Connectors

    In this example, I have chosen Outlook.com because this is where the email account that I am using resides. As is the case with other types of Logic App connectors, there are multiple email connectors, such as Outlook 365 and Gmail. Choose the right connector for your needs!

    Sign in to the Outlook account; the connector will allow you to configure the email that will be sent when the Playbook is triggered. Add the address(es) to which you want the email to be sent, the title of the email, and the body of the email. There are other optional parameters that you can choose to configure at this point. It is possible to add dynamic content from the alert, such as the name of the alert, the severity, a description, and so on that will be populated dynamically from the alert when the Playbook is triggered. This helps give SOC analysts a better idea of what is happening in the initial notification, as shown in Figure 3-64.

    

13. Click Save at the top-left of the Logic App Designer page.

14. Navigate to the Overview page of the Playbook.

15. Select Run Trigger > Run to test-run your Playbook, as shown in Figure 3-65. Look to the bottom-right of the page to see whether the Playbook ran successfully.

    

Arguably, the most attractive aspect of a SOAR capability is the automation aspect. Rather than relying on a human resource to take actions, automation can do things quicker and more reliably. We already touched on this during the analytics rule section of this chapter, but let’s dive into how to configure analytics rules and incidents to trigger Playbooks.

Why do we want to attach Playbooks to an analytics rule? Quite simply, this is the quickest and easiest way to alert and/or respond to a threat that Azure Sentinel detects. By attaching a Playbook to an analytics rule, when that rule is triggered, the Playbook will automatically run and take the specified actions. Even before an SOC analyst goes to look at the incident or alert, the Playbook could have taken remedial action. Some SOCs who have a mature automation capability can use automation to entirely resolve incidents without human intervention. Microsoft’s own SOC tries to do this where possible. Of course, there will be incidents or attacks that have never been seen before and cannot be automated away, but this means that SOC analysts can be more efficient in the time they spend on investigations rather than repetitive tasks.

In this section, we will look at how you can customize analytics rules to optimize them. Let’s go back to the Analytics page and the analytics rule templates:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Analytics. The Analytics page appears.

5. Select the rule you want to attach a Playbook to and click Edit at the bottom of the rule template summary .

6. You will be taken to the Analytics Rule Wizard. Select the Automated Response tab.

7. Under Alert Automation, you can select Playbooks that are configured to run when the rule is triggered (see Figure 3-66).

   

8. On the Review And Create tab, click Save after the validation check.

Use Playbooks to remediate threats

As we discussed earlier in this section, one of the most powerful aspects of a SOAR capability is the ability to remediate threats automatically. Even with the best and smartest SOC analysts in the world, they will never be able to respond as quickly or efficiently to an incident as automation can, which means automation can reduce the potential impact of an incident. Using Playbooks to remediate threats also frees up SOC analysts’ time to concentrate on incidents that haven’t been seen before, cannot be remediated automatically, and therefore need human intervention.

Although this is not an exhaustive list, here are some examples of the kind of security incident remediation that are possible using Azure Logic Apps:

• Blocking a user account in Azure AD to prevent an account that might have been compromised from being able to access resources in the IT environment

• Writing a rule to a firewall to block an IP address from which malicious traffic is being sent

• Isolating a virtual machine in Azure that might have been compromised to prevent further lateral movement

• Taking a snapshot of an Azure virtual machine for evidence gathering purposes when the machine exhibits suspicious activity

• Blocking a malicious IP address on an on-premises Exchange server to prevent further access to the server from that specific address if suspicious traffic and/or activity has been seen originating from there

It can be challenging to decide what remediation action might be appropriate—especially if automated remediation is something that hasn’t been part of your security operations previously—so it’s important to take a step back and work out your use cases. Consider your most-raised security incidents and any repetitive tasks that SOC analysts undertake to resolve those incidents. Could these be automated? Are there some actions that you’d always like to take no matter what the incident is? What automation an organization chooses to take can vary widely. For example, a highly risk averse organization might choose to immediately block any Azure AD user accounts that exhibit suspicious activity, but this will put an additional burden on the service desk, which will have to re-enable accounts, posibly reducing productivity. There is also the risk of an account being blocked as a false positive because no SIEM can get things right 100 percent of the time.

Use Playbooks to manage incidents

In this chapter, it’s already been mentioned several times that even the best and most efficient SOC analysts don’t always have enough time to fully investigate every alert and incident that comes into an SOC. SOC analysts commonly undertake repetitive, time-consuming tasks to close incidents, change statuses, assign out incidents to on-shift analysts, and so on.

Aside from remediation, Azure Sentinel’s SOAR capabilities can help to reduce the administrative overhead of dealing with incidents. In this section, we’ll detail some examples—not an exhaustive list—of how overhead on incident management can be reduced using automation:

• Automatically assigning incidents to on-shift analysts Using the shifts feature in Teams, a Playbook can be used to decide which analyst is on-shift and available and automatically assign that ticket to that analyst. The Playbook can also send a notification to that analyst to let them know an incident has been assigned to them using a messaging system, email, and the like.

• Automatically assigning incidents to an entity owner Similar to the previous example, a Playbook can be used to look up the owner of an entity (for example, a host) that is involved in an incident and assign the incident for them to investigate. As in the previous example, the Playbook can also send a notification to the asset owner to let them know an incident has been assigned to them using a messaging system, email, and so on.

• Creating a ticket in a third-party ticketing system Many organizations utilize SaaS ticketing systems to manage their IT operations and can draw statistics and reporting from that system about the number of incidents, time to resolve, and so on. Although the incident system in Azure Sentinel is robust, it might make more sense for incidents to be managed in a third-party system (such as ServiceNow) to align with the wider organization’s IT operations and reporting. A Playbook can be used to create a ticket in a third-party system when an incident is triggered and populate all the details of the incident in that third-party system. This saves SOC analysts’ time and effort copying and pasting the details across from one system to another.

• Syncing third-party incident system updates with Azure Sentinel incidents Related to the previous example, if a third-party system is being used to track and manage incidents, a Playbook can be used to sync the information between Azure Sentinel and the third-party ticketing system, so there is no time-consuming copying and pasting between two systems (Copying and pasting between systems is not a good use of SOC analyst time!)

• Enrich incident details from third-party systems When an incident is raised, a Playbook can be used to enrich the details of the entities involved in an incident and post a comment to give the analyst additional context. This saves the SOC analyst time and effort to log in to another portal or system to look up these details manually. Typical examples of enrichment include looking up the asset owner of a host or checking if an IP address matches any threat intelligence feeds (such as VirusTotal).

• Add a user/host/IP address to a Watchlist A Playbook can take entities from an incident and add them into a watchlist so that they can be flagged in other analytics rules that refer to the watchlist. This saves an SOC analyst from having to do this manually.

Use Playbooks across Microsoft Defender solutions

If you’re using other Microsoft Defender solutions, using Playbooks is an ideal way to create a conjoined response to security events and incidents across your Microsoft product suite. As one would expect, Azure Logic Apps provides many ways to do this with Logic App connectors. As a reminder, when we’re discussing Microsoft Defender solutions, we’re talking about:

• Azure Defender

• Azure Defender for IoT

• Microsoft Defender for Endpoint

• Microsoft Defender for Identity

• Microsoft Defender for Office 365

As you can see in Figure 3-67, when you search for defender in the Logic Apps designer, there are already many built-in triggers and actions that you can select from.

Let’s look through an example to demonstrate how simple this can be. Rather than starting from scratch, as we have done earlier in this section, I’m going to use a template from the Azure Sentinel GitHub repo:

1. Navigate to the Azure Sentinel GitHub repo page by opening https://github.com/Azure/Azure-Sentinel.

2. Click playbooks to be taken to the Playbooks section of the repo.

3. Select the Playbook that you want to deploy. For this example, I’ll be selecting Isolate-MDATPMachine, as seen in Figure 3-68.

   

4. Click Deploy To Azure, and you will be taken to the Azure portal and the Custom Deployment Template.

   Note ARM Templates

   Playbook templates from the Azure Sentinel GitHub repo are all Azure Resource Manager (ARM) templates. You can read more about ARM templates at https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/.

5. As shown in Figure 3-69, you need to complete the parameters of the Playbook template so that it can be deployed. The exact parameters each template will ask for will depend on the Playbook contents, but you will always be asked for Subscription, Resource Group, Region, and Playbook Name.

   

6. Let the validation check complete and then click Create.

7. Wait for your template to deploy successfully. Once complete, you will see the message shown in Figure 3-70.

   

8. Navigate to Azure Sentinel in the Azure portal and open the Automation page. You should find that your newly deployed Playbook should now be listed under playbooks.

   Note Authorizing the Connection

   Playbook templates will deploy connections, but sometimes you will need to authorize the connection after deployment before the connection is made. This is expected behavior and does not mean the Playbook template is broken.

9. Open the Playbook and open the Logic App Designer.

10. Expand the steps of the Playbook, and you will see the triggers and actions have already been prepopulated and configured by the ARM template that we deployed, as shown in Figure 3-71.

    

11. This Playbook is ready to go and can now be attached to an analytics rule, as we discussed earlier in this section.

Let’s walk through what this Playbook is doing:

1. When an incident creation rule is triggered, the Playbook will run.

2. Details about the host entities from the Azure Sentinel incident that has been raised are sent to Microsoft Defender for Endpoint.

3. Subsequently, those machines are isolated.

Far quicker than any human, Azure Sentinel and Microsoft Defender for Endpoint have worked together to isolate the hosts that are part of a security incident to reduce the “blast radius” or impact of an incident and—if these machines have been breached by an attacker—are stopping the attackers from being able to cause any more damage or from moving laterally in your IT environment.

Investigate incidents in Azure Sentinel

Investigating incidents is critical for an SOC analyst to understand the severity and scope of a potential security issue. The investigation graph in Azure Sentinel allows an SOC analyst to quickly and efficiently investigate and query alerts and entities in an incident, and Azure Sentinel assists by suggesting additional context-aware queries to be run.

First, let’s look at the Incidents page in Azure Sentinel that is shown in Figure 3-72.

The Incidents page that lists all the open incidents in Azure Sentinel is self-explanatory and contains details of each open incident such as the title, severity, created time, and the owner of the incident. An incident is a collection of alerts.

So we’ve selected our incident that we want to investigate; let’s dig into how we do this:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Incidents. The Incidents overview page appears.

5. Select the incident you want to investigate and click View Full Details. The Incident page appears.

6. Click Investigate. You will be taken to the investigation graph. What you see on this page depends on the incident itself, but the structure of the interface is shown in Figure 3-73.

   

7. Figure 3-74 shows that alerts contained in the incident are denoted by a large circle containing an exclamation point. If you click the circle, a summary of the alert is shown on the left side of the investigation graph.

   

8. Figure 3-75 shows the entities that are related to the alert in question. They are linked in the investigation graph with lines. If you hover over an entity, Azure Sentinel will suggest further context-aware further queries that an SOC analyst might want to run to understand and investigate the incident further. In Figure 3-75, Azure Sentinel is suggesting that a number of different queries could be run against the account in question; let’s look at the Hosts The Account Failed To Log In To The Most query. If a user account has been compromised, understanding which hosts it has tried and failed to log in to can help an SOC analyst understand what other devices might be compromised and what kind of resources an attacker is looking to access in an organization’s environment.

   

9. Clicking the suggested query will display the results of that query in the investigation graph, too. Figure 3-76 shows that after running the Hosts The Account Failed To Log In To The Most query, additional entities are added to the investigation graph, and their relationship to the rest of the incident is displayed.

   

10. These additional entities can also be queried further to dig even deeper into what is happening in an incident. This can help an SOC analyst get a full picture of what is happening. Theoretically, this querying of entities could go on ad infinitum, but an SOC analyst usually will need to dig only a few “layers” deep to perform an effective investigation.

11. Let’s review the additional investigation panels that can also be used for investigation:

    • Timeline The Timeline panel will order all alerts in date and time order so that an SOC analyst can understand the order in which events in the incident happened (see Figure 3-77).

      

    • Info The Info panel provides more in-depth information about the selected alert or entity.

    • Entities The Entities panel provides a summary of the entities being displayed in the investigation.

    • Insights The Insights panel (see Figure 3-78) shows other insights about an entity that Azure Sentinel’s built-in UEBA engine thinks is relevant and useful for an SOC analyst to know.

      

Triage incidents in Azure Sentinel

Most of us are familiar with the concept of triage from a medical perspective, but how does that work in security operations? This is the dictionary definition of triage:

“The process of examining problems in order to decide which ones are the most serious and must be dealt with first.”

Therefore, it’s easy to see why triaging incidents is a critical activity for an SOC to decide how to prioritize which incidents to work on first.

How can we triage incidents in Azure Sentinel? There are several ways that this can be done, and a real-life SOC analyst would use a combination of these methods to maximize their triage effectiveness. It’s important to remember that triage won’t look the same for every organization: What one organization considers to be a high-severity incident might be another’s medium- or low-severity incident. SOC managers and analysts must work closely with their business stakeholders and technology risk professionals to align severity ratings with their security posture and risk tolerance.

When an incident is raised in Azure Sentinel, it can be triaged in several areas. Following is a chronological look at the lifecycle of an incident:

• In an analytics rule As discussed earlier in this chapter, when you configure an analytics rule, you must set the initial severity. This initial severity rating will help an SOC analyst decide how to triage the incident.

• In a watchlist If entities in the incident match a watchlist (for example, VIP users), an analytics rule can be configured to make the incident higher or lower severity.

• How many alerts are part of the incident As shown earlier in this chapter, it’s possible to configure an analytics rule to collect alerts in a set time period to be collected into one incident. An incident that involves many alerts is likely to be set at a higher priority than one with fewer alerts.

• Using TI matching Threat intelligence can be used to check if there are IOC (indicators of compromise) matches with any TI feeds that Azure Sentinel is ingesting. For example, if an IP address is raised in an incident, an analyst could run a Playbook to determine if this was a known-malicious IP and reference a third-party service such as Virus Total. If the IP was found to be a known-malicious IP, then the severity could be increased, and it could be prioritized.

• Using enrichment from third-party sources Similar to the previous point, enrichment doesn’t just have to be about TI. Enrichment could be an inventory management system that stores the criticality of an asset. This means an incident raised against a single user desktop might only be classified as being a low-priority incident. However, an incident raised against an e-commerce company’s web servers could be critical to investigate and resolve, and thus, it might be considered to be a high-priority incident. Once again, this could be automated in either a Playbook that is run as the incident is triggered, or it could be a manually run Playbook during the course of an investigation.

Respond to incidents in Azure Sentinel

Now we’ve investigated and triaged an incident, we need to respond to resolve it. Azure Sentinel makes this easy with automation. In this section, we will look at how you can run Playbooks to resolve incidents, so we’re not talking about how we can trigger automation to happen when an incident is raised; instead, we’re talking about the next step.

Let’s work through an example incident:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Incidents, which opens the Incidents overview page.

5. Select the incident you want to respond to and click View Full Details.

6. A list of all alerts that are part of an incident are shown on the Timeline tab (see Figure 3-79).

   

7. Click View Playbooks, and you will be taken to the Alert Playbooks page where all Playbooks that have been configured in the workspace will be listed. By clicking Run, you can trigger one or more Playbooks to run in the context of the selected alert (see Figure 3-80).

   

8. The SOC analyst who has been assigned the incident can then update the case notes as appropriate on the Comments tab of the incident’s page (see Figure 3-81).

   

Investigate multi-workspace incidents

When preparing for the SC-200 exam, it’s important to remember that there are two types of multi-workspace scenarios that you might be asked about in relation to Azure Sentinel:

• Cross-tenant scenario Where multiple Azure tenancies each have Azure Sentinel workspaces that need to be centrally managed

• Cross-workspace scenario Where there are multiple workspaces in one Azure tenancy that need to be centrally managed

At the beginning of this chapter, we discussed the considerations when designing Azure Sentinel workspaces and why some organizations might require more than one workspace in their deployment. Some customers might also choose to outsource the running of their SOC to a Managed Security Service Provider (MSSP) that will handle security operations on their behalf.

Aside from Azure Lighthouse, there are several Log Analytics and Azure Sentinel features that allow you to investigate incidents across workspaces in the same Azure tenant.

• Cross-workspace analytics rules Analytics rules can be configured to search other workspaces when they are correlating logs. To enable this capability, we have to use the workspace and union KQL operators. Using the queries we used in the “Define incident creation logic”section earlier in this chapter, let’s see how we would need to update them to make them cross-workspace. This is the original query:

  Click here to view code image

  OfficeActivity
  | where OfficeWorkload == "Exchange"

  To make convert this to a cross-workspace query, it would become:

  Click here to view code image

  union
  workspace('<workspaceA>'.OfficeActivity
  | union workspace('<workspaceB>').OfficeActivity
  | where OfficeWorkload == "Exchange"

• Cross-workspace hunting queries Like cross-workspace analytics rules, the workspace and union KQL operators can be used in hunting queries to proactively detect threats and anomalies across multiple workspaces in an environment.

• Cross-workspace workbooks The workspace and union KQL operators can also be used to display consolidated statistics and visualisations from different workspaces. This is particularly useful for centralized reporting for an SOC that uses more than one workspace.

Identify advanced threats with user and entity behavior analytics (UEBA)

Traditionally, user and entity behavior analytics (UEBA) was not incorporated into a SIEM solution. Instead, you would have to buy a third-party product or add-on to be able to get these insights. Having the power of UEBA built into Azure Sentinel—and have it as part of the same interface—allows SOC analysts to focus on a particular entity as part of their investigation. Also, Azure Sentinel’s UEBA can provide insights about an entity’s anomalous activities and behaviors. These insights are based on that entity’s previous behavior, and those behaviors are with the behavior of its peers (if a user) or similar endpoints (if a host). Figure 3-82 takes a look at the architecture of the UEBA feature.

As you can see in Figure 3-82, Azure Sentinel’s UEBA engine takes in raw data sources—from both the cloud and on-premises—that has already been ingested into the workspace, and it then takes users and groups information from Azure AD, enriches it all, and then populates the dedicated UEBA tables in Log Analytics. You will find these UEBA tables in your workspace (listed under Azure Sentinel UEBA) after you have enabled this feature.

By default, UEBA is not enabled in Azure Sentinel, so let’s first look at how to enable it:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Entity Behavior. The Entity Behavior page appears, as shown in Figure 3-83.

5. Click Configure UEBA.

   

6. You will be taken to the Entity Behavior Analytics Settings page. Click Configure UEBA one more time, which will open the Entity Behavior Configuration page, as shown in Figure 3-84.

   

7. Select the data sources that you want to enable for UEBA. You can only enable data sources for UEBA that are already being ingested into your Azure Sentinel workspace.

8. Click Apply. You have now enabled UEBA for the data sources you selected in your Azure Sentinel workspace.

   Note Additional Ingestion Charges

   Be mindful that turning on UEBA will generate additional ingestion charges for your Azure Sentinel workspace because new UEBA tables are created, and data is stored in them for the feature to work.

9. Now let’s explore the entity pages that use the UEBA feature: Return to the Entity Behavior page, as shown in Figure 3-85.

   

   Tip Entity Behavior Page Population

   In real life, after you have turned on UEBA, it can take an up to an hour for the Entity Behavior page to start being populated.

10. The top accounts and hosts by number of alerts will be shown. You can click them to be taken to that entity’s page, as shown in Figure 3-86.

    

Both entity and host pages are similar, follow a theme that is familiar to other pages in Azure Sentinel, and have a focus on timeline and insights: The page will show all the alerts related to that entity in a timeline fashion. Insights about that entity are found on the left side of the page. Insights are based on the following data sources:

• Syslog (Linux)

• SecurityEvent (Windows)

• AuditLogs (Azure AD)

• SigninLogs (Azure AD)

• OfficeActivity (Office 365)

• BehaviorAnalytics (Azure Sentinel UEBA)

• Heartbeat (Azure Monitor Agent)

• CommonSecurityLog (Azure Sentinel)

Entity pages and UEBA functionality are designed to fit into a larger incident investigation and management piece when an SOC is using Azure Sentinel, and they highlight anomalous behaviors and help with triaging.

Activate and customize Azure Sentinel workbook templates

As s you spend more time getting familiar with Azure Sentinel, Microsoft has provided out-of-the-box templates for workbooks that you can add to your workspace and customize them with minimal time spent on overhead. Workbook templates typically exist for any built-in data source for which you can find a connector in the data connectors gallery. Remember, there are workbook templates that aren’t directly related to a single specific data source, so make sure that you look through the workbook template gallery carefully and activate any workbooks that are relevant to your environment.

Let’s look at how to activate a workbook template in your Azure Sentinel workspace:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Workbooks. The Workbooks gallery appears, as shown in Figure 3-87.

   

5. Select the workbook you want to activate in your Azure Sentinel workspace, and the preview bar appears on the left side of the screen, as shown in Figure 3-88.

   

6. Click View Template. This will show you a preview of the workbook using the data in your workspace.

   Note Required Data Types

   A workbook template displays Required Data Types and will indicate if the logs required for the workbook to function properly are available in your workspace. Be prepared for a few errors if you activate the template and don’t have all the logs that the workbook uses!

7. Return to the workbooks gallery, and this time, select the workbook you want to activate and click Save.

8. You will be prompted select what location you want to save the workbook to, as shown in Figure 3-89.

   

9. Choose your desired location and click OK.

10. You will notice that the button options for that workbook have changed, and you now have the View Saved Workbook option, as shown in Figure 3-90.

    

11. Click View Saved Workbook.

12. You will be taken to your saved workbook template, which is populated with the data from your workspace, as shown in Figure 3-91.

    

13. If required, you can now use the Edit button and customize this template.

Create custom workbooks

Although workbook templates cover many use cases and eventualities in Azure Sentinel, in a large, complex organization, it is likely that you might need to create a workbook for reporting specific items for your organization from scratch.

Let’s look at how to create a useful, custom workbook:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Workbooks. The Workbooks Gallery appears.

5. Click Add Workbook. The New Workbook page appears, as shown in Figure 3-92.

   

6. Click Edit, and you will now be able to add items to your custom workbook, as shown in Figure 3-93.

   

7. Let’s briefly explain the items in the Add column:

   • Add Text As this self-explanatory option implies, text can be added to your workbook to explain the purpose of the workbook or any additional explanation of the visualizations being shown. Text in workbooks is formatted as markdown.

   • Add Parameters This is where parameters that can be iterated throughout the workbook are defined as shown in Figure 3-94.

     

   • Add Links/Tabs This self-explanatory option allows you to add relevant links and tabs to your workbook.

   • Add Query Arguably, the most important aspect of customizing workbooks, this is where you can add KQL queries that will bring back data to the workbook to be displayed. If you’ve learned KQL for searching logs and writing analytics rules, you’ll have a good idea what is required here from a query language perspective. There are several different visualizations that you can choose to use, which can be set in the user interface or in the query itself using the render KQL operator shown in Figure 3-95.

     Exam Tip

     Study some of the workbook templates and how they structure their KQL queries to help you understand how to construct KQL queries for visualizations in workbooks.

     

   • Add Group This is where workbook items can be grouped logically to make management of the workbook easier.

8. When you’ve finished creating your custom workbook, click Save.

9. Choose where you want to save your custom workbook to, as shown in Figure 3-96.

   

Configure advanced visualizations

There are many visualization options to choose from in Azure Sentinel when creating workbooks. Choosing the “correct” visualization for the data being displayed will be—to a large extent—a personal preference for those individuals creating the workbook and the organization’s preferences for how the data is to be displayed.

Having said this, certain query results don’t display well (or at all) in certain types of visualizations. In preparation for your SC-200 exam, make sure you are familiar with what does and doesn’t work in terms of the different visualization options and query results in Azure Sentinel. For example, queries that use the bin operator to summarize query results usually display best with a line or bar chart.

Let’s look at some of the visualization options available for you to use in workbooks:

• Charts Available chart types include line, bar, pie, and time. You can customize the chart’s height, width, color palette, legend, titles, and so on. Also, you can customize axis types and series colors using the chart settings. Figure 3-98 shows an example pie chart in a workbook:

  

• Grids Grids display the results of a query not unlike the results seen in the Log Analytics query interface, so using KQL you can choose the columns that appear in the grid. Figure 3-99 shows an example grid in a workbook.

  

• Tiles Tiles are a method of presenting summarized data in workbooks. Figure 3-100 shows an example of tiles in a workbook:

  

• Graphs Graphs can show the relationships between entities in the logs they are analyzing. See Figure 3-101 for an example of a graph visualization:

  

View and analyze Azure Sentinel data using workbooks

So far in this part of the chapter, we have discussed how you can configure and create workbooks, but next we’ll focus on how to use workbooks in the context of an SOC analyst responding to or investigating incidents. How an SOC uses workbooks is, of course, highly contextual and will depend on the organization’s wider IT operations processes, so this section will look at how workbooks can be used by an SOC in a more general context and explore some of the main analytical concepts.

For this example, we’re going to be using one of the built-in workbook templates, the Azure AD Sign-In Logs workbook:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Workbooks. The Workbooks gallery appears.

5. Click the Azure AD Sign-In Logs Workbook.

6. Click View Saved Workbook. The Sign-in Analysis Workbook appears, as shown in Figure 3-102.

   

7. Let’s look at this workbook from the perspective of an SOC analyst who wants to look for unusual activity in the Azure AD sign-in logs. As we can see in Figure 3-102, the number of sign-ins for each country is shown, as well as the trend for sign-ins from each country for the time range specified. The TimeRange is set to Last 7 Days.

8. Clicking in the Sign-Ins By Location box allows you to drill-in to more specific locations in the U.S., as shown in Figure 3-103.

   

9. We can see that both Redmond and Pflugerville are the most recorded locations for sign-ins in this workspace in the past 7 days and that there have been a couple of small spikes in sign-ins from these locations in the past week. (We can observe this by looking at the Trend column.)

10. Moving further down the workbook, the SOC analyst can analyze the number of successful versus unsuccessful sign-ins, as shown in Figure 3-104.

    

11. The workbook also shows the Summary Of Top Errors for the selected time period. Figure 3-104 shows that in this instance, the top sign-in error was Fresh Auth Token Is Needed. Have The User Re-Sign Using Fresh Credentials, followed by Invalid Username Or Password Or Invalid On-Premises Username Or Password. This information can assist an SOC analyst in understanding the baseline of the environment they manage, as well as help them look for anomalies.

12. The SOC analyst can choose to focus in on a specific user or change the time period that the workbook displays; this can be done via the parameters tabs—TimeRange, Apps, UserNamePrefix, and UserName—at the top of the workbook, as shown in Figure 3-105.

    

13. Finally, it might be necessary to print a hard or soft copy of the workbook to give to management for reporting purposes. This can be done by clicking the ellipsis (three dots) to the right of the workbook’s title, as shown in Figure 3-106.

    

Track incident metrics using the security operations efficiency workbook

Although all the Azure Sentinel workbook templates are very useful—they’ve all been written by experts in their field—they are often tied to a particular data source and will not be useful if you are not ingesting that data source into your Azure Sentinel workspace. However, the Security Operations Efficiency workbook is a workbook that uses the SecurityIncidents table to allow you to track key SOC metrics, such as the number of incidents raised, their severity, mean time to triage, and so on. Regardless of your organization’s security operations processes and the data sources you ingest, you will want to track key performance indicators (KPIs) of your SOC. (And even if you don’t, your management probably will!)

Let’s walk through this workbook to learn more about reporting:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Workbooks. The Workbooks gallery appears.

5. Click the Security Operations Efficiency workbook. The Security Operations Efficiency workbook appears, as shown in Figure 3-107. Here, you can see various SOC operational metrics for reporting purposes. In Figure 3-107, we can see that there was a large spike in new incidents on about May 2 and that most incidents are being raised as high-severity incidents.

   

6. Scrolling further into the workbook, we can see two key SOC operational metrics. Mean Time To Triage and Mean Time To Closure are—along with the number of incidents and severity—very commonly used SOC reporting metrics. (The example in Figure 3-108 shows a very slow SOC team. Mean Time To Triage should be much less than one day in real life. Fortunately, this is not a real SOC!)

   

Create custom hunting queries

If you’re reading this chapter from start to finish, you probably already know what this topic is going to start with: Microsoft includes many Sentinel hunting queries that you can use right out of the box that have been written by security experts. Before you take the time and effort to write a custom query, do look through the built-in hunting queries to see if they will meet your requirements. Hunting queries don’t have templates. The out-of-the-box queries exist in your hunting queries list, and you can’t edit them like you are able to with analytics rules and workbooks. If you want to edit an out of the box hunting rule, you will need to re-create the whole rule with your edited KQL query.

Let’s look at how to create a custom hunting rule in Azure Sentinel:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Hunting. The Hunting page appears, as shown in Figure 3-109.

   

5. Click New Query. The Create Custom Query page appears, as shown in Figure 3-110.

   

   Following is the KQL of the query shown in Figure 3-110:

   Click here to view code image

   SecurityEvent
   | where ParentProcessName contains "powershell.exe"

6. On the Create Custom Query page, you can define the Name, Description, Custom Query, Entity Mapping, and MITRE Tactics for your hunting query.

   Note Avoid References to Time Ranges

   Unlike an analytics rule, hunting query logic should not include any reference to time ranges because this prevents Azure Sentinel from showing you the change in query results over time to create a baseline for monitoring.

7. Click Create.

8. You will be returned to the Hunting page, and if you search for your custom hunting query by name, it should now appear in your hunting Queries list, as shown in Figure 3-111.

   

Run hunting queries manually

Hunting queries will almost always be run manually in Azure Sentinel, which we will explore in this topic:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Hunting, which opens the Hunting page.

5. Select the query that you want to run manually. The Hunting Query Preview pane appears, as shown in Figure 3-112.

   

6. Click Run Query. The hunting query will be run, and the Results column will be populated with the number of results that query has returned, as shown in Figure 3-113.

   

7. To see the hunting query results in more detail, click View Results in the Hunting Query Preview pane, and you will be taken to the Log Analytics page to see the raw results of the query.

8. To run multiple hunting queries at the same time, select the queries to be run on the Hunting page using the check boxes next to each query, as shown in Figure 3-114.

   

9. Click Run Selected Queries.

10. The selected queries will run, and the results will be displayed in the Results column in the same way they are shown when you run a single query.

Monitor hunting queries by using Livestream

Livestream is a way of running hunting queries continuously—every 30 seconds—and it will let you know if there are any new results matching the query. This is a great way to monitor for any baseline changes in your environment that can be indicative of a security issue but without raising any unnecessary false-positive incidents. Livestreams can also be useful for testing new queries, and it is also possible to “promote” a Livestream query to an analytics rule or just straight into an investigation.

Let’s look at how to use a Livestream in Azure Sentinel:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Hunting, which opens the Hunting page.

5. Click the Livestream tab, which opens the Livestream tab on the Hunting page, as shown in Figure 3-115.

   

6. The Livestream tab is not prepopulated with queries, so if this is your first time using the feature, it will be empty. To add a Livestream query, click New Livestream.

7. The Livestream page appears, as shown in Figure 3-116, using this the KQL:

   SecurityEvent
   | where EventID == 4625

   

8. Give your Livestream a Name and enter the KQL for the Livestream in the Query field. Remember, as for other hunting queries, you should not specify time periods in Livestream query logic because this prevents Azure Sentinel from detecting changes over time.

9. Click Save.

10. Click Play to start the Livestream.

    Tip Be Patient!

    If you are expecting immediate results with your livestream query, be patient! It can take up to 30 seconds for the Livestream to start and for results to be visible.

11. The Livestream will start running. Any results from the query will be displayed below on the Livestream page, as shown in Figure 3-117.

    

12. The Livestream results will be refreshed every 30 seconds.

13. You can pause the Livestream using the Pause button at the top of the Livestream page.

14. Return to the Livestream tab on the Hunting page, as shown in Figure 3-118.

    

15. The saved Livestream is now visible on the tab, and you can also see whether the Livestream is currently running by looking at the Status column.

16. You can pause and play livestreams from this page using the Livestream preview pane on the right side of the page, as shown in Figure 3-119.

    

17. Click Pause or Play in this pane. (This button toggles between Play and Pause.)

Track query results with bookmarks

An SOC analyst might look through hundreds of thousands of logs during a shift, and the human brain can only process and remember a limited number of details. This is where bookmarks come in handy. They are a tool in Azure Sentinel that allows you to save specific records from a query result that an SOC analyst can revisit if they need to. Bookmarks can be attached to incidents to assist with investigation, and as with most things in Azure Sentinel, they also are stored in their own table—the HuntingBookmark table—and they can be searched through using KQL.

Let’s look at how to create a bookmark in Azure Sentinel:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Hunting, which opens the Hunting page.

5. Select a hunting query to run. The Hunting Query Preview pane appears on the right side of the screen, as shown in Figure 3-120.

   

6. Click View Query Results or View Results. (Both take you to the Logs page.)

7. You will be taken to the Logs page, where you can view the results of the hunting query, as shown in Figure 3-121. The KQL of the query is as follows:

   Click here to view code image

   AzureActivity
   | where Category == "Administrative"

   

8. Select the record you want to bookmark by selecting that record and clicking Add Bookmark. The Add Bookmark pane appears, as shown in Figure 3-122.

9. You can enter details about the record you are saving as a bookmark, including the Name, Entity Mapping, Tags, and Notes to remind yourself or other SOC personnel what is important or noteworthy about this bookmark. When you’ve finished, click Create.

   Note Bookmarks in Investigation Graph

   To view a bookmark in the investigation graph, you need to map at least one entity in the bookmark.

   

10. To view your saved bookmarks, navigate to the Hunting page and click the Bookmarks tab, as shown in Figure 3-123.

    

11. You can also view saved bookmarks by navigating to the Bookmark Logs page and searching the HuntingBookmark table, as shown in Figure 3-124. The query being run is as follows:

    HuntingBookmark
    | take 10

    

Use hunting bookmarks for data investigations

Bookmarking certain records can be useful in isolation, but to get the full value of the bookmarks feature in Azure Sentinel, you need to use them to assist in investigations. There are several ways that you can use bookmarks to enhance your threat hunting when using Azure Sentinel.

Convert a hunting query to an analytics rule

We’ve spoken earlier in this section about how hunting is a proactive activity in security operations. But what if—during that proactive hunting activity—an SOC analyst finds an issue that needs to be escalated into an incident? This can be done quickly and easily on the Azure Sentinel hunting page either through a Livestream or direct from a hunting query.

Perform advanced hunting with notebooks

The Jupyter Project is an open-source project that was developed to assist data science computing across many programming languages. A Jupyter notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations, and more. Because security operations can essentially be thought of as a security-focused data science—in other words, looking for patterns and anomalies in data—Jupyter notebooks are very well suited to assisting SOC analysts to interpret data.

In Azure Sentinel, Jupyter notebooks run on the Azure Notebooks platform, which is directly connected to the Azure Sentinel user interface. Notebooks allow an SOC analyst to conduct investigations and hunting using a huge collection of programming libraries for machine learning, visualization, and data analysis. Microsoft has developed a library called Kqlmagic that allows you to take queries from Azure Sentinel and run them inside of a notebook. As the library name suggests, queries are still run using the KQL language.

As with the rest of Azure Sentinel, Microsoft have created notebook templates that can be used in production as they are, but they also give you some inspiration to make your own. Let’s go through how to start using notebooks:

1. Navigate to the Azure portal by opening https://portal.azure.com.

2. In the Search bar, type Sentinel, and under Services, click Azure Sentinel. The Azure Sentinel Workspace page appears.

3. Select the workspace you want to use. The Azure Sentinel | Overview page appears.

4. Click Notebooks, and the Notebooks page appears, as shown in Figure 3-136.

   

5. If you haven’t created an Azure Machine Learning Workspace yet, you’ll need to do this by clicking Create New AML Workspace. You will be taken to the Machine Learning page, as shown in Figure 3-137.

   

6. Complete the details of the deployment, which include Subscription, Resource Group, Workspace Name, Region, and so on. When you’ve finished, click Create.

7. Return to the Notebooks page in Azure Sentinel, select the notebook that you want to launch, and select Save Notebook.

8. You will be asked which Azure Machine Learning (AML) workspace you want to save the notebook to, as shown in Figure 3-138.

   

9. Click OK, and the notebook template will be saved to your AML workspace.

10. After the notebook has been saved, the Launch Notebook button will appear on the notebook preview pane, as shown in Figure 3-139.

    

11. Click Launch Notebook. Your notebook will open in the AML interface, as shown in Figure 3-140.

    

12. If this is the first time you’ve used your AML interface, you’ll also need to configure some compute to power your notebook before you can start running it.

13. At the top-right of the screen, you will see a + sign that displayes New Compute when you hover your mouse over it. Click the + and you will be taken to the Create Compute Instance page shown in Figure 3-141.

    

14. Select the VM you want to use for notebook compute, complete the Compute Name details, and click Create.

15. You can now launch a compute instance to run your notebook by selecting the compute instance and clicking the Start Compute triangle-shaped button, as shown in Figure 3-142.

    

16. You can now work through the notebook and execute the code in each cell by clicking the Run Cell button, as shown in Figure 3-143.

    

Security operations at Contoso Ltd.

You are the SOC manager for Contoso Ltd., a large global organization with offices and operations in several jurisdictions. The organization runs a hybrid environment with both on-premises and cloud IT infrastructure that needs to be monitored for any security breaches. Contoso Ltd. uses Azure Sentinel as its SIEM solution.

As a part of your duties for Contoso Ltd., you run a large follow-the-sun SOC across several countries with hundreds of staff: Tier 1 analysts run the initial triage and basic incident resolution; Tier 2 analysts handle incidents escalated to them from Tier 1; and Tier 3 analysts are the most experienced analysts who take on the most complex cases that Tiers 1 and 2 haven’t been able to resolve. Sometimes, this involves the need for Tier 2 analysts to change the configuration of Azure Sentinel.

Looking at your SOC metrics, you can see that both the mean time to triage and mean time to closure of the Contoso SOC is longer than you expected and that SOC analysts aren’t able to respond and close incidents as fast as your upper-management expects them to. Upon speaking to the SOC analysts, they tell you that they don’t get notified when a new incident is raised, and they have to manually check the incidents page to see if new incidents have occurred since they last checked. You want to configure more automation into your SOC processes to notify the SOC analysts when a new incident has been triggered.

With this information in mind, answer the following questions:

1. How can you ensure that Tier 1 and Tier 2 SOC analysts cannot change the data sources that are connected to Azure Sentinel and that only Tier 3 analysts have access to do this?

2. How can you monitor the mean time to triage and mean time in Azure Sentinel?

3. How can you ensure that an alert notification is sent to the SOC team when an incident is triggered?