9 Alerting
______________________________
In This Chapter
Introduction
Activity Alerts
Alert Policies
_______________________________
Introduction
Alerts and Alert Policies are intended to help automate certain types of notifications for suspicious activity, general user activity or even possibly administrator activities. Alerts can be shown in the Alerting Dashboard or send alerts to administrators via email. Microsoft is also making changes to Alerting, so expect updates in the future and possible elimination of PowerShell cmdlets to reflect those changes.
In the Security and Compliance Center, we have two types of alerts that we can configure:
Activity Alerts: These are the original alerting objects for the Security and Compliance Center. In this chapter we will cover them for historical purposes, but would encourage the creations of the newer Alert Policies, shown below.
Alert Policies: A more advanced version of the ‘Activity Alert’. Alert Polices provide some more advanced parameters in PowerShell and in the Security and Compliance Center. They are also the replacement for Activity Alerts:
Current Alert Polices Console:
Activity Alerts
Activity Alerts are designed to notify admins of certain types of activities as they occur in your tenant. These alerts are also being phased out for a new experience with Alert Policies. We can still review the PowerShell cmdlets and effort it takes to manage Activity Alerts in the Security and Compliance Center.
PowerShell
First we need a list of PowerShell cmdlets that we can use to work with SCC Activity Alerts:
Get-Command *ActivityAlert
This provides us with a small list of cmdlets:
Get-ActivityAlert
New-ActivityAlert
Remove-ActivityAlert
Set-ActivityAlert
First, let’s review what we have by default in our tenant:
None. Okay. So we have a blank slate to work with. Once we create a few of these Activity Alerts, we will revisit this cmdlet to pull some actual data and see how we can go about documenting and report on the alerts.
New-ActivityAlert
As is obvious by the cmdlet, we can use New-ActivityAlert to create new alerts in the SCC. What exactly does this mean? What alerts can we create with PowerShell and how detailed are we allowed to get. First, let’s check out the examples available with the cmdlet:
Get-Help New-ActivityAlert -Examples
Reviewing the single example provided, we see that we have the following parameters: name, operation, notifyuser, UserId, and Description. Giving the Alert a name should be easy and it should be relevant to the Alert being performed. What about Operation? Within the help for the cmdlet, there is a link to where we can find a list of the available alertable operations - https://docs.microsoft.com/en-us/office365/securitycompliance/search-the-audit-log-in-security-and-compliance. The list provided is long as it covers many operations:
File and Page Activities
Folder Activities
SharePoint list activities
Sharing and access request activities
Synchronization activities
Site permissions activities
Site administration activities
Exchange mailbox activities
Sway activities
User administration activities
Azure AD group administration activities
Application administration activities
Role administration activities
Directory administration activities
eDiscovery activities
Advanced eDiscovery activities
Power BI activities
Microsoft Teams activities
Yammer activities
As we can see, this is a pretty extensive list of general activities. Each of these activity groups has a lot of sub activities that are covered as well. Let’s run through a few examples of how we can create alerts and then monitor the progress of the alerts we create.
Example 1 - eDiscovery Alerting
For this example we want to create an alert that will notify users of a group called eDiscoveryManagers. The alert will look for times when cases are create/managed, or Content Searches are performed.
New-ActivityAlert -Name ‘eDiscovery - Cases and Search Alert’ -Operation CaseAdded, CaseUpdated, CaseAdminUpdated, CaseAdminAdded, CaseAdminRemoved -NotifyUser [email protected] -Description ‘Case Management Auditing’
Notice that no ‘UserID’ was specified like in the Example. This is because the cmdlet is not limited to a set of users, but looks for any user in the tenant that performs these actions. We also did not set a Category or Severity, but the default severity is low and the default category is none. If we would like to have either of these items used for filtered reporting (i.e. alerts by a particular category or severity level) then we would need to put a setting in.
Available Categories: None (This is the default value), DataLossPrevention, ThreatManagement, DataGovernance, AccessGovernance and Others
Available Severity Levels: None, Low, Medium or High
NotifyUser - Any notification messages that are sent, because of this Alert, will go to the specified email address.
Additional Options
Multiplier: Number of events that occur to trigger the alert
RecordType: Workload in Office 365 to be monitored. Values:
ExchangeAdmin, ExchangeItem, ExchangeItemGroup, SharePoint, SyntheticProbe, SharePointFileOperation OneDrive, AzureActiveDirectory, AzureActiveDirectoryAccountLogon, DataCenterSecurityCmdlet, ComplianceDLPSharePoint, Sway, ComplianceDLPExchange, SharePointSharingOperation, AzureActiveDirectoryStsLogon, SkypeForBusinessPSTNUsage, SkypeForBusinessUsersBlocked, SecurityComplianceCenterEOPCmdlet, ExchangeAggregatedOperation, PowerBIAudit, CRM, Yammer, SkypeForBusinessCmdlets, Discovery, MicrosoftTeams, MicrosoftTeamsAddOns, MicrosoftTeamsSettingsOperation, ThreatIntelligence
ScopeLevel: SingleUser, AllUser
Threshold: Number of events that trigger an alert, minimum of three, and works in conjunction with the TimeWindow Value
TimeWindow - specified in minutes, it represents a window of time in which the number of alerts passes the value specified in the Threshold value above. Type parameter must be ‘SimpleAggregation’
Type: Custom, ElevationOfPrivilege, SimpleAggregation, AnomalousAggregation
Example 2 - Exchange Mailbox Permissions Activity
In this example we are going to track activity for ‘Exchange mailbox activities’ for all users in the IT Admins group. For this we’ll use the ‘Multiplier’ parameter and set the value to three. We are looking for a lot of activity of Admins in Exchange mailboxes.
New-ActivityAlert -Name ‘Admin Query -Multiplier 3 - Exchange Mailbox Permissions’ -Operation AddMailboxPermissions,UpdateFolderPermissions,RemoveMailboxPermission -Notify [email protected]
Example 3
For this example, we want to monitor all Sway sharing activities. We review the list of what can be audited for operations and come up with this list - SwayShare,SwayChangeShareLevel,SwayRevokeShare,SwayExternalSharingOff,SwayExternalSharingOn. The Activity Alert will be called ‘Sway Share Alerts’:
New-ActivityAlert -Name ‘Sway Share Alerts’ -Operation SwayShare, SwayChangeShareLevel, SwayRevokeShare, SwayExternalSharingOff, SwayExternalSharingOn -Notify [email protected]
Remove-ActivityAlert
This cmdlet, Remove-ActivityAlert, has very few options. We can remove one or more of these Activity Alerts with this cmdlet:
Remove-ActivityAlert -Identity ‘eDiscovery - Cases and Search Alert’
The removal process will prompt to make sure removal is OK:
Now, you could also remove all Activity Alerts as well:
Get-ActivityAlert | Remove-ActivityAlert
Set-ActivityAlert
Set-ActivityAlert will allow us to modify existing Activity Alerts and change certain properties to adjust for whatever scenario you may need for the Alert. Since the default is low severity, you may want to increase it to Medium since it is an important alert to send out. Low would send the wrong impression.
Set-ActivityAlert -Identity ‘eDiscovery - Cases and Search Alert’ -Severity Medium
The change in severity does not register any feedback at all. However, we can verify the change like so:
(Get-ActivityAlert -Identity ‘eDiscovery - Cases and Search Alert’).Severity
We can also add a Category to the Alert:
With the Set-ActivityAlert we can change many options for the alerts:
UserId, Operation, RecordType, Severity, Category, Threshold
What do reports look like?
After we’ve configured our Alerts, some with notification, we can now review any alerting in the SCC console:
Alerts Polices
Alert Policies are here to replace the current Activity Alert. What we notice is that a lot of the parameters are the same, some are renamed and new ones have been added. We also have a similar set of PowerShell cmdlets:
PowerShell
First we need a list of PowerShell cmdlets that we can use to work with SCC Activity Alerts:
Get-Command *ProtectionAlert
This provides us with a small list of cmdlets:
Get-ProtectionAlert
New-ProtectionAlert
Remove-ProtectionAlert
Set-ProtectionAlert
First, let’s review what we may have by default in our tenant:
Get-ProtectionAlert | Ft Name,Severity,*severity,*type,*Scenario
Now that we’ve seen what is available by default, lets see what we can create for Alert Policies:
Get-Help
No Examples
Sample code from the Online DOCs page:
New-ProtectionAlert -Name “Content search deleted” -Category Others -NotifyUser [email protected] -ThreatType Activity -Operation SearchRemoved -Description “Custom alert policy to track when content searches are deleted” -AggregationType None
Description - This example creates an alert policy that triggers an alert whenever anyone in the organization deletes a Content Search in the Office 365 Security & Compliance Center.
So we can see that we have one example of New-ProtectionAlert work with. We can review the list of parameters of the cmdlet as well to see what we need to build a good ProtectionAlert. Remember that not all of these are required:
AggregationType: Determine how an alert is triggered with multiple occurrences. We can set this to none, SimpleAggregation or AnomalousAggregation. We can combine SimpleAggregation with the Threshold and TimeWindow parameters. AnomalousAggregation triggers alerts on highly unusual levels of activity (using baselines).
AlertBy: Specifies scope of the alerts - either based on activity or malware.
AlertFor: Reserved for Microsoft only.
Category: Adds a tag to the alert when the conditions of the alert are met. There are five available values.
Comment: Place to add extra details about the alert.
Description: Descriptive text for the alert.
Disabled: We can set this in case we need to turn an Alert Policy off.
Name: Identifies the rule for those reviewing alerts in the Alerts Dashboard.
NotificationCulture: Specify the language of alerts, using the values from the Microsoft .NET Framework CultureInfo class.
NotificationEnabled: Turns on or off notifications for the alert.
NotifyUser: Specify an email address of who to notify (internal or external address)
NotifyUserOnFilterMatch: Trigger an alert on a single match even if aggregate is specified ($True)
NotifyUserSuppressionExpiryDate: Date to which to temporarily suspend user alert notifications, after which alerts will resume.
NotifyUserThrottleThreshold: Sets a maximum number of alerts that can be sent within the NotifyUserThrottleWindow value specified.
NotifyUserThrottleWindow: Time interval used by the NotifyUserThrottleThreshold.
Operation: Like the New-ActivityAlert cmdlet, we can specify conditions to alert on. Complete list can be found here - https://go.microsoft.com/fwlink/p/?linkid=824986
Severity: Default is Low, but we can set the severity to none, low, medium or high.
ThreatType: Specify a type of activity to monitor [none, malware, phish, activity].
Threshold: Same as the New-ActivityAlert cmdlet.
TimeWindow: Same as the New-ActivityAlert cmdlet.
WhatIf: Reserved for Microsoft only.
For this next section we’ll run through a couple scenarios on creating alerts and then view them in the Security and Compliance Center to get a feel for how they operate in Office 365.
Scenario One
We have a brand new tenant and we want to start alerting on the removing of users. All of the alerts will be set to Medium severity as it does not require immediate attention by IT. Notification alerts will be sent to the admin responsible for the licensing and users in Office 365 - [email protected].
New-ProtectionAlert -Name ‘User Change Alert’ -Description ‘Alert when a user is added or deleted or licensing is modified.’ -Comment ‘Notify Frank with alerts.’ -ThreatType Activity -Operation DeletedUser -Notifyuser ‘[email protected]’ -Category AccessGovernance -Threshold 10 -TimeWindow 180
A second alert will be configured for suspicious changes, which in our case would be high volume changes of the first alert. Notification alerts will be sent to the IT groups email address ([email protected]).
New-ProtectionAlert -Name ‘Mass User Change Alert’ -Description ‘Alert when suspicious activity occurs for adding/removing users and licensing’ -Comment ‘Suspicious activity alert, sent to all IT.’ -ThreatType Activity -Operation DeletedUser -Notifyuser ‘[email protected]’ -Category AccessGovernance -AggregationType AnomalousAggregation
These new alerts we created will appear in the Security and Compliance Center Under Alerts --> Alert Policies:
Singular alerts will now be delivered to Frank in IT immediately, whereas the other alerts, for the IT group, will need at least a week before any alerts could be delivered as background logic creates a baseline for what is to be expected in an environment. Once the baseline is ready, then any suspicious activity with User Deletions will generate alerts for the IT group.
If we need to modify this alert for new users being added or other Operations, we can simply run a one-liner to update it. Remember this will completely replace the Operation currently set:
Set-ProtectionAlert -Identity ‘User Change Alert’ -Operation AddedUser
** Note ** We can add multiple Operations as long as the values are comma separated.
Scenario Two
A consulting firm uses Sway to create client presentations and these are all stored in their own corporate tenant. Various people have access to the presentation including other outside contractors. Problems in the past with presentations being removed without notification or permission has led to the need for some sort of alerting. For this we will create an alert to monitor the deletion of Sway data using Protection Alerts.
New-ProtectionAlert -Name ‘Sway Deletion Alert’ -Description ‘Alert when a user makes a Sway deletion.’ -Comment ‘Notify Sway Admin’ -ThreatType Activity -Operation SwayDelete -Notifyuser ‘[email protected]’ -Category AccessGovernance -AggregationType None
Now we have an Alert that notifies the email address [email protected] if a user deletes a Sway.
Beyond the New
Once we get beyond creating new alerts we only have three other cmdlets we can use to manage Protection Alerts. These cmdlets allow us to Get, Remove and Set. Remove is the easiest cmdlet to use as we only need an identifier for a Policy in order to remove it. However, that requires finding the Identity. We can do that with Get-Protection Alerts. However, if we run ‘Get-ProtectionAlert’ without any other switches or formatting, we get a blur of information that whirls by. What? It looks like Get-ProtectionAlert has Format-List as its default format. So we will simply add a Format-Table and choose two values we see in the Format-List below:
Get-ProtectionAlert | Ft Name
Now we have a list of all of the alerts and properties we can use with Remote and Set cmdlets:
To remove an alert, we can use either the name. Examples:
Remove-ProtectionAlert -Identity ‘User Change Alert’
Remove-ProtectionAlert -Identity ‘Mass User Change Alert’
Now, if we have items that we need to adjust or change on an alert, we can use the name of the alerts provided above as well as the numerous options that are available. For example we can change the notified user email address:
Set-ProtectionAlert -Identity ‘User Change Alert’ -Notifyuser ‘[email protected]’
We can change the severity of the alert:
Set-ProtectionAlert -Identity ‘Mass User Change Alert’ -Severity High
And possibly even change the Threshold values to something higher if we are getting too many alerts:
Set-ProtectionAlert -Identity ‘User Change Alert’ -Threshold 10 -TimeWindow 1200
None of these changes will provide immediate feedback, but we will see the changes in the Security and Compliance Center as well as in Powershell.