Management Events

Management events include operations that a principal executes (or attempts to execute) against an AWS resource. AWS also calls management events control plane operations. Management events are further grouped into write‐only and read‐only events.

• Write‐Only Events   Write‐only events include API operations that modify or might modify resources. For example, the RunInstances API operation may create a new EC2 instance and would be logged, regardless of whether the call was successful. Write‐only events also include logging into the management console as the root or an IAM user. CloudTrail does not log unsuccessful root logins.
• Read‐Only Events   Read‐only events include API operations that read resources but can't make changes, such as the DescribeInstances API operation that returns a list of EC2 instances.

Data Events

Data events track two types of data plane operations that tend to be high volume: S3 object‐level activity and Lambda function executions. For S3 object‐level operations, CloudTrail distinguishes read‐only and write‐only events. GetObject—the action that occurs when you download an object from an S3 bucket—is a read‐only event, whereas DeleteObject and PutObject are write‐only events.

Event History

By default, CloudTrail logs 90 days of management events and stores them in a viewable, searchable, and downloadable database called the event history. The event history does not include data events.

CloudTrail creates a separate event history for each region containing only the activities that occurred in that region. So, if you're missing events, you're probably looking in the wrong region! But events for global services such as IAM, CloudFront, and Route 53 are included in the event history of every region.

Trails

If you want to store more than 90 days of event history or if you want to customize the types of events CloudTrail logs—for example, by excluding specific services or actions, or including data events such as S3 downloads and uploads—you can create a trail.

A trail is a configuration that records specified events and delivers them as CloudTrail log files to an S3 bucket of your choice. A log file contains one or more log entries in JavaScript Object Notation (JSON) format. A log entry represents a single action against a resource and includes detailed information about the action, including, but not limited to, the following:

• eventTime   The date and time of the action, given in universal coordinated time (UTC). Log entries in a log file are sorted by timestamp, but events with the same timestamp are not necessarily in the order in which the events occurred.
• userIdentity  Detailed information about the principal that initiated the request. This may include the type of principal (e.g., IAM role or user), its Amazon resource name (ARN), and IAM username.
• eventSource  The global endpoint of the service against which the action was taken (e.g., ec2.amazonaws.com).
• eventName  The name of the API operation (e.g., RunInstances).
• awsRegion  The region the resource is located in. For global services, such as Route 53 and IAM, this is always us‐east‐1.
• sourceIPAddress  The IP address of the requester.

Log File Integrity Validation

CloudTrail provides a means to ensure that no log files were modified or deleted after creation. When an attacker hacks into a system, it's common for them to delete or modify log files to cover their tracks. During quiet periods of no activity, CloudTrail also gives you assurance that no log files were delivered, as opposed to being delivered and then maliciously deleted. This is useful in forensic investigations where someone with access to the S3 bucket may have tampered with the log file.

With log file integrity validation enabled, every time CloudTrail delivers a log file to the S3 bucket, it calculates a cryptographic hash of the file. This hash is a unique value derived from the contents of the log file itself. If even one byte of the log file changes, the entire hash changes. Hashes make it easy to detect when a file has been modified.

Every hour, CloudTrail creates a separate file called a digest file that contains the cryptographic hashes of all log files delivered within the last hour. CloudTrail places this file in the same bucket as the log files but in a separate folder. This allows you to set different permissions on the folder containing the digest file to protect it from deletion. CloudTrail also cryptographically signs the digest file using a private key that varies by region and places the signature in the file's S3 object metadata.

Each digest file also contains a hash of the previous digest file, if it exists. If there are no events to log during an hourlong period, CloudTrail still creates a digest file. This lets you know that no log files were delivered during the quiet period.

You can validate the integrity of CloudTrail log and digest files by using the AWS CLI. You must specify the ARN of the trail and a start time. The AWS CLI will validate all log files from the starting time to the present. For example, to validate all log files written from January 1, 2021, to the present, you'd issue the following command:

aws cloudtrail validate‐logs ‐‐trail‐arn arn:aws:cloudtrail:us‐east‐1:account‐id:trail/benpiper‐trail ‐‐start‐time2021‐01‐01T00:00:00Z.

CloudWatch Metrics

CloudWatch organizes metrics into namespaces. Metrics from AWS services are stored in AWS namespaces and use the format AWS/service to allow for easy classification of metrics. For example, AWS/EC2 is the namespace for metrics from EC2, and AWS/S3 is the namespace for metrics from S3.

You can think of a namespace as a container for metrics. Namespaces help prevent metrics from being confused with similar names. For example, CloudWatch stores the WriteOps metric from the Relational Database Service (RDS) in the AWS/RDS namespace, whereas the EBS metric VolumeWriteOps goes in the AWS/EBS namespace. You can create custom namespaces for custom metrics. For example, you can store metrics from an Apache web server under the custom namespace Apache. Metrics exist only in the region in which they were created.

A metric functions as a variable and contains a time‐ordered set of data points. Each data point contains a timestamp, a value, and optionally a unit of measure. Each metric is uniquely defined by a namespace, a name, and optionally a dimension. A dimension is a name/value pair that distinguishes metrics with the same name and namespace from one another. For example, if you have multiple EC2 instances, CloudWatch creates a CPUUtilization metric in the AWS/EC2 namespace for each instance. To uniquely identify each metric, AWS assigns it a dimension named InstanceId with the value of the instance's resource identifier.

Graphing Metrics

CloudWatch can perform statistical analysis on data points over a period of time and graph the results as a time series. This is useful for showing trends and changes over time, such as spikes in usage. You can choose from the following statistics:

• Sum   The total of all data points in a period
• Minimum   The lowest data point in a period
• Maximum   The highest data point in a period
• Average   The average of all data points in a period
• Sample count   The number of data points in a period
• Percentile   The data point of the specified percentile. You can specify a percentile of up to two decimal places. For example, 50 would yield the median of the data points in the period. You must specify a percentile statistic in the format p50.

To graph a metric, you must specify the metric, the statistic, and the period. The period can be from one second to 30 days, and the default is 60 seconds. If you want CloudWatch to graph each data point as is, use the Sum statistic and set the period equal to the metric's resolution. For example, if you're using detailed monitoring to record the CPUUtilization metric from an EC2 instance, that metric will be stored at one‐minute resolution. Therefore, you would graph the CPUUtilization metric over a period of one minute using the Sum statistic. To get an idea of how this would look, refer to Figure 7.1.

Note that the statistic applies only to data points within a period. In the preceding example, the Sum statistic adds all data points within a one‐minute period. Because CloudWatch stores the metric at one‐minute resolution, there is only one data point per minute. Hence, the resulting graph shows each individual metric data point.

The time range in the preceding graph is set to one hour, but you can choose a range of time between 1 minute and 15 months. Choosing a different range doesn't change the time series, but only how it's displayed.

Which statistic you should choose depends on the metric and what you're trying to understand about the data. CPU utilization fluctuates and is measured as a percentage, so it doesn't make much sense to graph the sum of CPU utilization over, say, a 15‐minute period, because doing so would yield a result over 100 percent. It does, however, make sense to take the average CPU utilization over the same timeframe. Hence, if you were trying to understand long‐term patterns of CPU utilization, you can use the Average statistic over a 15‐minute period.

The NetworkOut metric in the AWS/EC2 namespace measures the number of bytes sent by an instance during the collection interval. To understand peak hours for network utilization, you can graph the Sum statistic over a one‐hour period and set the time range to one day, as shown in Figure 7.2.

The Details column shows information that uniquely identifies the metric: the namespace, metric name, and the metric dimension, which in this case is InstanceId.

Metric Math

CloudWatch lets you perform various mathematical functions against metrics and graph them as a new time series. This capability is useful for when you need to combine multiple metrics into a single time series by using arithmetic functions, which include addition, subtraction, multiplication, division, and exponentiation. For example, you might divide the AWS/Lambda Invocations metrics by the Errors metric to get an error rate. Complete Exercise 7.2 to create a CloudWatch graph using metric math.

In addition to arithmetic functions, CloudWatch provides the following statistical functions that you can use in metric math expressions:

• AVG—Average
• MAX—Maximum
• MIN—Minimum
• STDDEV—Standard deviation
• SUM—Sum

Statistical functions return a scalar value, not a time series, so they can't be graphed. You must combine them with the METRICS function, which returns an array of time series of all selected metrics. For instance, referring to step 6 in Exercise 7.2, you could replace the expression m1+m2 with the expression SUM(METRICS()) to achieve the same result. This would simply add up all the graphed metrics.

As another example, suppose you want to compare the CPU utilization of an instance with the standard deviation. You would first graph the AWS/EC2 metric CPUUtilization for the instance in question. You'd then add the metric math expression METRICS()/STDDEV(m1), where m1 is the time series for the CPUUtilization metric. See Figure 7.3 for an example of what such as graph might look like.

Keep in mind that the function STDDEV(m1) returns a scalar value—the standard deviation of all data points for the CPUUtilization metric. You must therefore use the METRICS function in the numerator to yield a time series that CloudWatch can graph.

CloudWatch Logs

CloudWatch Logs is a feature of CloudWatch that collects logs from AWS and non‐AWS sources, stores them, and lets you search and even extract custom metrics from them. Some common uses for CloudWatch Logs include receiving CloudTrail logs, collecting application logs from an instance, and logging Route 53 DNS queries.

{$.eventSource = "signin.amazonaws.com" && $.responseElements.ConsoleLogin = "Failure" }

CloudWatch Alarms

A CloudWatch alarm watches over a single metric and performs an action based on a change in its value. The action CloudWatch takes can include tasks such as sending an email notification, rebooting an instance, or executing an Auto Scaling action.

To create an alarm, you first define the metric you want CloudWatch to monitor. In much the same way CloudWatch doesn't graph metrics directly but graphs metric statistics over a period, a CloudWatch alarm does not directly monitor metrics. Instead, it performs statistical analysis of a metric over time and monitors the result.

Amazon EventBridge

EventBridge (formerly known as CloudWatch Events) monitors for and takes an action either based on specific events or on a schedule. For example, a running EC2 instance entering the stopped state would be an event. An IAM user logging into the AWS Management Console would be another event. EventBridge can automatically take immediate action in response to such events.

EventBridge differs from CloudWatch Alarms in that EventBridge takes some action based on specific events, not metric values. Hence, EventBridge can send an SNS notification as soon as an EC2 instance stops. Or it can execute a Lambda function to process an image file as soon as a user uploads it to an S3 bucket.

The Configuration Recorder

The configuration recorder is the workhorse of AWS Config. It discovers your existing resources, records how they're configured, monitors for changes, and tracks those changes over time. By default, it monitors all items in the region in which you configure it. It can also monitor the resources of global services such as IAM. If you don't want to monitor all resources, you can select specific resource types to monitor, such as EC2 instances, IAM users, S3 buckets, or DynamoDB tables. You can have only one configuration recorder per region.

Configuration Items

The configuration recorder generates a configuration item for each resource it monitors. The configuration item contains the specific settings for that resource at a point in time, as well as the resource type, its ARN, and when it was created. The configuration item also includes the resource's relationships to other resources. For example, a configuration item for an EBS volume would include the instance ID of the instance it was attached to at the time the item was recorded. AWS Config maintains configuration items for every resource it tracks, even after the resource is deleted. Configuration items are stored internally in AWS Config, and you can't delete them manually.

Configuration History

AWS Config uses configuration items to build a configuration history for each resource. A configuration history is a collection of configuration items for a given resource over time. A configuration history includes details about the resources, such as when it was created, how it was configured at different points in time, and when it was deleted, if applicable. It also includes any related API logged by CloudTrail.

Every six hours in which a change occurs to a resource, AWS Config delivers a configuration history file to an S3 bucket that you specify. The S3 bucket is part of what AWS calls the delivery channel. Configuration history files are grouped by resource type. The configuration history for all EC2 instances goes into one file, and the history for EBS volumes goes into another. The files are timestamped and kept in separate folders by date. You can also view configuration history directly from the AWS Config console. You can optionally add an SNS topic to the delivery channel to have AWS Config notify you immediately whenever there's a change to a resource.

Configuration Snapshots

A configuration snapshot is a collection of all configuration items from a given point in time. Think of a configuration snapshot as a configuration backup for all monitored resources in your account.

Using the AWS CLI, you can have AWS Config deliver a configuration snapshot to the bucket defined in your delivery channel. By default, the delivery channel is named default, so to deliver a configuration snapshot, you would manually issue the following command:

aws configservice deliver-config-snapshot --delivery-channel-name default

AWS Config can automatically deliver a configuration snapshot to the delivery channel at regular intervals. You can't set automatic delivery of this snapshot in the console but must configure the delivery using the CLI. To do this, you must also specify a JSON file containing at a minimum the following items:

• Delivery channel name (default)
• S3 bucket name
• Delivery frequency of the configuration snapshot

The file may also optionally contain an SNS ARN. Let's refer to the following file as deliveryChannel.json:

{
  "name": "default",
  "s3BucketName": "my-config-bucket-us-east-1",
   "snsTopicARN": "arn:aws:sns:us-east-1:account-id:config-topic",
  "configSnapshotDeliveryProperties": {
    "deliveryFrequency": "TwentyFour_Hours"
  }
}

The delivery frequency can be every hour or every 24, 12, 6, or 3 hours. To reconfigure the delivery channel according to the settings in the deliveryChannel.json file, you'd issue the following command:

aws configservice put-delivery-channel --delivery-channel file://deliveryChannel.json

To verify that the configuration change succeeded, issue the following command:

aws configservice describe-delivery-channels

If the output matches the configuration settings in the file, then the configuration change was successful.

Monitoring Changes

The configuration recorder generates at least one new configuration item every time a resource is created, changed, or deleted. Each new item is added to the configuration history for the resource as well as the configuration history for the account.

A change to one resource will trigger a new configuration item for the changed resource and related resources. For instance, removing a rule in a security group causes the configuration recorder to create a new item for the security group and every instance that uses that security group.

Although you can't delete configuration items manually, you can configure AWS Config to keep configuration items between 30 days and 7 years. Seven years is the default. Note that the retention period does not apply to the configuration history and configuration snapshot files AWS Config delivers to S3.

aws configservice stop-configuration-recorder --configuration-recorder-name default

aws configservice start-configuration-recorder --configuration-recorder-name default

1. You've configured CloudTrail to log all management events in all regions. Which of the following API events will CloudTrail log? (Choose all that apply.)
   1. Logging into the AWS console
   2. Creating an S3 bucket from the web console
   3. Uploading an object to an S3 bucket
   4. Creating a subnet using the AWS CLI
2. You've configured CloudTrail to log all read‐only data events. Which of the following events will CloudTrail log?
   1. Viewing all S3 buckets
   2. Uploading a file to an S3 bucket
   3. Downloading a file from an S3 bucket
   4. Creating a Lambda function
3. Sixty days ago, you created a trail in CloudTrail to log read‐only management events. Subsequently someone deleted the trail. Where can you look to find out who deleted it? No other trails are configured.
   1. The IAM user log
   2. The trail logs stored in S3
   3. The CloudTrail event history in the region where the trail was configured
   4. The CloudTrail event history in any region
4. What uniquely distinguishes two CloudWatch metrics that have the same name and are in the same namespace?
   1. The region
   2. The dimension
   3. The timestamp
   4. The data point
5. Which type of monitoring sends metrics to CloudWatch every five minutes?
   1. Regular
   2. Detailed
   3. Basic
   4. High resolution
6. You update a custom CloudWatch metric with the timestamp of 15:57:08 and a value of 3. You then update the same metric with the timestamp of 15:57:37 and a value of 6. Assuming the metric is a high‐resolution metric, which of the following will CloudWatch do?
   1. Record both values with the given timestamp.
   2. Record the second value with the timestamp 15:57:37, overwriting the first value.
   3. Record only the first value with the timestamp 15:57:08, ignoring the second value.
   4. Record only the second value with the timestamp 15:57:00, overwriting the first value.
7. How long does CloudWatch retain metrics stored at one‐hour resolution?
   1. 15 days
   2. 3 hours
   3. 63 days
   4. 15 months
8. You want to use CloudWatch to graph the exact data points of a metric for the last hour. The metric is stored at five‐minute resolution. Which statistic and period should you use?
   1. The Sum statistic with a five‐minute period
   2. The Average statistic with a one‐hour period
   3. The Sum statistic with a one‐hour period
   4. The Sample count statistic with a five‐minute period
9. Which CloudWatch resource type stores log events?
   1. Log group
   2. Log stream
   3. Metric filter
   4. CloudWatch Agent
10. The CloudWatch Agent on an instance has been sending application logs to a CloudWatch log stream for several months. How can you remove old log events without disrupting delivery of new log events? (Choose all that apply.)
    1. Delete the log stream.
    2. Manually delete old log events.
    3. Set the retention of the log stream to 30 days.
    4. Set the retention of the log group to 30 days.
11. You created a trail to log all management events in all regions and send the trail logs to CloudWatch logs. You notice that some recent management events are missing from the log stream, but others are there. What are some possible reasons for this? (Choose all that apply.)
    1. The missing events are greater than 256 KB in size.
    2. The metric filter is misconfigured.
    3. There's a delay between the time the event occurs and the time CloudTrail streams the event to CloudWatch.
    4. The IAM role that CloudTrail assumes is misconfigured.
12. Two days ago, you created a CloudWatch alarm to monitor the VolumeReadOps on an EBS volume. Since then, the alarm has remained in an INSUFFICIENT_DATA state. What are some possible reasons for this? (Choose all that apply.)
    1. The data points to monitor haven't crossed the specified threshold.
    2. The EBS volume isn't attached to a running instance.
    3. The evaluation period hasn't elapsed.
    4. The alarm hasn't collected enough data points to alarm.
13. You want a CloudWatch alarm to change state when four consecutive evaluation periods elapse with no data. How should you configure the alarm to treat missing data?
    1. As Missing
    2. Breaching
    3. Not Breaching
    4. Ignore
    5. As Not Missing
14. You've configured an alarm to monitor a metric in the AWS/EC2 namespace. You want CloudWatch to send you a text message and reboot an instance when an alarm is breaching. Which two actions should you configure in the alarm? (Choose two.)
    1. SMS action
    2. Auto Scaling action
    3. Notification action
    4. EC2 action
15. In a CloudWatch alarm, what does the EC2 recover action do to the monitored instance?
    1. Migrates the instance to a different host
    2. Reboots the instance
    3. Deletes the instance and creates a new one
    4. Restores the instance from a snapshot
16. You learn that an instance in the us‐west‐1 region was deleted at some point in the past. To find out who deleted the instance and when, which of the following must be true?
    1. The AWS Config configuration recorder must have been turned on in the region at the time the instance was deleted.
    2. CloudTrail must have been logging write‐only management events for all regions.
    3. CloudTrail must have been logging IAM events.
    4. The CloudWatch log stream containing the deletion event must not have been deleted.
17. Which of the following may be included in an AWS Config delivery channel? (Choose all that apply.)
    1. A CloudWatch log stream
    2. The delivery frequency of the configuration snapshot
    3. An S3 bucket name
    4. An SNS topic ARN
18. You configured AWS Config to monitor all your resources in the us‐east‐1 region. After making several changes to the AWS resources in this region, you decided you want to delete the old configuration items. How can you accomplish this?
    1. Pause the configuration recorder.
    2. Delete the configuration recorder.
    3. Delete the configuration snapshots.
    4. Set the retention period to 30 days and wait for the configuration items to age out.
19. Which of the following metric math expressions can CloudWatch graph? (Choose all that apply.)
    1. AVG(m1)‐m1
    2. AVG(m1)
    3. METRICS()/AVG(m1)
    4. m1/m2
20. You've configured an AWS Config rule to check whether CloudTrail is enabled. What could prevent AWS Config from evaluating this rule?
    1. Turning off the configuration recorder
    2. Deleting the rule
    3. Deleting the configuration history for CloudTrail
    4. Failing to specify a frequency for periodic checks
21. Which of the following would you use to execute a Lambda function whenever an EC2 instance is launched?
    1. CloudWatch Alarms
    2. EventBridge
    3. CloudTrail
    4. CloudWatch Metrics