Bandwidth

Bandwidth is the maximum rate of transfer over the network. Typically, this is defined in bits per second (abbreviated Bps), Mbps for one million bits per second, or Gbps for one billion bits per second. Network bandwidth defines the maximum bandwidth rate, but the actual user or application transfer rate will also be affected by latency, protocol, and packet loss. In comparison, throughput is the successful transfer rate over the network.

Latency

Latency is the delay between two points in a network. Latency can be measured in one-way delay or Round-Trip Time (RTT) between two points. Ping is a common way to test RTT delay. Delays include propagation delays for signals to travel across different mediums such as copper or fiber optics, often at speeds close to the speed of light. There are also processing delays for packets to move through physical or virtual network devices, such as the Amazon Virtual Private Cloud (Amazon VPC) virtual router. Network drivers and operating systems can be optimized to minimize processing latency on the host system as well.

Jitter

Jitter is the variation in inter-packet delays. Jitter is caused by a variance in delay over time between two points in the network. Jitter is often caused by variations in processing delays and queueing delays in the network, which increase with higher network load. For example, if the one-way delay between two systems varies from 10 ms to 100 ms, then there is 90 ms of jitter. This type of varying delay causes issues with voice and real-time systems that process media because the systems have to decide to buffer data longer or continue without the data.

Throughput

Throughput is the rate of successful data transferred, measured in bits per second. Bandwidth, latency, and packet loss affect the throughput rate. The bandwidth will define the maximum rate possible. Latency affects the bandwidth of protocols like Transmission Control Protocol (TCP) with round-trip handshakes. TCP uses congestion windows to control throughput. One side of the TCP connection will send a single segment of data and then wait for confirmation from the other side before sending more data. If the handshake continues at the single-segment rate, the throughput will be heavily affected by the latency involved in the round-trip confirmation. This is why TCP uses scaling window sizes to increase throughput. Conversely, User Datagram Protocol (UDP) doesn’t acknowledge packets with handshakes, though some applications built on top of UDP may implement RTT requirements. UDP will not adaptively throttle traffic rates when there is loss unless application logic decides to back off.

Packet Loss

Packet loss is typically stated in terms of the percentage of packets that are dropped in a flow or on a circuit. Packet loss will affect applications differently. TCP applications are generally sensitive to loss due to congestion control. For instance, TCP Reno, a common implementation of TCP, halves its congestion window on a single packet loss.

Packets per Second

Packets per second refers to how many packets are processed in one second. Packets per second are a common bottleneck in network performance testing. All processing points in the network must process each packet, requiring computing resources. Particularly for small packets, per-packet processing can limit throughput before bandwidth limits are reached. You can monitor the packets per second on AWS Direct Connect ports with Amazon CloudWatch metrics.

Maximum Transmission Unit

The Maximum Transmission Unit (MTU) defines the largest packet that can be sent over the network. The maximum on most Internet and Wide Area Networks (WANs) is 1,500 bytes. Jumbo frames are packets larger than 1,500 bytes. AWS supports 9,001 byte jumbo frames within a VPC. VPC peering and traffic leaving a VPC support up to 1,500 byte packets, including Internet and AWS Direct Connect traffic. Increasing the MTU increases throughput when the packet per second processing rate is the performance bottleneck.

Instance Networking

AWS offers instance families for different use cases, each with different network, computing, and memory resources.

Examples of Instance Families

• General Purpose (M4)
• Compute Optimized (C5)
• Memory Optimized (R4)
• Accelerated Computing (P2)

These families have different networking speeds and capabilities, such as enhanced networking and Amazon EBS-optimized networking. Each instance type’s network performance is documented as low, medium, high, 10 Gigabit, up to 10 Gigabit, or 20 Gigabit. Larger instance types in a family have bandwidth that generally scales with the vCPU quantity within the family.

Enhanced Networking

Enhanced networking uses Single Root I/O Virtualization (SR-IOV) and Peripheral Component Interconnect (PCI) passthrough to provide high-performance networking capabilities on supported instance types for Linux, Windows, and FreeBSD. SR-IOV and PCI passthrough are methods of device virtualization that provide higher I/O performance and lower CPU utilization when compared to traditional virtualized network interfaces. Enhanced networking provides higher bandwidth—over one million packets per second performance—and consistently lower inter-instance latencies. Combined with placement groups, it provides full bi-section bandwidth without bandwidth oversubscription for the largest instance types. Enhanced networking requires both operating system driver support and for the Amazon Machine Image (AMI) or instance to be flagged for Enhanced networking.

Enhanced Networking

If your application requires high network performance, we suggest using an instance type that supports enhanced networking. This is a fundamental step in reducing latency, packet loss, and jitter and in increasing bandwidth for instances. Remember to have both operating system support and for the instance to be flagged for enhanced networking support.

Jumbo Frames

For applications that require high throughput, such as bulk data transfer, increasing the MTU can increase throughput. In scenarios where the performance bottleneck is packets per second, increasing the MTU can increase overall throughput by sending more data per packet.

The most common MTU found on the Internet is 1,500 bytes, which is what AWS supports across AWS Direct Connect and Internet gateways. Any Ethernet frame larger than 1,500 bytes is called a jumbo frame. Certain instance families support an MTU of 9,001 bytes within a VPC. If you have a cluster in a placement group, enabling jumbo MTUs can increase the cluster performance. To enable jumbo MTUs, you will need to change the operating system network parameters. For example, on Linux this is a parameter of the ip command.

Network Credits

Instance families such as R4 and C5 use a network I/O credit mechanism. Most applications do not consistently need a high level of network performance, but can benefit from having access to increased bandwidth when they send or receive data. For example, the smaller R4 instance sizes offer peak throughput of 10 Gbps. These instances use a network I/O credit mechanism to allocate network bandwidth to instances based on average bandwidth utilization. These instances accrue credits when their network throughput is below their baseline limits and can then use these credits when they perform network data transfers.

If you plan on running performance baselines with instances that support network credits, we recommend accounting for built-up credits during the test. One approach is to test with freshly installed instances. You can also send a large amount of traffic until you get to a steady state of throughput after all credits have been exhausted. Currently, there is no metric to track network credits for instances.

Instance Bandwidth

Each instance type has a bandwidth definition ranging from low to 20 gigabit. Larger instance types have more bandwidth and packet per second capabilities. There are no explicit bandwidth limits for any single VPC, VPC peering connection, or Internet gateway. We suggest trying a larger instance type if your application has a bandwidth bottleneck. If you are not sure about your performance bottleneck, trying a larger instance size is the easiest method to determine whether bandwidth supported by the instance is your bottleneck. The instance’s allowed bandwidth is roughly proportional to the size of the instance. Different instance families, such as C3 and C4, use different hardware and potentially networking implementations, so they may have slightly different performance characteristics as traffic gets closer to your networking limits. The Compute-Optimized (C family) and General-Purpose (M family) instances are common choices for network-bound applications. Instance bandwidth is also dependent on the network driver in use. Instance types using the Intel 82599 interface with the ixgbevf module have both an aggregate and flow-based bandwidth limit of 5 Gbps. Instances with the AWS ENA driver have a 5 Gbps flow limit outside of a placement group but can achieve an aggregate bandwidth of 25 Gbps within a VPC or a peered VPC with multiple flows.

Flow Performance

In addition to instance bandwidth, the quantity of flows that your application uses also affects throughput. In a placement group, any single flow will be limited to 10 Gbps. This is important to understand so that you can use the full bandwidth of any instance with greater than 10 Gbps performance.

Outside of a placement group, the maximum throughput for a single flow is 5 Gbps. Examples include traffic between Availability Zones in the same VPC, a flow between an Amazon EC2 instance and Amazon S3, and traffic between an instance and an on-premises resource.

Load Balancer Performance

If your application will be using Elastic Load Balancing, you have multiple choices for load balancing. The Application Load Balancer has many HTTP and Layer 7 features. The Network Load Balancer has more TCP and Layer 3 features. These options are covered in more depth in Chapter 6, “Domain Name System and Load Balancing.”

The advantages of the Network Load Balancer are performance and scale. Since it is less computationally complex to forward packets without looking inside them, the Network Load Balancer scales faster and has lower latency. If your application does not require HTTP or Layer 7 features, you can improve performance with a Network Load Balancer. The additional latency is measured in microseconds for Network Load Balancer packet processing.

Virtual Private Network (VPN) Performance

The Virtual Private Gateway (VGW) is the AWS managed Virtual Private Network (VPN) service. When a VPN connection is created, AWS provides tunnels to two different VPN endpoints. These VPN endpoints are capable of approximately 1.25 Gbps per tunnel depending on packet size.

To increase bandwidth into AWS, you can forward traffic to both endpoints. This design requires that on-premises equipment support Equal Cost Multipath (ECMP) to load balance traffic across both links or to balance more preferred prefixes on each VPN endpoint. It is possible to set different route preferences so that traffic leaves from both VPN endpoints for egress diversity.

In addition to the AWS VGW, you can install a VPN endpoint on your own Amazon EC2 instances. This approach allows more options for routing, performance tuning, and encryption overhead. Note that AWS does not manage the availability of this option. You should either test the VPN endpoint performance in your own account or work with the software provider to obtain their performance evaluations.

AWS Direct Connect Performance

One of the primary reasons for using AWS Direct Connect is to obtain more predictable performance than can be obtained using a VPN. Using a dedicated circuit or existing network allows you to control the quality of the network between on-premises infrastructure and AWS. For example, you can use a dedicated fiber between your data center and the AWS Direct Connect facility to reduce latency.

Another advantage that AWS Direct Connect offers is high bandwidth. While the VGW service is multi-gigabit, it is not suitable for throughputs of 10 Gbps and higher. AWS Direct Connect allows for customers to provision multiple 10 Gbps connections and also aggregate those connections into a single 40 Gbps circuit.

Quality of Service (QoS) in a VPC

On-premises networks often support Quality of Service (QoS) with Differentiated Services Code Point (DSCP) in order to have more control over which traffic is prioritized in case of network congestion. All traffic is treated equally inside of a VPC. The DSCP is the seven bits in the IP header used to identify the priority of traffic. The DSCP is not used to modify traffic forwarding in AWS networks, but the header remains as it was received.

You can use AWS Direct Connect in conjunction with QoS to improve application performance and reliability. When packets leave on-premises and traverse any service provider networks that honor DSCP bits, QoS can be applied normally. The goal is to use service provider networks that honor QoS so that performance is improved from on-premises infrastructure to AWS. Even so, packets are not differentiated at the AWS edge of the connection. This is common for real-time communications packets and other flows that are sensitive to packet loss.

High Performance Computing

High Performance Computing (HPC) allows scientists and engineers to solve complex, compute-intensive, and data-intensive problems. HPC applications often require high network performance, fast storage, large amounts of memory, very high compute capabilities, or all of these. HPC performance can be bound by network latency, so it is important to minimize latency within a cluster.

Using placement groups with HPC enables access to a low-latency, high-bandwidth network for tightly coupled, IO-intensive, and storage-intensive workloads. For faster Amazon EBS IO, we recommend using Amazon EBS-optimized instances and Provisioned IOPS volumes for high performance.

Real-Time Media

Real-time media services include applications like Voice over IP (VoIP), media streaming using the Real-time Transport Protocol (RTP) or Real-time Messaging Protocol (RTMP), and other video and audio applications. Real-time media use cases include enterprise migrations of existing communications infrastructure as well as service provider telephony and video services.

Media streams can have different requirements on the network depending on the implementation and architecture. Video workloads can have varying bandwidth requirements during an existing flow, which can be dependent on the complexity of movement in the video. Audio flows may also change their bandwidth requirements if the audio stream supports redundancy or adaptive changes. In most cases, both audio and video streams are highly sensitive to packet loss and jitter. Packet loss and jitter can cause distortion and gaps in the media, which are easily detected by end users. We recommend taking steps to reduce loss and jitter.

The first step to reducing loss and jitter on AWS is to make sure that enhanced networking is enabled for real-time media applications. This feature provides a smoother packet delivery. If AWS Direct Connect is used, you can use QoS on the circuit if the provider or equipment supports it, reducing the chance of packet loss.

Detailed monitoring and proactive routing control can also mitigate network congestions and challenges. For highly sensitive media with multiple potential network paths, you can configure monitoring probes on Amazon EC2 instances to report on link health. That information can be used centrally to modify routes to alternative network paths that are healthy.

Some media applications support buffering traffic before the media is played to the user. This buffering can help guard against jitter and varying network latencies. For media streams that can buffer audio or video, decreasing jitter is more important than reducing the average latency.

Data Processing, Ingestion, and Backup

When you want to move, process, or back up terabytes of data in AWS, the network is an important consideration. Data transfer can occur from on premises or within a VPC. It may also include different storage services such as Amazon EBS and Amazon S3, which have different networking characteristics.

You should understand potential performance limitations for data transfers, particularly if the transfer rate is important for your operation. Data processing and transfer in AWS generally follows this flow:

1. Read data and potentially metadata from storage.
2. Encapsulate the data in a transfer protocol, such as File Transfer Protocol (FTP), Secure Copy (SCP), or HTTP.
3. Transfer the data over a network, such as VPN, the Internet, or AWS Direct Connect.
4. Decapsulate the data and perform validation or other processes.
5. Write the data to storage.

Network transfer is one part of the overall performance equation. The other performance components are storage IOPS, read performance, metadata processing, and write performance. It is possible for storage performance to be the primary bottleneck. You can try benchmarking different network configurations, test an entirely local transfer, and monitor storage performance rates to determine the relationship between the network and the storage transfer rate. If there are Amazon EC2 instances involved, such as VM Import/Export, you can also try using different instance sizes.

On-Premises Data Transfer

On-premises use cases may include migration to AWS, on-premises processing, or backup data. In addition to the storage components mentioned above, on-premises networking affects data movement performance.

One primary performance aspect is the existing Internet or private circuits available between transfer points. For Internet transfers, the existing Internet connection bandwidth and utilization can be a bottleneck. If there is a single 20 Mbps Internet connection that’s 50 percent utilized, that provides 10 Mbps of available bandwidth. For large transfers, AWS Direct Connect can provide a dedicated circuit with less latency and more predictable throughput. Provisioning AWS Direct Connect takes more time than configuring a VPN, however, so timing is a consideration.

Security is an important factor for data transfer. If the data transfer is over insecure protocols, we suggest using encryption over any untrusted connections. The techniques for increasing performance through VPN mentioned in the VPN Performance section of this chapter can be used in this scenario. Note that IP Security (IPsec) can limit performance due to the encapsulation involved.

There are many additional services and concepts that are outside the scope of the AWS Certified Advanced Networking – Specialty Exam. Consider using services like AWS Snowball, AWS Snowmobile, or AWS Storage Gateway for transferring datasets larger than 1 TB. Amazon S3 has additional optimizations such as Transfer Acceleration and multipart uploads. There may also be additional optimizations at the operating system level to tune window scaling, interrupts, and Direct Memory Access (DMA) channels.

Network Appliances

Routers, VPN appliances, NAT instances, firewalls, intrusion detection and prevention systems, web proxies, email proxies, and other network services have historically been hardware-based solutions in the network. On AWS, these solutions are implemented virtually on Amazon EC2. Since these solutions are deployed on operating systems on Amazon EC2, they begin to look like applications themselves, even if they participate in routing.

Some, or even all, of your VPC traffic can be forwarded through these instances, so it’s important to improve performance. You should size the instance correctly by testing your required throughput on different instance types. Enhanced networking is highly important for performance as well. It’s common for the network appliance to connect to external networks. If this is the case, placement groups will not increase performance to destinations outside of a placement group.

These network appliances may require multiple interfaces in different subnets to achieve different routing policies. You should understand that there are maximum interface counts and maximum amounts of IP addresses that you can apply per interface. As of this writing, a c4.8xlarge can have 8 interfaces with 30 IP addresses per interface. The quantity of network interfaces does not affect performance characteristics if the instance type supports enhanced networking. Remember, additional network interfaces do not change the networking performance.

For Amazon EC2 VPN instances, you should understand that the additional IPsec headers reduce the overall throughput because there is less data in each 1,500-byte frame. It’s important to reduce the MTU to allow additional room for headers for protocols like IPsec. Most applications will have mixed packet sizes that are less than the MTU, so Amazon EC2 VPN endpoints are likely to be bound by packets per second rather than CPU, memory, or network bandwidth.

One of the benefits of operating on AWS is scalability and the ability to build fault-tolerant applications—this concept applies to network appliances. If possible, network appliances should be able to use Auto Scaling and interact with Elastic Load Balancing to scale and be fault tolerant. The Network Load Balancer supports long-lived sessions based on source IP address, destination IP address, source port, destination port, and protocol (5-tuple hash), making it well suited for network applications that use TCP.

Routing traffic through instances is accomplished by modifying the routing table of subnets. Each subnet can have a route for a certain prefix (for example, default route) to the elastic network interface of a network appliance instance. When routing traffic to an elastic network interface, remember that you are responsible for the fault tolerance and availability of that route. By default, this route or elastic network interface requires additional configuration to be fault tolerant. Some approaches include Amazon EC2 instances with AWS Identity and Access Management (IAM) roles that allow them to modify routes or attach the elastic network interface to a new instance when the instance detects a failure. AWS has published some example scripts to accomplish failover in NAT instances (see https://aws .amazon.com/articles/2781451301784570). Another approach could include using AWS Lambda to monitor and provide fault tolerance.

Amazon CloudWatch Metrics

Amazon CloudWatch metrics make it easy to observe and collect data about your networks. Amazon CloudWatch metrics are available for many AWS Cloud services, including Amazon EC2. Amazon EC2 instances have a variety of CPU, memory, disk, and networking metrics available by default in five-minute increments. To receive one-minute metrics, detailed monitoring can be enabled for an additional cost. For the exam, you should understand which metrics are available, but you do not need to memorize specific details. Table 9.1 lists available instance networking metrics in Amazon CloudWatch.

In addition to instance metrics, Amazon CloudWatch metrics is available for Amazon VPN. Table 9.2 lists available Amazon EC2 VPN metrics in Amazon CloudWatch.

Table 9.3 lists available AWS Direct Connect metrics in Amazon CloudWatch.

Testing Methodology

Application performance is a combination of memory, CPU, networking, storage, application architecture, latency, and other factors. Testing different configurations and settings is a cost-effective way to determine how to increase performance. There are a wide array of tools and methods available, so this section will focus on testing as it relates to AWS networking.