Chapter 14. Traffic Shaping
14.0. Introduction
Quality of Service (QoS) deals with the order of packets in transmission queues, also called outgoing interface buffers, by controlling the order in which packets are dispatched from those queues. The simplest form is called FIFO (first in, first out) queuing, whereby an interface has a single outgoing buffer in which packets are dispatched into the link in the order they arrive in the queue. More sophisticated forms use multiple queues and complex distribution schemes, and can even manipulate higher-protocol-layer traffic control mechanisms. QoS is an important part of today’s shared public backbone networks. Although QoS is essential for providers to deliver service-level agreements (SLAs) to their subscribers, it never gained the same significance in enterprise networks.
On ScreenOS devices, QoS was developed to provide better utilization for narrow band access links, in particular those with a speed of 1.5 Mbps or lower. QoS can help to soften the spikes in bursty traffic flows, therefore increasing overall bandwidth utilization, but it cannot increase bandwidth over those provided by the network, as it is not a compression technology. It is also possible to prioritize one traffic flow over another, causing degradation of the lower-priority traffic flow, and it can even block the lower-priority traffic flow altogether. QoS is therefore not a magic bullet for solving bandwidth problems. Increasing network bandwidth would be the solution for that.
As wide area networks (WANs) became faster and cheaper over the past few years, the original reason for implementing QoS in the enterprise disappeared because the effects of bursty traffic are lower the faster the network becomes (and it’s never desirable to drop traffic, even if it is lower-priority). In recent years, QoS has became popular again, not for assisting traffic on slow links but to increase service quality for real-time traffic flows such as Voice over IP (VoIP). Real-time traffic suffers from delay and jitter. Delay is caused when a real-time traffic packet has to wait in a transmission queue for a larger packet to be dispatched first. Jitter occurs when the gap in time between different packets changes from packet to packet. Because real-time flows are usually distributed over very small packets, which are dispatched with an equal spacing, delays in interface buffers can alter the inter-packet delay differently for each packet. This is also most noticeable on slow links. The benefits of QoS diminish on Ethernet-speed links, but it can be quite significant for remote sites with slow access links, such as VoIP over Asymmetric Digital Subscriber Line (ADSL) with a slow uplink speed.
QoS has no effect on transmission quality in the network unless the quality is agreed upon with the provider in what is referred to as traffic qualification or traffic marking. In this context, QoS signals a desired behavior within the network. Providers police traffic according to agreements and the SLA specifies the preferred traffic classes and priorities for that customer. The Internet is commonly a best-effort service level, whereas Multiprotocol Label Switching (MPLS) providers commonly offer different service levels to their customers. (The old Plesiochronous Hierarchy [PDH] and Sonet networks are line rate—there is a fixed share of bandwidth reserved whether it is used by the subscriber or not.) Therefore, traffic marking has benefits only when an agreement with a service provider exists to honor the marking and the provider uses a shared infrastructure such as MPLS or IP Security (IPSec) virtual private networks (VPNs).
For enterprise networks, the benefits of QoS can be seen at remote sites. Therefore, traffic shaping is not implemented on large ASIC devices, such as the ISG and the NS-5000 Series, but it is implemented for SSG devices and most of the older ScreenOS devices. All models support traffic marking.
There are essentially two ways to configure traffic shaping: strict priority queuing and an automated or custom distribution scheme often called fair queuing or custom queuing. ScreenOS uses a combination of the two. In ScreenOS, there are eight strict priority queues, controlled by a dual-bucket model per policy. In simple terms, traffic shaping is configured per policy by placing that policy in a certain priority queue. Higher-priority queues can crowd out traffic for lower-priority queues, so there is a safety mechanism similar to custom queuing that can be configured on a per-policy basis, called maximum bandwidth (MBW). MBW avoids runaway queues; after the MBW for a policy is reached, or there is no more traffic queued in that queue, the next queue is served. In addition to having MBW on a per-policy basis, a guaranteed bandwidth (GBW) can be configured. GBW reserves traffic of a certain portion of the network speed for a given policy. The GBW of all policies in the same interface (zone) combination cannot add up to more than the bandwidth provided in the network. (Although technically possible, oversubscription would break the model and should never be configured.) Traffic above GBW is shared on a first come, first serve basis within a queue priority.
Traffic shaping can be defined as ingress or egress. This means that from the perspective of the policy, egress policing is configured via the MBW and the GBW object from the client-to-server direction, but is also applied for the return flow. Ingress policing is configured via a policing bandwidth (PBW) and GBW when the traffic enters the firewall, but it is applied as well for both directions of a flow. (PBW was introduced in ScreenOS 5.3.)
GBW, MBW, PBW, and queue priorities are associated with policies. Within each policy and within each queue, traffic flow is determined by first come, first serve. Traffic is controlled when a packet enters the firewall and it is matched to a policy. Both branches of a flow, from client to server and reverse, are associated with the same policy. First the policy is checked to see whether it has GBW configured. If it does but GBW for that policy has not been reached, the packet is queued for immediate transmission. Because all of the GBW together must be below network speed and therefore at or below the line rate, no packet reordering needs to take place. If GBW has been reached, or no GBW was configured, MBW (or PBW) is checked. If the policy reached MBW (or PBW), the packet is dropped. If the policy is under MBW (or PBW), the packet is placed in the queue, configured on that policy. Because higher-priority queues are served first, it is possible that higher-priority queues will crowd out all lower-priority queues and that a packet will never be transmitted. If this happens, the packet is saved in the queue until the buffer depth is reached and then dropped. There is no prioritization of traffic within a flow or session, and therefore, the packet order stays the same within each flow.
The buffer depth depends on the device model and is usually a few milliseconds deep. Deeper queues cause more jitter; therefore, it may be more beneficial to simply drop a packet if it has not been transmitted after a few milliseconds instead of delaying it. If a packet is being dropped, the traffic control mechanism of higher layers kicks in. For instance, Transmission Control Protocol (TCP) flows will decrease window size and transmission rate until traffic is no longer dropped. ScreenOS does not implement Weighted Fair Queuing (WFQ) or Random Early Drop (RED). (WFQ is a technology to enhance the feel of interactive traffic on very slow links, and RED is a simple throttling technology used in high-speed links in provider networks.) ScreenOS instead changed the window size to throttle TCP traffic. User Datagram Protocol (UDP), unicast, and multicast rely on the application or out-of-band mechanism to control transmission rate.
Beginning with ScreenOS 5.3, queues are also implemented on virtual interfaces. All GBW and MBW is shared in an interface tree. On the root of the tree is the physical interface. A virtual local area network (VLAN) subinterface could be, for instance, a child interface of a physical Ethernet interface. A VLAN subinterface itself could be a parent for a VPN tunnel interface. ScreenOS associates a Traffic Management Object (TMNG) to each parent and child interface. The meaning of GBW does not change.
It is the maximum network speed and cannot be larger than the speed of the physical access link. This means all GBW from all interfaces in a tree cannot add up to more than the physical interface’s line rate. MBW and PBW are spillover objects. Whatever is not used up from one child interface is available to the parent interface. The internal workings of TMNG and child and parent interfaces are transparent to the user. There is no configuration parameter associated with it. Remember, all GBW in an interface tree cannot be higher than the (real) access rate of the interface for a given direction.
Policy-based traffic shaping is an initial ScreenOS feature, but is implemented only on non-ASIC platforms; interface policing, meanwhile, was implemented only on the NS-5000 with some special code trains of ScreenOS 4.0 and later. ScreenOS 5.4 introduced interface policing for all platforms, whereas policy-based traffic shaping is still supported only on non-ASIC platforms. With interface policing, bandwidth can be throttled for all traffic entering or exiting the firewall through a given interface. Interface policing is a simple throttling mechanism, whereas policy-based traffic shaping applies an elaborate scheduling scheme with multiple queues.
All platforms support marking, also called traffic classification. Traffic is marked when you change the value of the Type of Service (ToS) byte in the IP header of each packet. ScreenOS does not react on premarked packets. It can pass this information unchanged, or it can overwrite it. The default is to pass it. Traffic shaping is independent of QoS marking. ScreenOS does not mark QoS fields in Layer 2 headers, such as the Class of Service (CoS) field in the Ethernet header. Although marking can be used in any way the user wants it, two common schemes exist: Integrated Services (IntServ) and Differentiated Services (DiffServ).
IntServ is described in RFCs 1633, 2212, and 2215. It defines eight priority classes, which are, by default, associated with the eight queues in ScreenOS. Queue 0 is assigned to IntServ 7 and queue 7 is assigned to IntServ 0. Queue 0 or IntServ 7 is the highest. The first three most significant bits are called the precedence bits in the ToS byte of the IP header. Many QoS schemes, such as the COS bits of Ethernet or the EXP bits of MPLS, follow the same scheme.
DiffServ is described in RFCs 2474 and 2475. In DiffServ, there are four types of perhop behavior (PHB). DiffServ uses six bits as opposed to the three bits in IntServ. The big difference, however, comes from the philosophy of how traffic is treated. In the IntServ model, each device should control traffic, whereas in the DiffServ model, the edge devices mark while core devices execute. This difference has less to do with how to configure ScreenOS than it does with understanding the fundamental differences in concepts. DiffServ uses only priority classes for backward compatibility with IntServ and calls that the Class Selector PHB. DiffServ itself classifies traffic in three types of PHB: Default PHB, Expedited Forwarding (EF) PHB, and Assured Forwarding (AF) PHB. All traffic is, by default, in the Default PHB (0×00). The EF PHB (0×2E) has strict priority over other traffic. Classified user traffic ends up in one of 12 AF groups. There is no predefined priority among AF groups; service levels are left up to custom definitions.
IntServ and DiffServ are more concepts in designing a network with QoS than they are mandatory to the network, a firewall, or ScreenOS in particular. ScreenOS prior to 5.2 supported only the three IntServ bits. In ScreenOS 5.2, the Class Selector PHB was added. ScreenOS 5.3 added support for all six DiffServ bits.
14.1. Configure Policy-Level Traffic Shaping
Solution
Configure egress traffic shaping on the policy by placing the policy into a priority queue and configuring optional GBW and MBW:
set policy from "Trust" to "Untrust" "Any" "Any " "FTP" permit traffic priority 3 mbw 1024 set policy from "Trust" to "Untrust "Any" "Any " "BGP" permit traffic priority 0 mbw 512 set policy from "Trust" to "Untrust" "Any" "Any " "MAIL" permit set policy from "Trust" to "Untrust" "Any" "Any " "SIP" permit traffic gbw 144 priority 2 mbw 288
You can alternatively configure ingress traffic shaping with PBW instead of MBW:
set policy from "Trust" to "Untrust" "Any" "Any " "FTP" permit traffic priority 3 pbw 1024
Discussion
To control packet ordering on interface buffers, configure traffic shaping in a policy. ScreenOS does not honor any ToS byte marking, such as IntServ or DiffServ bits. As explained in the introduction to this chapter, ScreenOS supports eight strict priority queues with GBW and MBW. This is a combination of strict priority and custom queuing.
MBW is used to avoid runaway queues in a strict priority scheme where a higher-priority queue can outcrowd all lower-priority queues. GBW is used to reserve bandwidth for a given policy, which is independent of priority queuing. A packet is dispatched immediately as long as GBW is not satisfied. If GBW is satisfied, traffic is sorted into the priority queue as long as MBW is not satisfied. However, if MBW is satisfied, the packet is dropped. Higher-priority queues are cleared before a lower-priority queue is allowed to transmit a packet. If a packet remains in a lower-priority queue for more than a few milliseconds, it will be dropped. This triggers transport and application layer backup algorithms. If bandwidth is exhausted, even if MBW has not been satisfied, the packet is dropped because the queue overruns. In other words, if line rate has been reached, the packet is dropped even if MBW has not been reached. Per its definition, GBW has to be at line rate or lower, and has to be equal to or lower than MBW. All GBW together should not be higher than the Committed Information Rate (CIR), which is usually lower than the access rate or line rate on that interface.
Recall that queue 7 is the default queue for all traffic. By default, GBW is set to 0 and MBW is set to an access rate of the interface, that is, 100 Mbps:
set policy from "Trust" to "Untrust" "Any" "Any " "FTP" permit traffic priority 3 mbw 1024 set policy from "Trust" to "Untrust" "Any" "Any " "BGP" permit traffic priority 0 mbw 512 set policy from "Trust" to "Untrust" "Any" "Any " "MAIL" permit set policy from "Trust" to "Untrust" "Any" "Any " "SIP" permit traffic gbw 144 priority 2 mbw 288
As shown in the output, the rule with BGP has the highest priority and traffic is sorted into queue 0. Protect this high-priority queue from runaway traffic by setting the MBW to 512 kbps. Configure interactive File Transfer Protocol (FTP) traffic higher than background MAIL traffic by configuring the rule to priority 3. You still want VoIP (Session Initiation Protocol [SIP]) traffic higher than FTP so that you can enjoy a clear phone conversation while downloading a file by configuring the policy into queue 2. You also guarantee bandwidth for the VoIP traffic by configuring GBW to guarantee 64 kbps plus header overhead. MAIL traffic ends up in the default queue with the implied priority 7.
As a standard, traffic shaping is applied automatically on outbound interface buffers for ingress and egress traffic. Even so, the policy defines only traffic from in the client-to-server direction. Instead of controlling outbound buffers, you can also control inbound buffers. This is rarely done, as traffic already was transmitted over the network, but it can be done for other reasons, such as enforcing service agreements. To do this, configure PBW instead of MBW:
set policy from "Trust" to "Untrust" "Any" "Any " "FTP" permit traffic priority 3 pbw 1024
Traffic shaping in ScreenOS follows a complex algorithm that can degrade performance on a device by as much as 60 percent. Because traffic shaping is most beneficial on edge devices with slow access links, this is acceptable as you will have plenty of performance left even if you do encryption for VPN. Traffic shaping has little benefit on fast links and networks, except when implemented in a provider environment while enforcing service agreements. Traffic shaping is therefore not implemented on the ASIC platforms of the ISG and NS-5000 Series.
Traffic shaping is switched off by default. It will automatically be switched on when a single policy is configured with the traffic keyword. However, you can activate or deactivate traffic shaping manually:
set traffic-shaping mode off
You can check your configuration and its effectiveness by looking at token use on the policy and on the interface. There are regular and borrowed tokens. A token is borrowed if GBW was reached or was not configured. You would configure GBW, for instance, for VoIP, which always has higher priority over everything else. If you see borrowed tokens in this case, you know that the GBW is not high enough. Remember that you cannot guarantee one policy 100 Mbps and another 100 Mbps while the total interface link speed is only 100 Mbps. Both policies may claim their guarantees at the same time. GBW is configured only if minimal service levels should be guaranteed and is usually lower than the expected average bandwidth for that traffic. Therefore, to see borrowed tokens is normal for most policies with or without GBW configured. If you see dropped packets, you reached either MBW or line rate. You should not see extensive dropped packets unless you want to police bandwidth intentionally with MBW. If you see extensive drops and you did not reach MBW, you know your network bandwidth is too low for your traffic requirements. Occasional drops are normal, and are caused when the TCP traffic control mechanism is trying to learn the speed of the connection. Sudden extensive drops can also be a sign of a runaway queue caused, for instance, by a rogue or faulty application.
FIREWALL->get policy id 1009name:"none" (id 1009), zone Trust -> Untrust,action Permit, status "enabled" src "voip", dst "sip.juniper.net", serv "SIP" Policies on this vpn tunnel: 0 nat src, Web filtering disabled vpn unknown vpn, policy flag 00000020, session backup: on, idle reset: on traffic shapping on, scheduler n/a, serv flag 00 log no, log count 0, alert no, counter no(0) byte rate(sec/min) 0/0 total octets 11178, counter(session/packet/octet) 0/0/0priority 2,diffserv marking Off tadapter: state on, gbw/mbw 144/288 policing (no) -----------------------------------------------------------------tmng (17): interface ethernet1 state on priority 2bw usage [for last one second]: 0 kbps pak queue(cur/max): 0/14 pak received: 9 pak dropped(out/shared): 0/0 PreShapingBytes (dropped/total): 0/5341 diffserv-marking: 0x0 elapsed time: 3570537 ms gbw/mbw: 144/288 (kbps) gbw_q/mbw_q: 18/36 shared_tmng: 5 PostShapingBytes(total/borrowed):5341/0 tokens (regular/borrowd): 0/5341 token bucket (gbl/mbl): 18000/40500 tokens(gua/max): 18000/40500 -----------------------------------------------------------------tmng (18): interface ethernet3 state on priority 2bw usage [for last one second]: 0 kbps pak queue(cur/max): 0/14 pak received: 7 pak dropped(out/shared): 0/0 PreShapingBytes (dropped/total): 0/5846 diffserv-marking: 0x0 elapsed time: 3570531 ms gbw/mbw: 144/288 (kbps) gbw_q/mbw_q: 18/36 shared_tmng: 7 PostShapingBytes(total/borrowed):5846/281 tokens (regular/borrowd): 0/5846 token bucket (gbl/mbl): 18000/40500 tokens(gua/max): 18000/40500 No Authentication No User, User Group or Group expression set
You can see the GBW and MBW configured on each TMNG. One TMNG exists for each policy on each interface. The TMNG number corresponds to the TMNG on the ingress and egress interfaces. The first TMNG configured on the policy and the shared TMNG is the default queue on the interface.
14.2. Configure Low-Latency Queuing
Solution
Configure the policy, define the GBW and MBW, and classify traffic into priority queue 2:
set policy from "Trust" to "Untrust" "voip" "sip.juniper.net" "SIP" nat src permit traffic gbw 144 priority 2 mbw 288
Configure all other policies without GBW into queue 7:
set policy from "Trust" to "Untrust" "Any" "Any" "ANY" nat src permit log
Discussion
Low-latency queuing is a simplified subset of strict-priority queuing. A single traffic flow, usually VoIP, is preferred over all other flows, which end up in a default queue. Whenever a VoIP packet is ready for transmission, it is transmitted before any other queued packet, therefore minimizing delay and jitter. This design is typical for remote office devices.
To implement low-latency queuing, first configure the VoIP policy. Because VoIP protocols use static control ports and dynamic ports for media, configure a service in the policy which references an application layer gateway (ALG) (see Chapter 11). For instance, say that the predefined service SIP reference port 5060/UDP, which is the control port for SIP, and the SIP ALG listen automatically on 5060/UDP for a call to arrive. The user agent and the SIP proxy negotiate a media channel, commonly a UDP port between 16384 and 32768. Within the UDP segment, a small Real-time Transport Protocol (RTP) header is added, followed by a few bytes of payload data. The RTP header describes the encapsulated media. Although you do not know on which port the media is sent, the ALG automatically makes sure the traffic shaping configuration is also applied to the negotiated media channels:
set policy from "Trust" to "Untrust" "voip" "sip.juniper.de" "SIP" nat src permit traffic gbw 144 priority 2 mbw 288
In this example, a GBW of 144 kbps was configured, which is more than twice the payload of a single media channel of 64 kbps, encoded in G.711. You also have to account for the RTP (12 bytes), UDP (8 bytes), and IP (20 bytes) headers. The pay-load is typically 32 bytes, which makes it a 72-byte packet with a 144 kbps bandwidth requirement. To avoid runaway traffic, you should also configure an upper cap with MBW. Runaway traffic can occur if a faulty or malicious application hits the policy, allowing the packets to be prioritized over all other traffic, thereby starving all other traffic.
Make sure no other policy uses GBW or priority 2 or higher:
set policy from "Trust" to "Untrust" "Any" "Any" "ANY" nat src permit log
Notice that the preceding policy also enabled Port Address Translation (PAT) with the keyword nat src, which is not a requirement but is typical for this kind of deployment. It is also typical in such a deployment to have a return policy (so that VoIP from both sides of the firewall can negotiate a call):
set interface ethernet3 dip interface-ip incoming set policy from "Untrust" to "Trust" "Any" "DIP(ethernet3)" "SIP" permit traffic gbw 144 priority 2 mbw 288
Notice when you configure PAT that your destination address object is the default dynamic IP (DIP) of the incoming interface, here configured with the keyword incoming. This is also independent of traffic shaping, but it is a typical configuration parameter for this kind of deployment with VoIP. The return policy is not necessary to have bidirectional media traffic because the ALG takes care of this automatically; however, it is necessary because both sides need to initiate a call.
Warning
Do not use the STUN protocol with the ALG, as the ALG makes the VoIP protocol PAT-compatible. Stateful firewalls are not compatible with STUN, as it takes advantage of the limited state in simple Network Address Translation (NAT) routers to open incoming connections, something a stateful firewall was designed to deny.
You can check your configuration as follows:
FIREWALL->get policy id 1009name:"none" (id 1009), zone Trust -> Untrust,action Permit, status "enabled" src "voip", dst "sip.juniper.net",serv "SIP"Policies on this vpn tunnel: 0 nat src, Web filtering disabled vpn unknown vpn, policy flag 00000020, session backup: on, idle reset: on traffic shapping on, scheduler n/a, serv flag 00 log no, log count 0, alert no, counter no(0) byte rate(sec/min) 0/0 total octets 8626, counter(session/packet/octet) 0/0/0priority 2,diffserv marking Off tadapter: state on,gbw/mbw 144/288 policing (no)-------------------------------------------------------------------- tmng (17): interface ethernet1 state on priority 2 bw usage [for last one second]: 0 kbps pak queue(cur/max): 0/14 pak received: 7 pak dropped(out/shared): 0/0 PreShapingBytes (dropped/total): 0/4307 diffserv-marking: 0x0 elapsed time: 22658 ms gbw/mbw: 144/288 (kbps) gbw_q/mbw_q: 18/36 shared_tmng: 5 PostShapingBytes(total/borrowed):4307/0 tokens (regular/borrowd): 0/4307 token bucket (gbl/mbl): 18000/40500 tokens(gua/max): 18000/40500 ---------------------------------------------------------------------- tmng (18): interface ethernet3 state on priority 2 bw usage [for last one second]: 0 kbps pak queue(cur/max): 0/14 pak received: 5 pak dropped(out/shared): 0/0 PreShapingBytes (dropped/total): 0/4320 diffserv-marking: 0x0 elapsed time: 13819 ms gbw/mbw: 144/288 (kbps) gbw_q/mbw_q: 18/36 shared_tmng: 7 PostShapingBytes(total/borrowed):4320/281 tokens (regular/borrowd): 0/4320 token bucket (gbl/mbl): 18000/40500 tokens(gua/max): 18000/40500 No Authentication No User, User Group or Group expression set
You can see the configured GBW and MBW in the priority queue 2. Note that traffic shaping was applied in both directions from just the one outbound rule. Beyond receiving some insider information about token bucket allocation, you might receive more interesting information if tokens are borrowed, which would mean there is not enough GBW configured for that policy. You want to prioritize VoIP over all other traffic under any condition. You can see that in the inbound direction the traffic stream is higher than 144 kbps because, in addition to media traffic, there is also control traffic. You could configure GBW a notch higher to include control traffic, but it probably would make little difference during the call because little control traffic will be exchanged.
14.3. Configure Interface-Level Traffic Policing
Solution
Configure ingress traffic policing on the interface:
set interface e3 bandwidth ingress mbw 1024
Or, configure egress traffic policing:
set interface e3 bandwidth egress mbw 1024
Discussion
The difference between interface-based traffic policing and policy-based traffic shaping is that with interface-based traffic policing, all traffic, regardless of the policy through which it is permitted, is limited to a certain bandwidth. This is independent of queues, their priorities, and which policy actually is permitting the traffic.
Interface-based traffic policing is used to police and shape traffic to a certain CIR. It is not used for preferring certain flows over others or for smoothing out bursts. Interface-based traffic shaping is supported in ScreenOS 5.4 and later on all devices, as well as with some special, earlier code trains on the NS-5000 Series. It is most often used in connection with WAN (subinterface) or VPN links. ASIC support exists for interface-based traffic policing on NS-5000 Series devices, but not for policy-based traffic shaping, which is not supported on ASIC platforms.
You configure interface-based traffic policing ingress or egress on the interface through which you want to police traffic:
set interface tun.1 bandwidth ingress mbw 1024 set interface s1.1 bandwidth egress mbw 1024
Check your configuration as follows:
FIREWALL->get traffic-shaping interface e3------------------------------------ ethernet3: physical bw=100000kbps, config gbw=2048kbpsmbw = 2048kbps, current bw=0kbps total allocated gbw=208kbps Shaping parameters ---------------------------------------------------------------------- tmng (7): interface ethernet3 state on priority 7 bw usage [for last one second]: 12 kbps pak queue(cur/max): 0/26 pak received: 11657 pak dropped(out/shared): 0/0 PreShapingBytes (dropped/total): 0/1477604 diffserv-marking: 0x0 elapsed time: 6979458 ms gbw/mbw: 1840/2048 (kbps) gbw_q/mbw_q: 230/256 shared_tmng: 0 PostShapingBytes(total/borrowed):1477604/0 tokens (regular/borrowd): 0/1640540 token bucket (gbl/mbl): 128000/128000 tokens(gua/max): 128000/6249835
Notice the configured MBW bandwidth of 2 Mbps, or 2,048 kbps. You can also see that the interfaces have a GBW set to MBW. The maximum amount of bandwidth of an interface is the highest amount of guaranteed bandwidth an interface can provide. In the lower portion of the output, you can see that 1,840 kbps are available for guarantees because tokens for 208 kbps, as seen in the top line, are already allocated for a VoIP call. Bandwidth usage for the last second was only 12 kbps, and the interface counters show that MBW was never reached and no packets were dropped. You can also see that interface MBW uses the TMNG of the default and the lowest priority queue 7. Packet drops would be a sign that you reached the configured MBW.
14.4. Configure Traffic Classification (Marking)
Solution
Configure DiffServ marking within a policy:
set policy from "Trust" to "Untrust" "voip" "sip.juniper.net" "SIP" permit traffic dscp enable value 18
Discussion
Traffic classification is used on edge devices to a backbone network, commonly called Customer Edge (CE) routers. Its purpose is to classify traffic flows into service levels for special treatment within the network cloud. Marking refers to overwriting the first three or six bits of the ToS byte within the header of each IP packet. This is different from actually acting on a marking. ScreenOS devices do not act on ToS byte marking, but can pass the current marking or overwrite it. Marking is supported on all NetScreen, SSG, and ISG platforms, and is ASIC-assisted on ASIC platforms, providing marking in line rate. There is no performance penalty with marking. The more granular traffic can be classified, the more control you have on the treatment of traffic within the network cloud. Therefore, edge firewalls are a very good place to do marking because traffic is commonly classified via policies and policies usually can be more complex than ACLs on a router.
Although marking can be totally arbitrary and is up to the service provider of the backbone network, two standards are common: IntServ and DiffServ.
IntServ uses only the first three bits of the ToS byte (also called the Precedence bits, as detailed in Table 14-1). Higher IntServ numbers have higher priority over lower values; this is the opposite of the numbering of queue priorities within ScreenOS, where lower-numbered queues have higher priority over higher-numbered queues. Default traffic is tagged Routine, whereas VoIP traffic is usually tagged Immediate. The two highest queues are reserved for control traffic such as routing protocols.
|
Value |
Description |
|
0 or 0 (0 × 00) |
Routine |
|
1 or 8 (0 × 08) |
Priority |
|
2 or 16 (0 × 10) |
Immediate |
|
3 or 24 (0 × 18) |
Flash |
|
4 or 32 (0 × 20) |
Flash Override |
|
5 or 40 (0 × 28) |
Critical |
|
6 or 48 (0 × 30) |
Inter-network Control |
|
7 or 56 (0 × 38) |
Network Control |
DiffServ, on the other hand, uses the first six bits of the ToS byte. DiffServ uses the Class Selector PHB to set the first three bits exactly as IntServ, but it also sets the next three bits to zero. DiffServ uses a Default PHB with all bits set to zero. DiffServ also defines an Expedited Forwarding PHB 46 (0 × 2E) for network control traffic or other urgent traffic. All other traffic is sorted into Assured Forwarding PHBs (as detailed in Table 14-2). Unlike with IntServ, AF PHBs do not have a default hierarchy but usually depend on the assignment to a service class with the individual provider.
|
Class 1 |
Class 2 |
Class 3 |
Class 4 | |
|
Low drop |
AF11 |
AF21 |
AF31 |
AF41 |
|
10 (0 × 0A) |
18 (0 × 12) |
26 (0 × 1A) |
34 (0 × 22) | |
|
Medium drop |
AF12 |
AF22 |
AF32 |
AF42 |
|
12 (0 × 0C) |
20 (0 × 14) |
28 (0 × 1C) |
36 (0 × 24) | |
|
High drop |
AF13 |
AF23 |
AF33 |
AF43 |
|
14 (0 × 0E) |
22 (0 × 16) |
30 (0 × 1E) |
38 (0 × 26) |
An alternative marking scheme is explained in RFC 1349. The use of the first three bits is identical with IntServ and DiffServ Class Selector PHBs. The next three bits are assigned to Minimize Delay, Maximize Throughput, and Maximize Reliability. A seventh bit, which ScreenOS does not support, is called Minimize Monetary Cost. Historically, some routing protocols were supporting those markings, making different routing decisions for different ToS settings, but this never enjoyed widespread commercial implementation. For instance, today’s Open Shortest Path First (OSPF) protocol implementation supports only one topology for Routine or Default PHB traffic. An IP router uses the same route for any packet, regardless of how it is marked. Marking has an effect only for scheduling a packet in an egress interface buffer, which is an independent function from determining which interface is the actual egress interface, and to which next hop the packet should be sent.
You can activate marking by simply enabling it on a policy, in which case marking is done automatically in accordance to traffic queue classification. This is more a convenience feature, as queuing and marking are two different functions. You enable marking with the keyword dscp, which stands for DiffServ Code Point:
set policy from "Trust" to "Untrust" "voip" "sip.juniper.de" "SIP" permit traffic priority 2 dscp enable
You can review or change the default mapping between queues and markings with the following:
FIREWALL-> get traffic-shaping ip_precedence
priority to IP precedence mapping:
priority 0 -> 7
priority 1 -> 6
priority 2 -> 5
priority 3 -> 4
priority 4 -> 3
priority 5 -> 2
priority 6 -> 1
priority 7 -> 0
set traffic-shaping ip_precedence 7 6 5 4 3 2 1 0Only decimal IntServ values can be configured. Starting with ScreenOS 5.2, you can make the default marking compatible with the DiffServ Class Selector PHB by configuring the next three bits to zero:
set traffic-shaping dscp-class-selector
Since ScreenOS 5.4, it is also possible to configure marking independent of priority queues to any of the DiffServ code points or any other six-bit custom scheme. You enter the format in decimals. Tables 14-1 and 14-2 provide the conversion from hexadecimal to decimal.
set policy from "Trust" to "Untrust" "voip" "sip.juniper.net" "SIP" permit traffic dscp enable value 18
As mentioned earlier, marking is maintained if packets were previously marked and marking was not configured on the policy. Marking is supported on non-ASIC and ASIC platforms. Tables 14-3, 14-4, and 14-5 detail some special considerations for VPN passthrough and for terminating VPN traffic on ASIC platforms.
|
Description |
Non-ASIC platforms |
ASIC platforms |
|
Clear packet |
No marking |
No marking |
|
Clear packet and marking in policy |
Mark packet |
Mark packet (Does not work for IPSec passthrough) |
|
Premarked packet with no marking in policy |
Retain marking |
Retain marking |
|
Premarked packet with marking in policy |
Mark packet |
Mark packet (Does not work for IPSec passthrough) |
There is no difference in marking for policy-based VPN traffic between the different platforms. Marking within the policy would affect only the outer header, visible to the network. Any premarking, if it exists, is retained in the inner header, as detailed in Table 14-4.
|
Description |
Non-ASIC platforms |
ASIC platforms |
|
Clear packet |
No marking |
No marking |
|
Clear packet with marking in policy |
Mark the outer header |
Mark the outer header |
|
Premarked packet with no marking in policy |
No marking in outer header; retain marking in inner header |
No marking in outer header; retain marking in inner header |
|
Premarked packet with marking in policy |
Mark outer header; inner header retains marking |
Mark outer header; inner header retains marking |
There are some differences between the platforms with route-based VPNs. On ASIC platforms, marking is done only on the inner header, whereas on non-ASIC platforms, the outer header is also marked, as detailed in Table 14-5.
|
Description |
Non-ASIC platforms |
ASIC platforms |
|
Clear packet |
No marking |
No marking |
|
Clear packet with marking in policy |
Mark the inner header |
Mark the inner header |
|
Premarked packet with no marking in policy |
Retain marking in the inner header and copy to the outer header |
Retain marking in the inner header |
|
Premarked packet with marking in policy |
Overwrite marking in the inner header and copy marking to the outer header |
Overwrite marking in the inner header |
On some high-end platforms, you also may have to enable marking on VPNs systemwide:
set envar ipsec-dscp-mark=yes
On ScreenOS 6.1 and later, marking can also be enabled on a VPN tunnel instead of on the policy:
set vpn vpn1 dscp-marking 52
You can check marking with the following:
islandgeeksfw->get policy id 1015name:"none" (id 1015), zone Trust -> Untrust,action Permit, status "enabled" src "voip", dst "sip.juniper.net", serv "SIP" Policies on this vpn tunnel: 0 nat off, Web filtering : disabled vpn unknown vpn, policy flag 00104000, session backup: on, idle reset: on traffic shapping on, scheduler n/a, serv flag 00 log no, log count 0, alert no, counter no(0) byte rate(sec/min) 0/0 total octets 0, counter(session/packet/octet) 0/0/0 priority 7,diffserv marking On dscp value: 18tadapter: state on, gbw/mbw 0/0 policing (no) No Authentication No User, User Group or Group expression set
14.5. Troubleshoot QoS
Problem
Traffic shaping is not doing what you thought you configured, and you need more background on how traffic shaping works on ScreenOS.
Solution
Check settings and bucket utilization on the policy:
get policy id 1009
Check settings and bucket utilization on the interface:
get traffic-shaping interface e3
Discussion
Traffic shaping is implemented in ScreenOS via a dual-token bucket model (see Figure 14-1). One bucket is used to control GBW, while tokens overspill into a second bucket for shared bandwidth in excess of all GBW. Tokens are added to the GBW bucket at the rate the GBW was configured. Any traffic falling within GBW takes tokens out of the GBW bucket. Any unused tokens overflow into a second bucket for shared bandwidth. If a packet is transmitted, it takes tokens in the amount of that traffic out of the GBW and shared buckets. Only if tokens are left in the buckets can a packet be transmitted. In ScreenOS, a token has the size of 1 byte.
The two buckets relate to the concept of one bucket measuring guarantees and another serving as a spillover for shared bandwidth. Each policy maintains a GBW and an MBW bucket, as shown in Figure 14-2, where both are used for measuring traffic on that policy. Only GBW buckets spill over to shared buckets.
Leftover bandwidth is shared among queues but also among dependent interfaces. Dependent interfaces are logical subinterfaces, or a virtual interface such as a tunnel or loopback interface. A tunnel interface can send traffic over a VLAN subinterface, which again is a child of a physical Ethernet interface. A loopback interface can represent multiple sub-or physical interfaces in a loopback group—for example, when a dynamic routing protocol would converge via alternate interfaces. Shared bandwidth therefore needs to be tracked not just for each priority, but also for each dependent interface. Shared bandwidth is tracked via TMNG, as depicted in Figure 14-3, where there are two TMNGs for each policy. Interfaces, and therefore TMNGs, are dependent on each other according to their child/parent relationship and their priority class. Each TMNG tracks GBW and MBW and shared bandwidth for the corresponding policy.
There is a shared bucket for each TMNG. A TMNG exists for each policy on each logical, virtual, or physical interface. Those shared buckets spill over like a waterfall in a certain order. Spillover means unused tokens are shared, and therefore, higher-priority traffic was not using up those tokens. Higher-queue TMNG spills over into lower-queue TMNG. TMNGs also spill over within their queue to a child interface, if it exists. This system ensures that unused tokens can be used by other flows in other queues and dependent interfaces.
Traffic shaping is configured and measured on a per-policy basis. But it is applied to outbound (or inbound) interface buffers in both directions of a traffic flow. This means that each policy has two TMNG objects associated with it. TMNGs are created on demand. Each interface automatically has a TMNG for the lowest-priority queue 7. Other TMNGs are created when policies exist that reference those priority queues.
TMNGs deal with shared bandwidth, and the use of tokens from a TMNG is called borrowing. A policy may or may not have GBW configured. If no guarantees have been defined, or guarantees are already satisfied, the policy needs to borrow from a shared pool of bandwidth. It will borrow, as long as it has tokens available, according to its priority queue and the configured MBW. The MBW, or PBW, puts a cap on borrowing, if so configured. There are some simple rules for this behavior:
The GBW of all policies going between the same interfaces cannot together be higher than the access rate of the root interface. For instance, all policies referencing a tunnel interface (via its “from” or “to” zone), using the same outgoing interface, cannot have a higher GBW than the outgoing interface can carry. Ideally, it should not have a higher rate than the network can carry.
GBW can never be higher than MBW.
If there is bandwidth left after satisfying all GBW to all policies going between the same interfaces, that bandwidth is shared among other policies.
There are eight strict priority queues per interface. Shared bandwidth is distributed among traffic flows according to their queue classification. Higher queues are served before lower queues.
Higher queues can starve lower queues. MBW is used to set a cap on queue use on a per-policy basis.
In other words, you can guarantee a certain bandwidth to a policy which can never be higher than what the interface can transmit or the network is able to transmit. You have to take into account that all guarantees may be served at the same time. Whatever is left after serving all guarantees is shared. Higher priorities are served before lower priorities. You can limit the bandwidth consumption of a policy by setting its MBW.
Let’s see how this relates to troubleshooting traffic shaping on ScreenOS.
You can review the default TMNG of each interface with the following:
FIREWALL-> get traffic-shaping interface e3
------------------------------------
ethernet3: physical bw=100000kbps, config gbw=2048kbps mbw = 2048kbps,
current bw=0kbps
total allocated gbw=208kbps
Shaping parameters
----------------------------------------------------------------------
tmng (7): interface ethernet3 state on priority 7
bw usage [for last one second]: 0 kbps
pak queue(cur/max): 0/26
pak received: 209094
pak dropped(out/shared): 0/0
PreShapingBytes (dropped/total): 0/26261329
diffserv-marking: 0x0
elapsed time: 65128055 ms
gbw/mbw: 1840/2048 (kbps)
gbw_q/mbw_q: 230/256
shared_tmng: 0
PostShapingBytes(total/borrowed):26261329/0
tokens (regular/borrowd): 0/27556895
token bucket (gbl/mbl): 128000/128000
tokens(gua/max): 128000/128000TMNGs are unique to policies, but each policy is also referencing shared TMNGs, which are the default TMNGs of the interface ingress and egress interfaces.
FIREWALL-> get pol id 1014
name:"none" (id 1014), zone Trust -> Untrust,action Permit, status
"enabled"
src "Any", dst "Any", serv "HTTP"
Policies on this vpn tunnel: 0
nat off, Web filtering : disabled
vpn unknown vpn, policy flag 00000000, session backup: on,
idle reset:on
traffic shapping on, scheduler n/a, serv flag 00
log no, log count 0, alert no, counter no(0) byte rate(sec/min) 0/0
total octets 132, counter(session/packet/octet) 0/0/0
priority 5, diffserv marking Off
tadapter: state on, gbw/mbw 512/1024 policing (no)
---------------------------------------------------------------------
tmng (21): interface ethernet1 state on priority 5
bw usage [for last one second]: 0 kbps
pak queue(cur/max): 0/21
pak received: 0
pak dropped(out/shared): 0/0
PreShapingBytes (dropped/total): 0/0
diffserv-marking: 0x0
elapsed time: 0 ms
gbw/mbw: 512/1024 (kbps)
gbw_q/mbw_q: 64/128
shared_tmng: 5
PostShapingBytes(total/borrowed):0/0
tokens (regular/borrowd): 0/0
token bucket (gbl/mbl): 64000/144000
tokens(gua/max): 64000/144000
----------------------------------------------------------------------
tmng (22): interface ethernet3 state on priority 5
bw usage [for last one second]: 0 kbps
pak queue(cur/max): 0/21
pak received: 2
pak dropped(out/shared): 0/0
PreShapingBytes (dropped/total): 0/132
diffserv-marking: 0x0
elapsed time: 2919 ms
gbw/mbw: 512/1024 (kbps)
gbw_q/mbw_q: 64/128
shared_tmng: 7
PostShapingBytes(total/borrowed):132/0
tokens (regular/borrowd): 0/132
token bucket (gbl/mbl): 64000/144000
tokens(gua/max): 64000/144000
No Authentication
No User, User Group or Group expression setYou can see how much GBW is being used and whether tokens are borrowed or packets are even dropped in the output from the interface and the policy get commands.
If the shared TMNG has no more tokens available, traffic will be dropped on the root interface:
FIREWALL-> get counter statistic interface e3 | incl buffer
in no buffer 0 | out no buffer 234 | re-xmt limit 0Table 14-6 shows the meaning of the different values in the TMNG object.
|
Value |
Description |
|
|
Bandwidth usage for the last second |
|
|
Queue size for the current and peak values, measured in packets |
|
|
Number of packets received since reset |
|
|
Packet drops when MBW was reached, or when not enough band width was available |
|
|
Bytes before traffic shaping was applied; dropped in total count |
|
|
Marking value (Differentiated Services Code Point [DSCP]) if enabled on the policy, in hex |
|
|
Time since last shaping |
|
|
Configured GBW and MBW |
|
|
Queue depth in packets |
|
|
Spillover TMNG |
|
|
Bytes after TS was applied (total bytes and borrowed bytes) |
|
|
Borrowed tokens in bytes |
|
|
GBW and MBW bucket size in bytes |
|
|
Available GBW and spillover tokens in bytes |
Take note of the difference between PreShapingBytes and PostShapingBytes. If a traffic stream should always stay within the GBW, you should never see borrowed tokens, but you will always see borrowed tokens when no GBW was configured. Configuring GBW is optional, but when you calculate GBW and MBW, take header overhead into account. Higher counters in the pak queue can create higher delays in traffic because a packet needs to wait before a higher-priority queue is cleared. If such a delay is a problem, reclassify traffic into a higher priority, or add bandwidth (remember that QoS cannot create additional bandwidth, but rather is an allocation scheme of existing bandwidth). pak dropped can provide insight into why packets are dropped: because MBW was reached, or because not enough shared bandwidth was available.
Most other values hold significance only for JTAC or Juniper Engineering because those provide further information about the internal state of traffic shaping in ScreenOS. Other than changing the priority and GBW and MBW, there are no tunable values on ScreenOS. Buffer size and queue depth cannot be changed.
Queuing and prioritization means packet reordering. You need to be careful and consider whether applications can tolerate reordered packets. It’s best if you classify all traffic belonging to the same application into the same priority queue. For example, classify VoIP control traffic and VoIP media traffic in the same queue to avoid reordering issues. As always, adding the bandwidth to your network that your applications need is much better than any fancy queuing scheme.