Forwarding Fundamentals
Table 2-1 provided an introductory glance at MPLS through terminology and definitions. This section goes into more depth about how all these concepts come together.
What Is a Label?
Labels, as you can probably guess, are an integral part of Multiprotocol Label Switching. The label allows the decoupling of routing from forwarding, which lets you do all sorts of neat things.
But what is a label? Before we define what a label is, you should know that MPLS can operate in one of two modes:
Frame mode
Cell mode
Frame Mode
Frame mode is the term used when you forward a packet with a label prepended to the packet in front of the Layer 3 header (the IP header, for example).
RFC 3031, “Multiprotocol Label Switching Architecture,” defines a label as “a short fixed length physically contiguous identifier which is used to identify a FEC, usually of local significance.”
Put simply, a label is a value prepended to a packet that tells the network where the packet should go. A label is a 20-bit value, which means that there can be 220 possible label values, or just over 1,000,000.
A packet can have multiple labels, carried in what's known as a label stack. A label stack is a set of one or more labels on a packet. At each hop in a network, only the outermost label is considered. The label that an LSR uses to forward the packet in the data plane is the label it assigned and distributed in the control plane. Hence, the inner labels have no meaning as far as the midpoints are concerned.
When labels are placed on a packet, the 20-bit label value itself is encoded with some additional pieces of information that assist in the forwarding of the labeled packet through a network.
Figure 2-1 illustrates the encoded MPLS header packet format.
Figure 2-1. MPLS Header Packet Format
This 32-bit quantity is known as a label stack entry, but is often referred to as just a label. So, when labels are discussed, the discussion could be either about the 20-bit label value or the 32-bit label stack entry. The additional 12 bits are made up of the following:
EXP— EXP bits are technically reserved for experimental use. Cisco IOS Software (and pretty much every MPLS implementation) uses these EXP bits to hold a QoS indicator—often a direct copy of the IP precedence bits in an underlying IP packet. When MPLS packets are queued up, it is possible to use EXP bits in the same way that IP precedence bits are used now. You'll read more about this in Chapter 6, “Quality of Service with MPLS TE.”
S— The S bit is the bottom-of-stack bit. It is possible (and common) to have more than one label attached to a packet. The bottommost label in a stack has the S bit set to 1; other labels have the S bit set to 0. The S bit is there because it's sometimes useful to know where the bottom of the label stack is, and the S bit is the tool to use to find it.
TTL— Time To Live bits are often (but not always) a direct copy of the IP TTL header. They are decremented at every hop to prevent routing loops from creating infinite packet storms; this is just like IP. TTL bits can also be set to something other than the TTL on the IP packet. This is most often used when a network operator wants to hide the underlying network topology from traceroutes from the outside world.
NOTE
In some cases, such as for security concerns or to meet service-level agreements (SLAs) (although this might come across as a deception), you might need to hide the core of a service provider's network from the user community. You can do this on Cisco routers using the command no mpls ip propagate-ttl {forwarded | local}. This command, when used with the forwarded option, affects only traffic forwarded through the router. This lets TTL be used in traceroute commands to troubleshoot problems in the core.
Cell Mode
Cell mode is the term used when you have a network consisting of ATM LSRs that use MPLS in the control plane to exchange VPI/VCI information instead of using ATM signalling.
In cell mode, the label is said to be encoded in a cell's VPI/VCI fields (see Figure 2-2). After label exchange is done in the control plane, in the forwarding plane, the ingress router segments the packet into ATM cells, applying the appropriate VPI/VCI value that was exchanged in the control plane, and transmits the cells. Midpoint ATM LSRs behave like normal ATM switches—they forward a cell based on the incoming VPI/VCI and the incoming port information. Finally, the egress router reassembles the cells into a packet.
Figure 2-2. MPLS Layer 2 Encapsulation
Cell mode is also called Label-Controlled ATM (LC-ATM). LC-ATM label distribution is discussed in more depth in the section “Label Distribution Concepts.” The cell-mode discussion was included for the sake of completeness. It is not required for understanding MPLS traffic engineering concepts in this book because MPLS TE is not supported in cell mode on Cisco routers as of this writing.
NOTE
In some of the examples containing MPLS-related output in this chapter, you'll notice that ATM VPI/VCI values show up in the outgoing tag column. These are cases in which a VPI/VCI was exchanged in an MPLS control plane over an ATM interface and the downstream neighbor on that interface expects to see that VPI/VCI value on the cell it receives.
ATM in Frame Mode and Cell Mode
As you have seen so far, ATM switches can act as LSRs. When ATM switches are a part of the core, they can operate in two modes:
Frame mode
Cell mode
When a conventional ATM PVC is built to achieve classic IP over ATM (aal5snap encapsulation, for example) and MPLS is sent over that PVC, this is still called frame-mode MPLS. To understand this better, refer to the MPLS header format, also known as the label stack entry, illustrated in Figure 2-1.
Figure 2-2 shows the MPLS label in relation to Layer 2 and Layer 3 headers. The PPP and LAN headers show the label being inserted between the Layer 2 and Layer 3 headers (Ethernet and IP, for example). This is called a shim header. When operating in frame-mode MPLS, you always see a shim header. This is also applicable when you are simply connecting routers over ATM PVCs and doing MPLS in a classic IP-over-ATM environment.
When running in cell mode, ATM LSRs act as routers in the control plane. In other words, they need to exchange routing information through IGP protocols, such as OSPF, and need to run a label distribution protocol, such as TDP or LDP.
NOTE
You might think that ATM switches forward only ATM cells, so whenever ATM switches are involved in the MPLS core, they must be acting as ATM LSRs, in cell mode. This is not true. The reason this is not always true is because the ATM switch could be used to build a conventional ATM point-to-point PVC between two routers. When this is done, the routers on either end of the PVC can be directly connected LSRs. When forwarding packets to each other, they would first have to build the IP packet and insert an MPLS header in front of it and then segment the entire packet (IP packet plus MPLS header) into ATM cells. When these cells reach the router at the other end of the PVC, they are reassembled into a packet. If further forwarding is required, the forwarding is based on the label value inside the label header. In this case, even though the MPLS packets were segmented into ATM cells, there was no mapping of MPLS label to the VPI/VCI fields of the ATM cell. Thus, this would be considered frame mode.
Control Plane Versus Data Plane
The control plane is where the routing information and other control information, such as label bindings, are exchanged between LSRs. MPLS is a control plane-driven protocol, meaning that the control information exchange must be in place before the first data packet can be forwarded. The forwarding of data packets is done in the data plane.
Classification
When an IP packet arrives at a LER (the ingress router), just as in the case of normal IP forwarding, a longest-match lookup is performed by comparing the entries in the FIB against the destination IP address of the received packet. In MPLS terminology, this process is called classifying the packet. This section explains the term FEC (Forwarding Equivalence Class), as well as where classification is performed and how it differs from classification in conventional IP networks.
FEC
When IP packets destined for the same subnet arrive at an ingress router, the classification for all these packets is the same—it is based on the longest-match lookup in the FIB. For example, assume you have an entry in the FIB for 171.68.0.0/16 with a next-hop address of 12.12.12.12. If you now receive two packets with destination IP addresses 171.68.1.1 and 171.68.23.5, both these packets are forwarded to the same next hop—12.12.12.12. In most cases, it could be said that 171.68.1.1 and 171.68.23.5 share the same FEC.
However, the classification into a particular FEC need not be restricted to the destination IP address of the received packet. Classification into a FEC could be based on the interface on which the packet arrived, the IP precedence values in the packet's IP header, the packet's destination port number, or any arbitrary scheme you can imagine. Regardless of the basis of the classification, all the packets that are classified into the same FEC receive the same treatment. This treatment can be forwarding the packet down a certain path, providing the packet some preferential treatment within the core, or even dropping the packet. The current Cisco IOS Software implementation classifies IP packets based on their destination IP address, in the absence of any tools such as policy-based routing.
Translating MPLS terminology into IP terminology, the FEC is nothing but the route (also called the prefix) found in the FIB that was the best match for the incoming packet.
Living on the Edge
In conventional IP networks, the forwarding of a packet is based on the packet's destination IP address. Each node along the packet's path can forward the packet only after examining the destination IP address contained in the packet. This means that each node along the packet's path classifies the packet. This is discussed in further detail in the section “MPLS Versus IP.”
In MPLS-based forwarding, after the ingress LER at the edge of the network does the classification, it pushes a label on the data packet that matches that packet's FEC. This process is called label imposition or label pushing. The LSRs in the network's core are not required to reclassify the packet. When a core router receives a labeled packet, it does three things:
It does a label lookup on the incoming label.
It finds the outgoing interface and outgoing label for this packet.
It swaps the received (incoming) label for the proper outgoing label and sends the packet out the outgoing interface.
This process is known as label swapping.
How an LSR knows what label the downstream LSR expects is based on the label bindings that are exchanged in the control plane using a label distribution protocol (LDP, RSVP, BGP, and so on) prior to forwarding packets.
When a packet reaches the end of the network, the packet's outermost label is removed, and the remainder of the packet is forwarded to the next hop. The act of removing a label from a packet is called label popping or label disposition.
The three fundamental label operations (push/impose, swap, and pop/dispose) are all that is needed for MPLS. Label imposition/disposition and forwarding allow for an arbitrarily complex classification scheme that needs higher processing power to be enforced at the edge, while keeping the core simply forwarding MPLS packets.
Control Planes in an MPLS Network
This section looks at the processes needed to get a packet through a network—first an IP network and then an MPLS network.
For both the IP network and the MPLS network, consider the topology shown in Figure 2-3. It represents a service provider network in which gateway routers 7200a and 7200b peer with external BGP peers 7500a and 12008c. The core routers in AS1 (12008a and 12008b) are only involved in IBGP peering.
Figure 2-3. Packet Life: Both IP and MPLS
In order for any data packets to be passed through this network, first the control plane mechanisms have to be set up.
In an IP network, the control plane mechanisms consist of the following:
Interior Gateway Protocol (IGP)— Most often OSPF or IS-IS in service provider networks. Can also be EIGRP, RIP, or just static routing.
Border Gateway Protocol (BGP)— Used to advertise routes that are learned from external neighbors. External BGP (EBGP) is spoken between 7200b and 12008c, as shown in Figure 2-3. 7200b then communicates what it has learned to all other routers in AS1. In this example, 7200b has all other routers as IBGP neighbors; in real-life networks, a Route Reflector (RR) would probably be used. The important point here is that all the routers in AS1 need to learn the route from 7200b.
In an MPLS network, the control plane mechanisms are as follows:
IGP— This is no different from the IGP used for an IP-only network. If the MPLS network were using traffic engineering, the IGP would have to be a link-state protocol, either OSPF or IS-IS. Because traffic engineering is not being considered in this example, the IGP doesn't matter.
Label distribution protocol— The three principal label distribution protocols in an MPLS network are
- Tag Distribution Protocol (TDP)
- Label Distribution Protocol (LDP)
- RSVP
RSVP is used for traffic engineering and is not considered in this example. TDP and LDP are actually two different versions of the same thing; TDP is older, and LDP is standardized. So, assume that LDP is used to distribute labels.
What exactly does label distribution mean? A label binding is an association of a label to a prefix (route). LDP works in conjunction with the IGP to advertise label bindings for all non-BGP routes to its neighbors. LDP neighbors are established over links enabled for LDP. So, when 12008a and 12008b in Figure 2-3 become LDP neighbors, they advertise labels for their IGP-learned routes to each other, but not the BGP routes learned from 7200b.
BGP— Here's where the key difference is between MPLS and non-MPLS networks. Instead of needing to put BGP on every router, BGP is needed only at the edges of the network. Instead of 7200b having three BGP peers (7200a, 12008a, 12008b), it has only one—7200a.
Why is BGP unnecessary in the core? Because an ingress LER, which has to have full BGP routes, knows the next hop for all BGP-learned routes. A label is put on the packet that corresponds to a packet's BGP next hop, and the packet is delivered across the network to that next hop using MPLS. The section “MPLS Versus IP” deals with this issue in great detail.
Scaling issues because of large IBGP meshing can be solved using route reflectors or confederations; BGP scales well when deployed properly. However, some people like to totally avoid running BGP in the core. Route flaps outside the network can lead to instability in the core, and the fewer BGP speakers you have, the less you have to manage. In certain cases, the core routers might still need to run BGP for other reasons, such as for multicast.
Forwarding Mechanics
This section explains the differences between forwarding a packet in an IP network and forwarding a packet in an MPLS network. A sample service provider network is used to clarify this concept. So far, you have read about FIB and its role in forwarding packets in a Cisco router. This section covers the role of FIB, LIB, and LFIB tables in forwarding packets in an MPLS-enabled network.
MPLS Versus IP
RFC 3031 defines the MPLS architecture. The points where MPLS forwarding deviates from IP forwarding are as follows:
IP forwarding is based on the destination IP address and the FIB.
MPLS forwarding is based on the MPLS label and the Label Forwarding Information Base (LFIB).
Both MPLS and IP forwarding are done hop-by-hop. IP forwarding involves packet classification at every hop, whereas in MPLS forwarding, the classification is done only by the ingress LSR.
Figure 2-4 illustrates a typical ISP backbone, in which external routes are learned through EBGP and are distributed to the core routers through full IBGP mesh. (Route reflectors or confederations are used in larger cores where a full IBGP mesh would not scale.) The route 171.68.0.0/16 is learned from an external peer by gateway router 7200b. All other routers in the core learn about this route through IBGP. Also, the core routers know how to reach each other from routes learned over IGP routing protocols, such as OSPF or IS-IS.
Figure 2-4. Forwarding Table on Ingress Router 7200a
NOTE
Although OSPF and IS-IS seem to be the choice of IGP routing protocols in the service provider backbone, MPLS forwarding doesn't care what your IGP is. For traffic engineering, you need to run IS-IS or OSPF (see Chapter 3, “Information Distribution,” for details on why), but if you're not using traffic engineering, you can use any IGP you want.
In Figure 2-4, 7200a is the ingress router that receives packets destined for network 172.68.0.0. Example 2-1 shows the output that displays the contents of the routing table (RIB) on 7200a. As you can see, the entry for 172.68.0.0/16 is the external route that 7200a learned through IBGP.
NOTE
You know that 172.168.0.0 is an IBGP-learned route, not an EBGP-learned route because the administrative distance field in the table shows 200, which indicates that it is an IBGP-learned route, not an EBGP-learned route, whose administrative distance is 20.
Example 2-1. Router 7200a Routing Table
7200a#show ip route Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area * - candidate default, U - per-user static route, o - ODR Gateway of last resort is 7.1.5.1 to network 0.0.0.0 B 171.68.0.0/16 [200/0] via 12.12.12.12, 01:10:44 3.0.0.0/32 is subnetted, 1 subnets |
When it comes to actually forwarding a data packet, 7200a consults the FIB that is built using the routing table. Example 2-2 shows the FIB entry for 171.68.0.0/16 on 7200a.
Example 2-2. FIB Entry for 171.68.0.0/16 on 7200a
7200a#show ip cef 171.68.0.0
171.68.0.0/16, version 69, cached adjacency to POS3/0
0 packets, 0 bytes, wccp tag 139
via 12.12.12.12, 0 dependencies, recursive
next hop 10.0.3.5, POS3/0 via 12.12.12.12/32
valid cached adjacency
|
Now that you have examined the contents of the RIB and FIB on 7200a, you know that the control information has been exchanged and that 7200a is ready to forward data destined for network 171.68.0.0. Similarly, the forwarding tables are created on each of the routers in the core (12008a and 12008b in this example). Next, you need to know how forwarding works step by step.
Consider an IP packet with a destination IP address of 171.68.1.1 arriving at the ingress router (7200a in Figure 2-4). When the packet arrives at the ingress port, the router consults the FIB. The destination IP address of 171.68.1.1 is compared to the FIB entries, and the longest-match entry is selected. As a result of this operation, the ingress router (7200a) knows that the packet has to eventually reach 7200b (the egress router) to exit the network. This usually involves forwarding the packet to one of the immediately connected neighbors, which in this case happens to be 12008a. This process is repeated until the packet reaches the exit point. Figure 2-5 shows the next-hop router 12008a consulting the FIB in order to forward the packet. Note that 12008a has two outbound interface entries in the forwarding table, resulting in load sharing.
Figure 2-5. Forwarding Table on Core Router 12008a
|
Because you are so accustomed to IP forwarding, you might take for granted how the packet reaches the network's exit point. On closer observation, this process of consulting the FIB for a longest match and mapping each set of destinations to a next-hop router happens on every router in the forwarding path. |
Now take a look at Figure 2-6. MPLS forwarding is now turned on in the core. IBGP is now only between gateway routers and need not be run on the core routers. Right after the IGP converges, the loopback address of 7200b 12.12.12.12 has been learned by 7200a. At this time, LDP also converges. As a result, 7200a receives a label of 12323, corresponding to 7200b's loopback address 12.12.12.12 from 12008a. 12008a itself has received a similar label of 12324 from 12008b. 12008b has received a label of POP from 7200b because it is the penultimate hop router and is responsible for removing the top-level label.
Figure 2-6. Internet Service Provider Backbone with MPLS Forwarding Enabled
NOTE
Labels distributed by TDP/LDP and RSVP are, in most cases, link-local—meaning it is between any two neighbors and not flooded like OSPF or ISIS. This means that the label value 12323 distributed by 12008a that maps to 12.12.12.12/32 has no relation to the label value 12324 that's received by 12008a, other than the fact that 12008a associated the incoming label value 12323 with the outgoing label value 12324. In other words, 12000b could have given 12008a the label value 42, 967, or 41243, and 12008a still could have distributed the label value 12323.
Next, focus your attention on the data plane. Consider the data packet destined for 171.68.1.1 entering the network at 7200a. 7200a still consults the FIB table because the incoming packet is an IP packet. The difference this time is that 7200a is responsible for label imposition. The longest-match IP lookup in the FIB table occurs. As in the case when MPLS forwarding was not turned on in the core, 7200a concludes that the packet needs to eventually reach 7200b—the exit point for this packet. However, now the FIB table has an entry for the label to be imposed for packets destined for 7200b. This is the value of the Out Label column in Figure 2-6, which happens to be 12323. Example 2-3 shows the FIB table on 7200a. If you focus your attention on the highlighted portion of the output, you'll notice the tags imposed field that is now present after MPLS forwarding was enabled in the core. This means that if 7200a receives either an IP packet that needs to be forwarded to 12.12.12.12 or an MPLS packet that has a label value of 36, 7200a switches that packet out as an MPLS packet on POS 3/0 with a label value of 12323.
Example 2-3. FIB Entry for 171.68.0.0 on 7200a After MPLS Forwarding Has Been Turned On
7200a#show ip cef 171.68.0.0 detail 171.68.0.0/16, version 1934, cached adjacency to POS3/0 0 packets, 0 bytes tag information from 12.12.12.12/32, shared local tag: 36 fast tag rewrite with PO3/0, point2point, tags imposed {12323} via 12.12.12.12, 0 dependencies, recursive next hop 10.0.3.5, POS3/0 via 12.12.12.12/32 valid cached adjacency tag rewrite with PO3/0, point2point, tags imposed {12323} |
Figure 2-7 shows the packet as it enters 12008a, a core router. The packet is an MPLS packet with a label of 12323.
Figure 2-7. LFIB on Core Router 12008a
|
People who are new to MPLS often wonder how a router knows that this is an MPLS packet and not an IP packet. If you ask yourself how a router knows an IP packet from an IPX packet, you have answered the question. The Layer 2 encapsulation that precedes the IP or IPX header contains a protocol type field. In LAN environments, this is ethertype. For PPP encapsulation, the Network Control Protocol (NCP) identified what type of Layer 3 packet was being carried, and so on. For MPLS packets, new ethertypes and NCPs have been defined. They are listed in Table 2-2. |
| Encapsulation | Value (in Hexadecimal) |
|---|---|
| MPLS Control Packet (MPLSCP) for PPP | 0x8281 |
| PPP Unicast | 0x0281 |
| PPP Multicast | 0x0283 |
| LAN Unicast | 0x8847 |
| LAN Multicast | 0x8848 |
12008a no longer needs to look at the Layer 3 IP address. It simply consults the LFIB table and knows that there are two places to send an incoming MPLS packet with a label of 12323—POS 1/0 with a label of 12324, and ATM 3/0.1 on the VPI/VCI 1/49. Why are there two paths for this label? Because two equal-cost routes to the destination exist—one via POS and one via ATM. 12008a forwards the packet down POS 1/0 using frame mode. For the path down the ATM 3/0.1 interface, 12008a segments the packet into ATM cells, with each cell using a VPI/VCI value of 1/49. The following discussion focuses on forwarding the packet down the POS 1/0 interface in frame mode. The interesting aspects of cell-mode MPLS are discussed in the section “Label Distribution Concepts.” In this chapter, the term label, when used in the context of cell-mode MPLS, refers to ATM VPI/VCI.
When the packet with label 12324 enters 12008b on POS1/1, it goes through the same exercise 12008a went through and consults the LFIB table, as shown in Figure 2-7. But because 12008b is the penultimate-hop router and has received a label of POP from 7200b, it removes the label of 12324, exposes the IP header, and forwards the packet to 7200b. It is important to note that all along, the packet's destination IP address was 171.68.1.1, for which neither 12008a nor 12008b had a RIB/FIB entry after BGP was removed from their configs. When the packet enters router 7200b, because the packet is IP, again the FIB is consulted. Because 7200b is a gateway (edge) router, it is running BGP and has learned 171.68.0.0/16 over the EBGP connection. Therefore, it can forward the packet.
Example 2-4 shows the LFIB table that router 12008a uses to forward labeled packets. To forward packets to the 171.68.0.0 network, packets need to be sent to 12.12.12.12 (7200b)—the egress router. Upstream routers, such as 7200a, impose a label 12323 that corresponds to the next-hop address 12.12.12.12 (7200b). Notice, in the highlighted part of Example 2-4, that label 12323 falls under the Local column because it was what 12008a assigned for FEC 12.12.12.12 and it distributed this label to 7200a—its upstream neighbor.
Example 2-4. Displaying 12008a's LFIB Table on the Router
12008a#show mpls forwarding Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 12318 Pop tag 10.0.57.0/24 0 PO1/0 point2point 1/43 10.0.57.0/24 0 AT3/0.1 point2point 12319 12320 10.0.86.0/24 0 PO1/0 point2point 1/44 10.0.86.0/24 0 AT3/0.1 point2point 12320 12321 10.1.1.1/32 0 PO1/0 point2point 1/45 10.1.1.1/32 0 AT3/0.1 point2point 12321 12322 10.1.1.2/32 0 PO1/0 point2point 1/46 10.1.1.2/32 0 AT3/0.1 point2point 12322 12326 16.16.16.16/32 0 PO1/0 point2point 1/51 16.16.16.16/32 0 AT3/0.1 point2point 12323 12324 12.12.12.12/32 575 PO1/0 point2point 1/49 12.12.12.12/32 0 AT3/0.1 point2point 12324 12325 13.13.13.13/32 0 PO1/0 point2point 1/50 13.13.13.13/32 0 AT3/0.1 point2point 12325 12327 17.17.17.17/32 144 PO1/0 point2point |
As the output in Example 2-4 shows, 12008a has two ways of reaching 12.12.12.12 (the loopback interface of 7200b).
With MPLS forwarding, just as in IP forwarding, the CEF table (the same as FIB) is consulted. If there are multiple outbound links to the next hop, load sharing is possible, as demonstrated in Example 2-5. The highlighted portion shows that 12008a is doing per-destination load sharing.
Example 2-5. Router 12008a's CEF Table Shows Load Sharing for Labeled Packets
12008a#show ip cef 12.12.12.12 internal 12.12.12.12/32, version 385, per-destination sharing 0 packets, 0 bytes tag information set, shared local tag: 12323 via 10.0.5.11, POS1/0, 0 dependencies traffic share 1 next hop 10.0.5.11, POS1/0 unresolved valid adjacency tag rewrite with PO1/0, point2point, tags imposed {12324} via 10.0.4.11, ATM3/0.1, 1 dependency traffic share 1 next hop 10.0.4.11, ATM3/0.1 unresolved valid adjacency tag rewrite with AT3/0.1, point2point, tags imposed {1/49(vcd=65)} 0 packets, 0 bytes switched through the prefix tmstats: external 0 packets, 0 bytes internal 0 packets, 0 bytes Load distribution: 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 (refcount 2) Hash OK Interface Address Packets Tags imposed 1 Y POS1/0 point2point 0 {12324} 2 Y ATM3/0.1 point2point 0 {1/49} 3 Y POS1/0 point2point 0 {12324} 4 Y ATM3/0.1 point2point 0 {1/49} 5 Y POS1/0 point2point 0 {12324} 6 Y ATM3/0.1 point2point 0 {1/49} 7 Y POS1/0 point2point 0 {12324} 8 Y ATM3/0.1 point2point 0 {1/49} 9 Y POS1/0 point2point 0 {12324} 10 Y ATM3/0.1 point2point 0 {1/49} 11 Y POS1/0 point2point 0 {12324} 12 Y ATM3/0.1 point2point 0 {1/49} 13 Y POS1/0 point2point 0 {12324} 14 Y ATM3/0.1 point2point 0 {1/49} 15 Y POS1/0 point2point 0 {12324} 16 Y ATM3/0.1 point2point 0 {1/49} refcount 5 |
As shown in Example 2-5, packets destined for 12.12.12.12 on 12008a are load shared on the two outbound links. The load sharing is CEF's standard per-source-destination by default. This means that the packet's source and destination IP addresses are hashed. As a result, it uses one of the 16 buckets. You can also turn on per-packet load sharing by configuring the outbound interfaces in question.
FIB, LIB, and LFIB and Their Roles in MPLS Forwarding
If you are wondering how the FIB, LIB, and LFIB tables relate to each other, this section summarizes the roles of each of these tables and how they are populated.
The FIB table knows only about IP packets and therefore is consulted only when the incoming packet is an IP packet. Although the incoming packet is an IP packet, the outgoing packet might not be! If one or more label bindings have been received for the packet's destination, the packet is MPLS forwarded. Looking at the CEF table entry for destination 12.12.12.12 on 12008a (as in Example 2-5) tells you whether the outgoing packet is an IP packet or an MPLS packet. If it has an entry of tags imposed against the CEF entry, the outgoing packet is MPLS.
In the case of 7200a, destination 12.12.12.12 has an outgoing label of 12323. This results in the packet's entering the next-hop router—12008a with an MPLS label on it. This time, the LFIB table is consulted on 12008a. Example 2-6 shows that if an MPLS packet came in with a label of 12323, it would have to be switched out of the ATM 3/0.1 interface with a VPI/VCI value of 1/49 or with a label of 12324 on interface POS1/0.
Example 2-6 shows a segment of the LFIB table corresponding to 12.12.12.12.
Example 2-6. Segment of 12008a's LFIB Table Corresponding to 12.12.12.12
Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 12323 12324 12.12.12.12/32 575 PO1/0 point2point 1/49 12.12.12.12/32 0 AT3/0.1 point2point |
Now consider the case of the packet being switched over the POS link with an MPLS label of 12324.
Where did all these labels come from? Labels can be distributed between LSRs using various methods. If LDP or TDP protocols are used, label bindings are exchanged between LSRs and their neighbors. This information is stored in the LIB. You can view the LIB's contents using the show mpls ip bindings address command. You can see in Example 2-7 the contents of the LIB that holds the label bindings for 12.12.12.12.
Example 2-7. Viewing LIB Contents
12008a#show mpls ip binding 12.12.12.12 32
12.12.12.12/32
in label: 12325
out label: 36 lsr: 4.4.4.4:0
out label: 12324 lsr: 11.11.11.11:0
out label: 37 lsr: 3.3.3.3:0
out vc label: 1/49 lsr: 11.11.11.11:2 ATM3/0.1
Active ingress 1 hop (vcd 18)
|
Notice in Example 2-7 that several remote bindings exist in the LIB, but the forwarding table shows only two entries. This is because only the bindings that are received from the current IGP next-hop router are used, even though all the bindings are retained on the Cisco routers because they employ the liberal retention mode (discussed further in the section “Liberal and Conservative Retention Modes”). Look at 12008a's routing entry for 12.12.12.12 in Example 2-8.
Example 2-8. RIB Entry for 12.12.12.12 on 12008a
12008a#show ip route 12.12.12.12 Routing entry for 12.12.12.12/32 Known via "ospf 100", distance 110, metric 3, type intra area Last update from 10.0.4.11 on ATM3/0.1, 00:41:50 ago Routing Descriptor Blocks: * 10.0.5.11, from 12.12.12.12, 00:41:50 ago, via POS1/0 Route metric is 3, traffic share count is 1 10.0.4.11, from 12.12.12.12, 00:41:50 ago, via ATM3/0.1 Route metric is 3, traffic share count is 1 Route metric is 3, traffic share count is 1 |
To figure out exactly how labels 12324 and 1/49 became the outgoing labels for 12.12.12.12, as shown in Example 2-6, you have to first look at the next hops for 12.12.12.12 from Example 2-8. They happen to be 10.0.5.11 and 10.0.4.11 (which are highlighted in Example 2-8). Incidentally, these next hops happen to be two links of 12008b, whose router ID is 11.11.11.11. Using this information, you can go back to the LIB (refer to Example 2-7) to look for what label bindings 12008a received from 11.11.11.11. You'll find labels 12324 and 1/49 over the two interfaces POS 1/0 and ATM 3/0.1, respectively.
Table 2-3 summarizes the input and output packet types and the table used for forwarding.
| Packet Type | Table Used for Packet Lookup | How to Look at This Table |
|---|---|---|
| IP to IP | FIB | show ip cef |
| IP to MPLS | FIB | show ip cef |
| MPLS to MPLS | LFIB | show mpls forwarding-table |
| MPLS to IP | LFIB | show mpls forwarding-table |
Label Distribution Concepts
The preceding section elaborated on how forwarding works after the FIB and LFIB tables have been populated. This section covers the various methods of distributing label bindings.
When labels are distributed, what's actually distributed is a label, an IP prefix, and a mask length. Generally, this entire process is called label distribution rather than label, prefix, and mask distribution.
To understand how LSRs generate and distribute labels, you need to understand some terminology introduced in RFC 3031.
Ordered Versus Independent Control
As far as generating labels is concerned, regardless of what control method is applicable, the LSRs generate labels independently and have no relation to the received labels. As you would anticipate, there have to be reserved label values that are either used for control or have some special meaning.
The label values 0 to 15 are reserved. This means that the lowest label number you see that maps to an IP prefix is 16. Because the label space is 20 bits, the highest label you ever see advertised is 220–1, or 1,048,575. This is subject to change, though. As long as an allocated label value is between 16 and 1,048,575, it's legal.
Only four out of the 16 reserved label values are currently defined in RFC 3032, “MPLS Label Stack Encoding”:
Except for MPLS edge applications, labels are generated only for IGP-learned prefixes (including static routes) in the routing. Why aren't labels allocated for BGP-learned routes? Because doing so is completely unnecessary. Again, for IPv4 routes (the non-MPLS-VPN case), if the egress LER set next-hop-self in BGP, all that is needed is a label and an IGP route for the next hop of the BGP-learned route. For example, consider the external route 171.68.0.0 that is learned by 7200b in Figure 2-6. By doing next-hop-self, 7200b sets the next hop for 171.68.0.0 to 12.12.12.12 (7200b's BGP router ID and Loopback0) before it advertises 171.68.0.0 to 7200a with IBGP. Because IGP routes have been exchanged and label distribution has occurred, 7200a has a label for 12.12.12.12. If it uses this label for packets destined for 171.68.0.0, the packet is delivered to 7200b as a result of MPLS forwarding. There is no need for a label for 171.68.0.0. Any packets destined for 171.68.0.0 are simply delivered to 12.12.12.12, which then routes the IP packet normally.
After LSRs have generated these labels that are associated with IP prefixes, two ways exist to distribute them:
Ordered LSP control mode— The LSR waits to receive bindings from its downstream neighbors before sending the labels it generated to its upstream neighbors.
Independent LSP control mode— LSRs are free to distribute the label bindings to all their neighbors (downstream and upstream) without waiting to receive bindings from their downstream neighbors.
LDP uses independent LSP control mode in frame-based networks and ordered LSP control mode in cell-mode networks; RSVP uses ordered LSP control mode.
Unsolicited Downstream Versus Downstream-on-Demand
In unsolicited label distribution, a router advertises labels for all prefixes in its IGP to all neighbors, whether or not they ask for them. Because the label advertisements are not used to make any routing decisions, it's OK for a router to send labels for all prefixes it knows about to all neighbors, even if those neighbors are not using the advertising router as the next hop for a route. LDP is not a routing protocol; it depends on the routing protocol to make loop-free routing decisions (although there are some loop detection and loop prevention measures in LDP). Figure 2-8 shows how routers 12008a and 12008b look up their RIB and assign labels for all IGP-derived routes. They then distribute these routes to their neighbors in an unsolicited manner.
Figure 2-8. Unsolicited Downstream Distribution of Labels Destined for IGP-Derived Routes
Unsolicited mode works fine in most cases. However, with label distribution over ATM, multiple labels might need to be distributed for the same destination IP address/mask, depending on the number of neighbors requiring these labels. This happens when an ATM switch cannot support something called VC Merge (discussed in the later section “VC Merge”). For now, just realize that rather than one label per destination prefix, an ATM LSR might need to distribute one label per destination prefix, multiplied by the number of neighbors. If an ATM LSR has 500 routes and 12 neighbors, it would have to distribute 500 * 12 = 6,000 labels. Remember that a label in cell mode is a VPI/VCI. Because of this, the VPI/VCI space can be depleted quickly on platforms that have limited VPI/VCI space. Therefore, ATM switches do not hand out labels for prefixes unless they are asked for labels for those prefixes. This is known as Downstream-on-Demand (DoD) label distribution.
With DoD label distribution, an LSR determines if it needs to obtain a label for a prefix. After this determination has been made, the LSR asks its next hop (downstream neighbor) for a label. The downstream neighbor, even though it might have generated a label for the prefix in question, abstains from sending its label bindings until it has received such a binding from its downstream neighbor. This process continues until the penultimate-hop LSR requests the LER (the egress LSR for this LSP) for a label binding. After this happens, label bindings flow from downstream to upstream LSRs—hence the term downstream-on-demand. Figure 2-9 illustrates an example of DoD label distribution.
Figure 2-9. DoD and ATM
In Figure 2-9, the ingress router 7200a sends a label request for 12.12.12.12 to the ATM LSR ls1010a. ls1010a then sends a request to the egress LSR (router) 7200b on the right. The egress LSR replies with a label mapping with a label value of 1/130. After ls1010a receives this binding, it sends a label of 1/125 (label mapping) to 7200a. When an IP packet enters 7200a, it is segmented into ATM cells with a VPI/VCI value of 1/125 (exchanged in the MPLS control plane) and is sent to ls1010a. ls1010a swaps labels (VPI/VCIs) and switches out the cells with label values of 1/130.
The label range (VPI/VCI) and the VPI that each ATM LSR uses can be configured.
NOTE
There's no such thing as “MPLS cells” in an ATM network. ATM-LSRs don't distinguish MPLS-created ATM VPI/VCIs from those created with other ATM signalling protocols. To the ATM switching hardware, incoming cells are just ATM cells and are treated as such. If an incoming cell has a certain VPI/VCI value, it is switched out on a different port with a particular VPI/VCI value. The only place where a distinction is made between MPLS-derived ATM VPI/VCIs and VPI/VCIs derived from ATM signalling protocols is at the control plane level.
Liberal and Conservative Label Retention Modes
When LSRs receive label bindings from neighbors that are not routing next hops, they might choose to keep the bindings for future use in case the LSRs that sent these bindings become next hops. This approach is called liberal label retention mode. When LSRs choose to discard label bindings that are not currently useful, this is called conservative label retention mode. Liberal label retention mode uses more space in the LIB table but expedites convergence. This is because you don't have to wait for label bindings to arrive after routing converges in case of a link failure to an LSR that was previously the next hop. In conservative label retention mode, the opposite is true. This mode saves space in the LIB but suffers from slower convergence. Current Cisco IOS Software routers use liberal label retention mode.
Label Space
LSRs can generate labels that have either global space (sometimes known as per-platform space) or interface space. Sometimes, the word scope is substituted for space, so global label space and global label scope are interchangeable terms. Global label space is when a router generates a label that's unique to a particular destination FEC, but that is not associated with any particular incoming interface. An LSR could also choose to generate labels that are unique only to the interface or port used to send the label binding to the upstream node. This is called per-interface label space. Per-interface label space is not a problem as long as an LSR can differentiate between packets coming in on one interface and packets coming in on another. Cisco routers allocate labels from per-platform label space for frame-mode and per-interface label space for cell-mode; ATM switches use per-interface label space so as to conserve VCI/VPI space.
Keeping track of these concepts and in which environment they apply can be tricky. Understanding the label distribution methods is simple when you understand the underlying protocols, though. Table 2-4 summarizes this information.
| Label Distribution Method | Control | Distribution | Retention | Label Space |
|---|---|---|---|---|
| Frame Mode TDP and LDP | Unordered | Downstream unsolicited | Liberal | Per platform |
| Cell Mode (LC-ATM) TDP and LDP | Ordered | Downstream-on-Demand | Conservative | Per interface |
| RSVP-TE (Frame Mode Only) | Ordered | Downstream-on-Demand | Conservative | Per platform |
Cell Mode Idiosyncrasies
Certain restrictions apply in cell-mode MPLS:
The ATM header doesn't carry the TTL field.
PHP is not supported.
No route summarization can be done on ATM LSRs.
You might wonder how permanent packet loops are prevented without the use of TTLs when ATM LSRs are used in the core. The two pieces to solving this problem are
A feature of LDP known as the hop count TLV is used in LSP signalling with LDP in cell-mode MPLS networks. Because ATM LSRs don't have a way to decrement the packet TTL at every hop, the router that is an ingress to a cell-mode MPLS cloud must decrement the packet TTL by the number of hops in the cell-mode cloud.
A feature of LDP known as the path vector TLV contains a list of all hops an LSP traverses within a cell-mode cloud. The path vector can be used for loop detection in the control plane. For example, if the same LSR shows up twice in the path vector for a particular LSP, the LSP must be looping and must not be used.
NOTE
LDP path vector and hop count TLV are not used in frame-mode networks because the TTL in a label is decremented at every hop, just like an IP header. When the TTL value reaches 0, the packet is no longer forwarded.
If the ingress LSR determines that the packet would not make it through the core, it is the ingress LER's responsibility to send an “ICMP unreachable TTL expired” message to the source of the data packet.
PHP is not supported on ATM LSRs because it would require the ATM LSR to send out a packet without a top label on it, which would mean sending out the cells without a VPI/VCI. This makes no sense in an ATM environment.
Even though an ATM LSR is required to run a routing protocol in the control plane, it should not perform route summarization. The reason for this is that the LSR performing route summarization would be required to do an IP lookup. Although an ATM LSR is capable of doing an IP lookup, this would require reassembling a packet's cells back into an entire packet. This turns an ATM switch into a router, which is not what an ATM switch is best at.
When operating in cell mode, ATM LSRs set up a control VC (VCI/VPI 0/32 by default) to exchange all control information.
Example 2-9 shows output from a router that is speaking cell mode to an ATM switch.
Example 2-9. ATM VC Table on a Router Operating in Cell Mode
gsr4#show atm vc VCD / Peak Avg/Min Burst Interface Name VPI VCI Type Encaps Kbps Kbps Cells Sts 5/0.1 1 0 32 PVC SNAP 622000 622000 UP 5/0.1 84 1 61 TVC MUX 622000 622000 UP 5/0.1 85 1 62 TVC MUX 622000 622000 UP 5/0.1 86 1 63 TVC MUX 622000 622000 UP 5/0.1 87 1 64 TVC MUX 622000 622000 UP 5/0.1 88 1 65 TVC MUX 622000 622000 UP 5/0.1 89 1 66 TVC MUX 622000 622000 UP 5/0.1 90 1 67 TVC MUX 622000 622000 UP 5/0.1 91 1 68 TVC MUX 622000 622000 UP 5/0.1 92 1 69 TVC MUX 622000 622000 UP 5/0.1 93 1 70 TVC MUX 622000 622000 UP 5/0.1 94 1 71 TVC MUX 622000 622000 UP |
The highlighted text in Example 2-9 shows the MPLS control VC being set up on VPI/VCI 0/32. The others that show up as TVCs are tag VCs (also called Label Virtual Circuits [LVCs]) that are set up as a result of the label binding exchange that occurs on the control VC. A TVC is used to transport the data cells that make up an MPLS packet.
Example 2-10 is from an ATM switch (LS1010) that is connected to ATM5/0 of gsr4 from Example 2-9. The output from the LS1010 ATM switch is similar to that of gsr4. Again, the control VC setup on VPI/VCI 0/32 is highlighted.
Example 2-10. ATM VC Table on ATM LSR
ls1010-18#show atm vc interface a1/0/0 Interface VPI VCI Type X-Interface X-VPI X-VCI Encap Status ATM1/0/0 0 32 PVC ATM2/0/0 0 55 SNAP UP ATM1/0/0 1 61 TVC(I) ATM2/0/0 0 71 MUX UP ATM1/0/0 1 62 TVC(I) ATM1/1/0 42 81 UP ATM1/0/0 1 63 TVC(I) ATM1/1/0 42 82 UP ATM1/0/0 1 64 TVC(I) ATM1/1/0 42 83 UP ATM1/0/0 1 65 TVC(I) ATM1/1/0 42 84 UP ATM1/0/0 1 66 TVC(I) ATM1/1/0 42 85 UP ATM1/0/0 1 67 TVC(I) ATM1/1/0 42 86 UP ATM1/0/0 1 68 TVC(I) ATM1/1/0 42 87 UP ATM1/0/0 1 69 TVC(I) ATM1/1/0 42 88 UP ATM1/0/0 1 70 TVC(I) ATM1/1/0 42 89 UP ATM1/0/0 1 71 TVC(I) ATM1/1/0 42 90 UP |
VC Merge
Figure 2-10 illustrates an example in which an ATM LSR, ls1010a, has two upstream routers, 7200a and 7500a, and one downstream neighbor, 7200b. As a result of DoD, ls1010a receives a label binding of 1/130 for prefix 12.12.12.12 from 7200b. Because ls1010a has two upstream neighbors, it generates two labels, 1/125 and 1/128, for 12.12.12.12. ls1010a sends label 1/125 to 7200a and 1/128 to 7500a. However, when 7200a and 7500a send cells belonging to IP packets 1 and 2, the label swapping on ls1010a results in all these cells being switched out on interface 0 with a label value of 1/130. The problem that this creates is that router 7200b, which is responsible for reassembling the cells into IP packets, cannot do so correctly because the cells are out of order. 7200b has no way of telling which cells belong to packet 1 and which belong to packet 2.
Figure 2-10. DoD Problem
Figure 2-11 shows one approach to solving the problem illustrated in Figure 2-10. ls1010a could request two labels from 7200b and map each one to the labels it sends to the upstream neighbors. Now, when 7200b receives the cells belonging to IP Packet 1 and IP Packet 2, it has no problem reassembling them, because all cells that have a label value of 1/130 belong to IP Packet 1, and those that have a label value of 1/140 belong to IP Packet 2. The problem with this approach is that you would use additional VCs between ls1010a and 7200b, which would be a concern on ATM LSRs that have limited VPI/VCI space.
Figure 2-11. DoD Nonmerge Solution
Figure 2-12 illustrates VC Merge. In this case, ls1010a buffers all the cells belonging to IP Packet 2 until IP Packet 1 is sent out. Now 7200b reassembles cells belonging to IP packet 1 first and then receives the cells belonging to IP Packet 2. There are no out-of-order cells.
Figure 2-12. DoD VC Merge Solution
Note that ls1010a does not do an IP lookup on the cells it has received. ls1010a doesn't pay attention to the packet's contents; it just buffers them in order to achieve VC Merge.