LDP signaling and MPLS forwarding in the Junos plane

Example 2-8 gives us a chance to look at a live demonstration; in this case, a loopback-to-loopback traceroute from CE1 to BR3 traversing the Junos plane (PE1, P1, PE3).

juniper@CE1> traceroute 192.168.20.3 source 192.168.10.1
traceroute to 192.168.20.3 (192.168.20.3) from 192.168.10.1 [...]
 1  PE1 (10.1.0.1)  7.962 ms  4.506 ms  5.145 ms
 2  P1 (10.0.0.3)  16.347 ms  10.390 ms  10.131 ms
     MPLS Label=300000 CoS=0 TTL=1 S=1
 3  PE3 (10.0.0.9)  9.755 ms  7.490 ms  7.409 ms
 4  BR3 (192.168.20.3)  8.266 ms  10.196 ms  6.466 ms

Let’s interpret the output step by step. As you saw in Chapter 1, PE1 has a BGP route toward BR3’s loopback, and the BGP next hop of this route is PE3. Then, PE1 resolves this BGP next hop by looking at the inet.3 auxiliary table, and this is how the Internet route (to BR3) gets a labeled forwarding next hop.

Let’s see the BGP next-hop resolution process in detail.

juniper@PE1> show route 192.168.20.3 active-path detail
[...]
                Protocol next hop: 172.16.0.33

juniper@PE1> show route table inet.3 172.16.0.33

inet.3: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.33/32     *[LDP/9] 11:00:49, metric 20
                    > to 10.0.0.3 via ge-2/0/4.0, Push 300000

juniper@PE1> show route forwarding-table destination 192.168.20.3
Routing table: default.inet
Internet:
Destination      Type Next hop  Type       Index  NhRef Netif
192.168.20.3/32  user           indr     1048574  3
                      10.0.0.3  Push 300000  593  2     ge-2/0/4.0

PE1 pushes an MPLS header with label 300000 and sends the packet to the forwarding next hop P1. Why label 300000? The answer is in Figure 2-3 and in Example 2-10. This is the label that P1 maps to FEC 172.16.0.33/32.

juniper@PE1> show ldp database | match "put|172.16.0.33"
Input label database, 172.16.0.11:0--172.16.0.1:0
 300000      172.16.0.33/32
Output label database, 172.16.0.11:0--172.16.0.1:0
 300432      172.16.0.33/32
Input label database, 172.16.0.11:0--172.16.0.22:0
  24000      172.16.0.33/32
Output label database, 172.16.0.11:0--172.16.0.22:0
 300432      172.16.0.33/32

This is an interesting command. It lets you know the label mappings that PE1 is learning (Input label database) and advertising (Output label database). This usage of the input and output keywords is sometimes a bit confusing:

• The Input label database contains MPLS labels that PE1 must add to a packet when sending it out to a neighbor. This is input for the control or signaling plane (LDP), but output for the forwarding (MPLS) plane.

• The Output label database contains MPLS labels that PE1 expects to receive from its neighbors. This is output for the control or signaling plane (LDP), but it’s input for the forwarding (MPLS) plane.

After this point is clarified, let’s answer the most important question of this LDP section. If PE1 learns label 300000 from space 172.16.0.1:0, and label 24000 from space 172.16.0.22:0, why is it choosing the first mapping to program the forwarding plane? The answer is on the IGP. Although most of the example topologies in this book use IS-IS, OSPF is an equally valid option and (unless specified otherwise), every statement henceforth applies to IS-IS and OSPF indistinctly.

The shortest path to go from PE1 to PE3 is via P1, so among the several label mappings available for 172.16.0.33/32, PE1 chooses the one advertised by P1. This tight coupling with the IGP is the conceptual key to understanding LDP.

Let’s move on to P1, a pure LSR or P-router.

juniper@P1> show ldp database | match "put|172.16.0.33"
Input label database, 172.16.0.1:0--172.16.0.2:0
  24000      172.16.0.33/32
Output label database, 172.16.0.1:0--172.16.0.2:0
 300000      172.16.0.33/32
Input label database, 172.16.0.1:0--172.16.0.11:0
 300432      172.16.0.33/32
Output label database, 172.16.0.1:0--172.16.0.11:0
 300000      172.16.0.33/32
Input label database, 172.16.0.1:0--172.16.0.33:0
      3      172.16.0.33/32
Output label database, 172.16.0.1:0--172.16.0.33:0
 300000      172.16.0.33/32

juniper@P1> show route table mpls.0 label 300000

mpls.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300000             *[LDP/9] 00:47:20, metric 10
                    > to 10.0.0.9 via ge-2/0/6.0, Pop
300000(S=0)        *[LDP/9] 00:47:20, metric 10
                    > to 10.0.0.9 via ge-2/0/6.0, Pop

juniper@P1> show route forwarding-table label 300000 table default
Routing table: default.mpls
MPLS:
Destination  Type RtRef Next hop         Index    NhRef Netif
300000       user     0 10.0.0.9   Pop     605     2    ge-2/0/6.0
300000(S=0)  user     0 10.0.0.9   Pop     614     2    ge-2/0/6.0

The IGP tells P1 that the next router in the path toward PE3 is PE3 itself. Naturally! And PE3 maps label 3 to FEC 172.16.0.33/32, its own loopback. This is a reserved label value called implicit null. It is not a real label, but a forwarding instruction that translates to pop the label. In other words, an MPLS packet never carries the label value 3, which is simply a signaling artifact. So, the IPv4 packet arrives unlabeled to PE3, and PE3 has the BGP route to reach BR3. The traceroute trip finishes here. This behavior is called Penultimate Hop Popping (PHP).

There is no label swap operation in a two-hop LSP with PHP. For a longer LSP such as PE1-P1A-P1B-PE3, P1A would perform a label swap.

So, is this an LSP? Yes, it is Label-Switched Path; there are MPLS labels after all. But it is signaled in a particular way. The Label Mapping messages depicted in Figure 2-3 allow any router in the network to send MPLS-labeled traffic toward PE3. This is a many-to-one or, in other words, an MP2P LSP.

Let’s finish with a useful toolset described in RFC 4379: MPLS ping and traceroute. These tools don’t require any specific configuration in Junos and they inject UDP-over-IPv4 data packets in an LSP. In that sense, they are very useful to test an LSP’s forwarding plane. The destination IPv4 address of these packets is in the range 127/8, which is reserved for loopback use and is not routable. The appropriate MPLS labels are pushed in order to reach the destination PE, in this case 172.16.0.33. Following is an MPLS traceroute.

juniper@PE1> traceroute mpls ldp 172.16.0.33
  Probe options: ttl 64, retries 3, wait 10, paths 16, exp 7[...]

  ttl    Label  Protocol    Address     Previous Hop   Probe Status
    1   300000  LDP         10.0.0.3    (null)         Success
  FEC-Stack-Sent: LDP
  ttl    Label  Protocol    Address     Previous Hop   Probe Status
    2        3  LDP         10.0.0.9    10.0.0.3       Egress
  FEC-Stack-Sent: LDP

  Path 1 via ge-2/0/4.0 destination 127.0.0.64

LDP signaling and MPLS forwarding in the IOS XR plane

Figure 2-4 presents a similar example, this time focusing on the IOS XR plane (PE2, P2, PE4). The logic is practically identical.

Figure 2-4. LDP label mapping messages for 172.16.0.44

Following is an IPv4 (non MPLS) traceroute from CE2 to BR4.

juniper@CE2> traceroute 192.168.20.4 source 192.168.10.2
traceroute to 192.168.20.4 (192.168.20.4) from 192.168.10.2 [...]
 1  PE2 (10.1.0.3)  4.358 ms  2.560 ms  5.822 ms
 2  P2 (10.0.0.5)  9.627 ms  8.049 ms  9.261 ms
     MPLS Label=24016 CoS=0 TTL=1 S=1
 3  PE4 (10.0.0.11)  8.869 ms  7.833 ms  9.193 ms
 4  BR4 (192.168.20.4)  10.627 ms  11.592 ms  11.593 ms

PE2 has a BGP route toward BR4’s loopback, and the BGP next hop of this route is PE4. As is expained in Chapter 1, IOS XR does not have an auxiliary table such as inet.3 in Junos. The actual forwarding is ruled by the Cisco Express Forwarding (CEF) entry for 172.16.0.44/32.

RP/0/0/CPU0:PE2#show route 192.168.20.4

Routing entry for 192.168.20.4/32
  Known via "bgp 65000", distance 200, metric 0
  Tag 65002, type internal
  Installed Nov 17 08:32:32.941 for 00:30:58
  Routing Descriptor Blocks
    172.16.0.44, from 172.16.0.201
      Route metric is 0
  No advertising protos.

RP/0/0/CPU0:PE2#show cef 172.16.0.44
172.16.0.44/32, version 91, internal [...]
 local adjacency 10.0.0.5
 Prefix Len 32, traffic index 0, precedence n/a, priority 3
   via 10.0.0.5, GigabitEthernet0/0/0/3, 6 dependencies [...]
    path-idx 0 NHID 0x0 [0xa0eb34a4 0x0]
    next hop 10.0.0.5
    local adjacency
     local label 24021      labels imposed {24016}

PE2 pushes an MPLS header with label 24016 and sends the packet to the forwarding next hop P2. Why label 24016? As you can see in Figure 2-4 and in Example 2-15, this is the label that P2 maps to FEC 172.16.0.44/32.

RP/0/0/CPU0:PE2# show mpls ldp bindings 172.16.0.44/32
172.16.0.44/32, rev 85
        Local binding: label: 24021
        Remote bindings: (2 peers)
            Peer                Label
            -----------------   ---------
            172.16.0.2:0        24016
            172.16.0.11:0       300224

Now, let’s see the LDP signaling and the forwarding state on P2, the next hop LSR.

RP/0/0/CPU0:P2# show mpls ldp bindings 172.16.0.44/32
172.16.0.44/32, rev 36
        Local binding: label: 24016
        Remote bindings: (3 peers)
            Peer                Label
            -----------------   ---------
            172.16.0.1:0        299840
            172.16.0.22:0       24021
            172.16.0.44:0       ImpNull

RP/0/0/CPU0:P2#show mpls forwarding labels 24016
Local  Outgoing  Prefix         Outgoing   Next Hop     Bytes
Label  Label     or ID          Interface               Switched
------ --------- -------------- ---------- ------------ ----------
24016  Pop       172.16.0.44/32 Gi0/0/0/5  10.0.0.11    379266

Unlike Junos, IOS XR uses MPLS forwarding to reach internal IPv4 prefixes. So, a plain IPv4 traceroute from PE2 to PE4 shows the label, too (although it provides less information than MPLS traceroute).

RP/0/0/CPU0:PE2#traceroute ipv4 172.16.0.44
[...]
 1  p2 (10.0.0.5) [MPLS: Label 24016 Exp 0] 9 msec  0 msec  0 msec
 2  pe4 (10.0.0.11) 0 msec  *  0 msec

Load-balancing hash algorithm

Load balancing is a complex topic that is intimately related to the hardware implementation of each platform. The good news is that Junos and IOS XR are both capable of doing per-flow load balancing of IP and MPLS traffic. Unlike stateful firewalls, LSRs perform packet-based (not flow-based) forwarding, so what is a flow in the context of a LSR?

A flow is a set of packets with common values in their headers. For example, all the packets of a TCP connection from a client to a server (or of a voice stream between two endpoints), have several fields in common: source and destination address, transport protocol, source and destination ports, and so on. To guarantee that all the packets of a given flow arrive to the destination in the correct order, they should all follow exactly the same path; indirectly, this means that they share respectively the same MPLS label values, hop by hop.

On the other hand, different flows should be evenly distributed across equal-cost next hops such as ECMP, LAG, and so on. Otherwise, some links would not be utilized and others would quickly saturate. This phenomenon is commonly called traffic polarization.

Let’s see how routers achieve per-flow load balancing. For every single packet, the router selects some header fields (plus a fixed local randomization seed) and applies a mathematical algorithm to them called a hash. This algorithm is very sensitive to small variations of its input values. The hash result determines (modulus the number of equal-cost next hops) the actual forwarding next hop to which the packet is mapped. All the packets of a given flow receive the same hash value and are hence forwarded out to the same next hop.

Basic per-flow load balancing is enabled by default in IOS XR, but it requires explicit configuration in Junos, which performs per-destination route hashing by default.

policy-options {
    policy-statement PL-LB {
      then load-balance per-packet; 
}}
routing-options {
  forwarding-table export PL-LB; 
}

Let’s forget for a moment that the topology has two vendor-specific planes. This is a vendor-agnostic analysis of an IP flow from CE1 to BR4:

• The ingress PE1 receives plain IPv4 packets from CE1 and applies a hash to them. Because all the packets belong to the same flow, the result of the hash is the same and they are all forwarded to the same next hop: P1 or PE2. If the next hop is PE2, there is only one shortest path remaining and the load-balancing discussion stops here.

• Let’s suppose that the next hop is P1. So, P1 receives MPLS packets and applies a hash to them. This hash takes into account the MPLS label value(s) and it might also consider the inner (e.g., IPv4) headers. As a result, all the packets of this flow are sent out to one and only one of the available next hops: PE3, P2-link1, or P2-link2.

MPLS hash and Entropy Labels

Many LSRs in the industry are able to include MPLS packet payload fields (like IP addresses, TCP/UDP ports) into the load-balancing hash algorithm. But some low-end (or old) platforms from different vendors cannot do that. This can be an issue if the number of active FECs is low. For example, in a domestic Internet Service Provider (ISP) that sends all the upstream traffic up to only two big Internet gateways, most of the packets carry either label L1 (mapped to FEC gateway_1) or label L2 (mapped to FEC gateway_2). Two different label values are clearly not enough to spread traffic across multiple equal-cost paths.

To ensure that there is enough randomness to achieve good load balancing on these devices, RFC 6790 introduces the concept of Entropy Labels. These labels have a per-flow random value and do not have any forwarding significance. In other words, they are not mapped to any FEC. Their goal is just to ensure smooth load balancing along the available equal cost paths. You can read more about Entropy Labels in Chapter 6.

There is a similar technology called Flow-Aware Transport (FAT, RFC 6391), but it is specific of Layer 2 (L2) services. Chapter 6 also covers this in greater detail.

Local FEC label binding/allocation

As shown earlier, PE3 and PE4 both advertise their own loopback mapped to the implicit null label. The following command shows all of the local (or egress) FECs that PE3 and PE4 advertise.

juniper@PE3> show ldp database session 172.16.0.44 | match "put|  3"
Input label database, 172.16.0.33:0--172.16.0.44:0
      3      10.0.0.10/31
      3      10.0.0.12/31
      3      10.2.0.0/24
      3      10.255.0.0/16
      3      172.16.0.44/32
Output label database, 172.16.0.33:0--172.16.0.44:0
      3      172.16.0.33/32

The only local FEC that PE3 (Junos) advertises via LDP is its primary lo0.0 address. This is a default behavior that you can change by applying an egress-policy at the [edit protocols ldp] hierarchy. A common use case covered in Chapter 3 is the advertisement of nonprimary lo0.0 IP addresses. Additionally, LDP export policies provide granular per-neighbor FEC advertisement.

On the other hand, PE4 (IOS XR) advertises label mappings for all its directly connected routes by default. Most services use LSPs whose endpoints are loopback addresses, though. In that sense, you can configure IOS XR to do the following:

• Only advertise /32 FECs by using mpls ldp address-family ipv4 label local allocate for host-routes

• Granular label binding and advertisement with policies applied at mpls ldp address-family ipv4 label.

The benefit is a lower amount of state to be kept and exchanged in the LIBs.

What about remote (nonlocal) FECs? By default, both Junos and IOS XR advertise label mappings for IGP routes, regardless of their mask. Again, the previously listed knobs make it possible to change this default behavior.

Label advertisement modes

Figure 2-3 and Figure 2-4 illustrate the Downstream Unsolicited (DU) LDP label advertisement (or distribution) mode that both Junos and IOS XR use by default. This elicits two questions:

• Why downstream? When it advertises label mapping (300000, 172.16.0.33/32), P1 is telling its neighbors: if you want to use me as a downstream LSR to reach 172.16.0.33/32, send me the packets with this label. So, P1 becomes a potential downstream LSR for that FEC.

• Why unsolicited? P1’s neighbors do not request any label mappings from P1; however, P1 sends the messages.

Chapter 16 briefly mentions another label distribution method called Downstream on Demand (DoD), which is also used by RSVP-TE.

Label distribution control modes

There are two label distribution control modes: ordered and independent. Junos implements the ordered mode, whereas IOS XR implements the independent mode.

In the ordered mode, the following sequence takes place in strict chronological sequence (see Figure 2-3):

1. PE3 advertises the label mapping (172.16.0.33/32, 3) to its neighbors.

2. P1 receives this label mapping from PE3, the egress LSR, and the shortest-path next hop from P1 to 172.16.0.33 is precisely the direct link P1→PE3.

3. P1 binds label 300000 to this FEC, installs the forwarding entry (300000→ pop to 10.0.0.9) in its Label Forwarding Information Base (LFIB) and advertises the Label Mapping (172.16.0.33/32, 300000) to its neighbors.

4. PE1 receives the label mapping from P1, and the shortest path next hop from PE1 to 172.16.0.33 is precisely P1.

5. PE1 binds label 300432 to the FEC, installs the forwarding entry (300432→ swap 300000 to 10.0.0.3) in its LFIB and advertises the label mapping (172.16.0.33/32, 300432) to its neighbors.

In a nutshell, before binding a label to a remote FEC, Junos LSRs first need to receive a label mapping from the shortest-path downstream LSR en route to the FEC. Likewise, if it loses the downstream labeled state to the FEC (due to an LDP event or to a topology change), after some time the Junos LSR removes the label binding and sends a Label Withdraw message out to its neighbors.

The ordered mode guarantees a strong consistency between the control and the forwarding plane; on the other hand, it requires a potentially higher time to establish the LSPs.

How about independent mode? P2 (IOS XR) binds and announces label mappings regardless of the FEC’s downstream label state.

Suppose that P2 has not established any LDP session yet. Nevertheless, P2 binds labels to local and remote FECs. Then, suppose that the LDP session between P2 and PE2 (and only this session) comes up. At this point, P2 advertises all the label mappings to PE2. These mappings include (172.16.0.33/32, 24000) and (172.16.0.44/32, 24016). As you can see in Example 2-21, the resulting LFIB entries at P2 are marked as Unlabelled.

RP/0/0/CPU0:P2#show mpls forwarding
Local  Outgoing    Prefix          Outgoing   Next Hop   Bytes
Label  Label       or ID           Interface             Switched
------ ----------- --------------- ---------- ---------- ---------
[...]
24000  Unlabelled  172.16.0.33/32  Gi0/0/0/2  10.0.0.6   25110
       Unlabelled  172.16.0.33/32  Gi0/0/0/3  10.0.0.24  2664
       Unlabelled  172.16.0.33/32  Gi0/0/0/5  10.0.0.11  2664
24016  Unlabelled  172.16.0.44/32  Gi0/0/0/5  10.0.0.11  134
[...]

What if P2 receives a packet whose outer MPLS label is 24000? The Unlabelled instruction means pop all the labels and forward to the next hop(s) in the LFIB. This is different from the Pop instruction, which just pops the outer label.

The outcome depends on the traffic flows:

• Internet traffic from CE2 to BR4 successfully reaches its destination.

• Internet traffic from CE2 to BR3 is forwarded by P2 across three equal-cost next hops. Two of them point to P1, which has no route toward the destination and thus drops the packets.

• VPN traffic with several labels in the stack might be mapped to the master routing instance (and likely discarded) by the next hop.

When all the LDP sessions come up and P2 receives all the label mapping messages from its neighbors, P2’s LFIB is programmed with the appropriate Swap (to a given label) and Pop instructions.

RP/0/0/CPU0:P2#show mpls forwarding
Local  Outgoing    Prefix          Outgoing   Next Hop   Bytes
Label  Label       or ID           Interface             Switched
------ ----------- --------------- ---------- ---------- ---------
[...]
24000  300000      172.16.0.33/32  Gi0/0/0/2  10.0.0.6   25110
       300000      172.16.0.33/32  Gi0/0/0/3  10.0.0.24  2664
       24000       172.16.0.33/32  Gi0/0/0/5  10.0.0.11  2664
24016  Pop         172.16.0.44/32  Gi0/0/0/5  10.0.0.11  134
[...]

The ordered and independent label distribution control modes are radically different and each has its pros and cons in terms of control and delay. The final state after LDP converges is the same, regardless of the implemented mode.

Label retention modes

Both Junos and IOS XR implement Liberal Label Retention Mode (as opposed to Conservative) by default, meaning that the LSRs accept and store all the incoming label mapping messages. For example, PE1 receives label mappings for FEC 172.16.0.33/32 from both P1 and PE2. Even though the forwarding next hop is P1, PE1 decides to store both label mappings. Why? Potentially, a topology change in the future might turn PE2 into the next hop. Therefore, PE1 keeps all the states, just in case.

FEC aggregation

Looking back at Example 2-20, PE4 advertises five different local FECs to PE3, all of them mapped to the implicit null label. Let’s focus on two of them: 172.16.0.44/32 and 10.0.0.10/31. By default, PE3 advertises them with the same label to P1.

This default behavior in Junos is called FEC aggregation, and you can disable it by configuring set protocols ldp deaggregate. Here is the outcome:

juniper@PE3> show ldp database | match "put|172.16.0.44|10.0.0.10"
[...]
Output label database, 172.16.0.33:0--172.16.0.1:0
 299856      10.0.0.10/31
 299856      172.16.0.44/32
Input label database, 172.16.0.33:0--172.16.0.44:0
      3      10.0.0.10/31
      3      172.16.0.44/32

juniper@PE3> show ldp database | match "put|172.16.0.44|10.0.0.10"
[...]
Output label database, 172.16.0.33:0--172.16.0.1:0
 299920      10.0.0.10/31
 299856      172.16.0.44/32
Input label database, 172.16.0.33:0--172.16.0.44:0
      3      10.0.0.10/31
      3      172.16.0.44/32

LDP IGP Synchronization (RFC 5443)

What happens if PE1 and P1 bring up an IS-IS adjacency together, but for whatever reason (routing/filtering issue, misconfiguration, etc.), they do not establish an LDP session to each other? From the point of view of PE1, the shortest path to PE3 is still PE1-P1-PE3. Unfortunately, this path is unlabeled, so P1 discards the customer traffic. In other words, CE1 can no longer ping BR3.

The LDP IGP Synchronization feature increases the IGP metric of a link if LDP is down on it. This way, the network dynamically skips unlabeled links and restores the service. Following is the syntax for IS-IS, which is very similar to the one for OSPF.

/* Junos sample configuration */
protocols {
  isis {
     interface ge-0/0/4.0 ldp-synchronization; 
 }}

/* IOS XR sample configuration */
router isis mycore
 interface GigabitEthernet0/0/0/3
  address-family ipv4 unicast
   mpls ldp sync

In the following example, the LDP IGP Synchronization feature is turned on for all the network core links, and all the LDP sessions are up except for the one between PE1 and P1. The customer traffic finds its way through a longer yet labeled path. So the end-to-end service is fine.

juniper@PE1> show isis database level 2 PE1.00-00 extensive
[...]
    IS extended neighbor: P1.00, Metric: default 16777214
    IS extended neighbor: PE2.00, Metric: default 10
[...]

juniper@PE1> show isis database level 2 P1.00-00 extensive
[...]
    IS extended neighbor: PE1.00, Metric: default 16777214
    IS extended neighbor: P2.00, Metric: default 10
    IS extended neighbor: P2.00, Metric: default 10
    IS extended neighbor: PE3.00, Metric: default 10
[...]

juniper@CE1> traceroute 192.168.20.3 source 192.168.10.1
traceroute to 192.168.20.3 (192.168.20.3) from 192.168.10.1 [...]
 1  PE1 (10.1.0.1)  7.577 ms  3.113 ms  3.478 ms
 2  PE2 (10.0.0.1)  14.778 ms  13.087 ms  11.303 ms
     MPLS Label=24000 CoS=0 TTL=1 S=1
 3  P2 (10.0.0.5)  11.723 ms  12.630 ms  14.843 ms
     MPLS Label=24000 CoS=0 TTL=1 S=1
 4  P1 (10.0.0.24)  14.599 ms  15.018 ms  23.803 ms
     MPLS Label=300032 CoS=0 TTL=1 S=1
 5  PE3 (10.0.0.9)  13.564 ms  20.615 ms  25.406 ms
 6  BR3 (192.168.20.3)  18.587 ms  15.589 ms  19.322 ms

Both Junos and IOS XR support this feature on IGP interfaces configured as point-to-point, which is the recommended mode for core links. In addition, IOS XR also supports it on broadcast links.

LDP Session Protection

Session Protection is another LDP robustness enhancement, based on the Targeted Hello functionality that is defined on RFC 5036. With this feature, two directly connected LDP peers exchange two kinds of LDP-over-UDP Hello packets:

LDP Link Hellos

Single-hop (TTL=1) multicast packets sourced at the link address, destined to 224.0.0.2 and sent independently on each link. These packets achieve basic discovery (see Figure 2-2).

LDP Targeted Hellos

Multihop (TTL>1) loopback-to-loopback unicast packets, enabled by using the Session Protection feature.

LDP Session Protection, as it name implies, maintains the LDP session up upon a link flap. Even if the direct PE1-P1 link goes down, the LDP-over-TCP session and the LDP-over-UDP targeted hello adjacency are both multihop. These packets are routed across the alternate PE1-PE2-P2-P1 path, and in this way the LDP session and the LDP hello adjacency between PE1 and P1 both remain up. The routers keep all the LDP label mappings, which adds forwarding plane robustness to the network.

Let’s look at the configuration and its outcome in Junos:

protocols {
    ldp {
        interface lo0.0;
        session-protection; 
}}

juniper@PE1> show ldp session
  Address           State        Connection  Hold time  Adv. Mode
172.16.0.1          Operational  Open          26         DU
172.16.0.22         Operational  Open          29         DU

juniper@PE1> show ldp neighbor
Address            Interface          Label space ID   Hold time
10.0.0.1           ge-2/0/3.0         172.16.0.22:0      13
10.0.0.3           ge-2/0/4.0         172.16.0.1:0       14
172.16.0.1         lo0.0              172.16.0.1:0       44
172.16.0.22        lo0.0              172.16.0.22:0      41

PE1 does not have parallel links to any neighboring router. So, there are two hello adjacencies to each peer (identified by a common Label space ID): the link hello and the targeted hello adjacency.

Finally, let’s see it on IOS XR:

mpls ldp
 session protection

RP/0/0/CPU0:PE2#show mpls ldp discovery brief

Local LDP Identifier: 172.16.0.22:0

Discovery Source     VRF Name   Peer LDP Id      Holdtime Session
-------------------- ---------- ---------------- -------- -------
Gi0/0/0/2            default    172.16.0.11:0       15       Y
Gi0/0/0/3            default    172.16.0.2:0        15       Y
Tgt:172.16.0.2       default    172.16.0.2:0        90       Y
Tgt:172.16.0.11      default    172.16.0.11:0       45       Y

RSVP-TE Tunnels, LSPs, and Sessions

Table 2-2 summarizes the different terminology used by RFC 3209, Junos, and IOS XR.

In the terms of RFC 3209, you configure tunnels on the ingress PE. A tunnel is incarnated through one or more LSPs. There are several reasons why you may have more than one LSP per tunnel, for example:

• A tunnel has a primary LSP protected by a standby LSP. This topic is discussed in Chapter 19. This type of tunnel has two persistent LSPs.

• A tunnel has only one primary LSP but it is being resignaled upon failure, reoptimization, or a change in TE constraints such as bandwidth. In these cases, the tunnel may transitorily have more than one LSP.

You can view an LSP as an incarnation of a tunnel. Two LSPs that belong to the same tunnel share the Tunnel ID value and have a different LSP ID that differentiates them.

In this book, the different vendor terminologies are used and you might see the words tunnel and LSP used in a relatively relaxed and interchangeable manner. This chapter uses the Junos terminology.

RSVP-TE LSP configuration

RSVP-TE LSPs are configured at the head-end (ingress) PE. This makes sense for P2P LSPs, because MPLS LSPs in general—with the exception of MP2MP—are unidirectional. So, even with no specific LSP configuration at PE3 and PE4, Example 2-32 and Example 2-33 are enough to signal the following LSPs.

From PE1 (Junos) to: PE2, PE3, and PE4.

1     groups {
2         GR-LSP {
3             protocols {
4                 mpls label-switched-path <*> adaptive;
5     }}}
6     protocols {
7         mpls {
8             apply-groups GR-LSP;
9             label-switched-path PE1--->PE2 to 172.16.0.22;
10            label-switched-path PE1--->PE3 to 172.16.0.33;
11            label-switched-path PE1--->PE4 to 172.16.0.44;
12    }}

From PE2 (IOS XR) to: PE1, PE3, and PE4.

group GR-LSP
 interface 'tunnel-te.*'
  ipv4 unnumbered Loopback0
  autoroute announce
  record-route
  path-option 1 dynamic
end-group
!
interface tunnel-te11
 apply-group GR-LSP
 signalled-name PE2--->PE1
 destination 172.16.0.11
!
interface tunnel-te33
 apply-group GR-LSP
 signalled-name PE2--->PE3
 destination 172.16.0.33
!
interface tunnel-te44
 apply-group GR-LSP
 signalled-name PE2--->PE4
 destination 172.16.0.44

Bidirectional end-to-end traffic (such as a successful ping between CE1 and BR3) also requires right-to-left LSPs for the return traffic. As a result, unless another MPLS flavor such as LDP or SPRING is enabled in the core, you also need to configure RSVP-TE LSPs rooted from PE3 and from PE4.

In this way, the network has a full mesh of PE→PE RSVP LSPs.

In “RSVP-TE in Action”, you will see that PE1 (Junos) automatically installs 172.16.0.33/32 in the inet.3 auxiliary table, pointing to LSP PE1--->PE3. On the other hand, PE2 (IOS XR) needs the autoroute announce command to make the CEF entry 172.16.0.44/32 point to interface tunnel-te44 (LSP PE2--->PE4). But this command has more implications, as you can see at the end of Chapter 3.

The Traffic Engineering Database

What happens directly after you configure a RSVP-TE LSP? By default, the ingress PE doesn’t leave anything to fate. It decides in advance the LSP’s exact itinerary by building an ordered list of the hops that the LSP should go through. This list is encoded in an Explicit Route Object (ERO). But where does the ingress PE find the information to compute the ERO? It finds it in the Traffic Engineering Database (TED).

Let’s have a sneak peek on a Junos router’s TED.

juniper@PE1> show ted database PE1.00
TED database: 7 ISIS nodes 7 INET nodes
ID                            Type Age(s) LnkIn LnkOut Protocol
PE1.00(172.16.0.11)           Rtr     198     2      2 IS-IS(2)
    To: P1.00(172.16.0.1), Local: 10.0.0.2, Remote: 10.0.0.3
    To: PE2.00(172.16.0.22), Local: 10.0.0.0, Remote: 10.0.0.1

juniper@PE1> show ted database P1.00
TED database: 7 ISIS nodes 7 INET nodes
ID                            Type Age(s) LnkIn LnkOut Protocol
P1.00(172.16.0.1)             Rtr      92     4      4 IS-IS(2)
    To: PE1.00(172.16.0.11), Local: 10.0.0.3, Remote: 10.0.0.2
    To: PE3.00(172.16.0.33), Local: 10.0.0.8, Remote: 10.0.0.9
    To: P2.00(172.16.0.2), Local: 10.0.0.6, Remote: 10.0.0.7
    To: P2.00(172.16.0.2), Local: 10.0.0.24, Remote: 10.0.0.25

juniper@PE1> show ted database PE3.00
TED database: 7 ISIS nodes 7 INET nodes
ID                            Type Age(s) LnkIn LnkOut Protocol
PE3.00(172.16.0.33)           Rtr     133     2      2 IS-IS(2)
    To: P1.00(172.16.0.1), Local: 10.0.0.9, Remote: 10.0.0.8
    To: PE4.00(172.16.0.44), Local: 10.0.0.12, Remote: 10.0.0.13

Similarly, PE2 (IOS XR) also has a TED (Example 2-35).

RP/0/0/CPU0:PE2#show mpls traffic-eng topology brief 172.16.0.22
[...]
IGP Id: 1720.1600.0022.00, MPLS TE Id: 172.16.0.22 Router Node
  (IS-IS mycore level-2)
  Link[0]:Point-to-Point, Nbr IGP Id:1720.1600.0002.00 [...]
  Link[1]:Point-to-Point, Nbr IGP Id:1720.1600.0011.00 [...]

RP/0/0/CPU0:PE2#show mpls traffic-eng topology brief 172.16.0.2
[...]
IGP Id: 1720.1600.0002.00, MPLS TE Id: 172.16.0.2 Router Node
  (IS-IS mycore level-2)
  Link[0]:Point-to-Point, Nbr IGP Id:1720.1600.0022.00 [...]
  Link[1]:Point-to-Point, Nbr IGP Id:1720.1600.0001.00 [...]
  Link[2]:Point-to-Point, Nbr IGP Id:1720.1600.0001.00 [...]
  Link[3]:Point-to-Point, Nbr IGP Id:1720.1600.0044.00 [...]

RP/0/0/CPU0:PE2#show mpls traffic-eng topology brief 172.16.0.44
[...]
IGP Id: 1720.1600.0044.00, MPLS TE Id: 172.16.0.44 Router Node
  (IS-IS mycore level-2)
  Link[0]:Point-to-Point, Nbr IGP Id:1720.1600.0002.00 [...]
  Link[1]:Point-to-Point, Nbr IGP Id:1720.1600.0033.00 [...]

The TED looks very much like a Link State Database (LSDB). Indeed, protocols such as IS-IS or OSPF feed the information to build the TED. In addition, both the LSDB and the TED contain per-link Traffic Engineering information that you can see by using the extensive keyword.

Here are the main differences between the IS-IS (or OSPF) LSDB and the TED:

• The TED is protocol agnostic. It can be populated by IS-IS, OSPF, or even BGP (with a special address family).

• The TED is unique and there is one separate LSDB per IGP (OSPF, IS-IS) instance or process.

• The IS-IS (or OSPF) LSDB has information about MPLS and non-MPLS interfaces, whereas the TED only contains MPLS interfaces.

And how can you tell from the LSDB whether a link has MPLS turned on? Let’s temporarily remove family mpls from PE1’s interface ge-2/0/4 (connected to P1). Or, alternatively, delete ge-2/0/4 from protocols rsvp | mpls. Example 2-36 shows PE1’s Link State Packet.

juniper@PE1> show isis database PE1 extensive
[...]
  TLVs:
    IS extended neighbor: PE2.00, Metric: default 10
      IP address: 10.0.0.0
      Neighbor's IP address: 10.0.0.1
      Local interface index: 336, Remote interface index: 0
      Current reservable bandwidth:
        Priority 0 : 1000Mbps
        Priority 1 : 1000Mbps
        Priority 2 : 1000Mbps
        Priority 3 : 1000Mbps
        Priority 4 : 1000Mbps
        Priority 5 : 1000Mbps
        Priority 6 : 1000Mbps
        Priority 7 : 1000Mbps
      Maximum reservable bandwidth: 1000Mbps
      Maximum bandwidth: 1000Mbps
      Administrative groups:  0 <none>
    IS extended neighbor: P1.00, Metric: default 10
      IP address: 10.0.0.2
      Neighbor's IP address: 10.0.0.3
      Local interface index: 337, Remote interface index: 0

juniper@PE1> show ted database PE1
TED database: 11 ISIS nodes 7 INET nodes
ID                            Type Age(s) LnkIn LnkOut Protocol
PE1.00(172.16.0.11)           Rtr     601     2      1 IS-IS(2)
    To: PE2.00(172.16.0.22), Local: 10.0.0.0, Remote: 10.0.0.1
      Local interface index: 336, Remote interface index: 0

Only the MPLS link (PE1-PE2) contains Traffic Engineering sub–Type Length Value (sub-TLVs), and as a result this is the only interface at PE1 that makes it to the TED. Let’s enable MPLS and RSVP on PE1-P1 interface again and move on.

Constrained Shortest-Path First

To compute the ERO for the PE1→PE3 LSP, PE1 runs an algorithm called Constrained Shortest-Path First (CSPF), which finds the best path to PE3 in the TED. Although this book does explore a wide variety of TE constraints later on in Chapter 13 through Chapter 15, the LSPs in Example 2-32 and Example 2-33 are so simple that they impose no constraints at all. And without constraints, CSPF looks very much like the traditional Shortest-Path First (SPF). Here is the outcome of the CSPF calculation that preceded PE1→PE3 LSP’s signaling from PE1:

juniper@PE1> show rsvp session name PE1--->PE3 detail
[...]
  Explct route: 10.0.0.1 10.0.0.5 10.0.0.24 10.0.0.9

Surprise! The PE1→PE3 LSP is now signaled via PE2, and it has four hops instead of two. Why? Remember that MPLS was temporarily disabled on the PE1→P1 link. This brought down the RSVP-TE LSP and triggered a CSPF computation through a longer alternate path. Yet, now that PE1→P1 is fine again from the point of view of MPLS, why is the LSP still following a longer path?

In both Junos and IOS XR, simple RSVP-TE LSPs tend to avoid flapping links. When they are signaled, RSVP LSPs can remain indefinitely on their current path. If there is a failure (e.g., in one of the path’s links or nodes), the ingress PE runs CSPF again and resignals the LSP.

Thus, the PE1→PE3 LSP has a suboptimal ERO. How can you reoptimize this LSP, or in other words, how can you trigger a CSPF recalculation? Manually flapping a link is not a good idea. There are better ways.

First, you can manually reoptimize an LSP by executing the following operational commands:

• Junos: clear mpls lsp name PE1--->PE3 optimize

• IOS XR: mpls traffic-eng reoptimize 44 (tunnel-te 44)

However, this is not scalable from an operational perspective. In both Junos and IOS XR, it is recommended that you configure a reoptimization timer. When the timer expires, the ingress PE runs CSPF again, and if the result is better than the current path, the LSP is resignaled.

You can configure reoptimization timers in Junos either globally or on a per-LSP basis, and they are global in IOS XR. Let’s call this timer T1 (in seconds):

• Junos: protocols mpls [label-switched-path <name>] optimize-timer <T1>

• IOS XR: mpls traffic-eng reoptimize <T1>

LSP optimization takes place in a make-before-break fashion. Before tearing down the original path, PE1 signals a new PE1→PE3 path and gracefully switches the traffic to it. In that sense, the change is not disruptive and does not cause any transit packet loss. In scaled environments, it is wise to delay this switchover, allowing time for the LSP’s forwarding plane to be ready before the routes point to the new path. Let’s call this timer T2 (in seconds):

• Junos: protocols mpls optimize-switchover-delay <T2>

• IOS XR: mpls traffic-eng optimize timers delay installation <T2>

How do T1 and T2 relate to each other? Let’s see an example, by using the Junos terminology from Table 2-2.

The PE1→PE3 LSP is initially mapped to RSVP session A, which follows the shortest IGP path PE1-P1-PE3. Then, the PE1-P1 link experiences a short flap (up→down→up).

Directly after the up→down transition, RSVP session A goes down, and PE1 signals a new RSVP session B through a (longer) available path—for example, PE1-PE2-P2-PE4-PE3. PE1 quickly activates the LSP on RSVP session B and starts timer T1. At this point, the user traffic is restored.

While T1 is ticking down, the link comes back up and IS-IS converges. That’s orthogonal to T1, which just keeps ticking down. When T1 expires, PE1 signals a new RSVP session C through the shortest path PE1-P1-PE3, and starts timer T2.

While T2 is ticking down, PE1 keeps both RSVP sessions B and C up, but the LSP and the user traffic are still on session B. Only when T2 expires, PE1 switches the LSP and the user traffic to session C.

RSVP-TE messages

After the ingress PE computes the ERO, it begins to signal the LSP. Let’s focus on the PE1→PE3 example. As shown in Figure 2-5, the ingress PE (PE1) sends Path messages and the egress PE (PE3) answers with Resv messages. These RSVP messages are encapsulated directly on top of IP (RSVP = IPv4 protocol 46).

Figure 2-5. RSVP-TE Path and Resv messages

In addition to the ERO, a Path message contains several objects, including the Record Route Object (RRO). The ERO and the RRO have symmetrical roles: whereas the ERO shrinks hop by hop (as there are less hops to go), the RRO grows hop by hop (as there are more hops left behind).

RSVP Path messages have a destination IPv4 address equal to the egress PE’s loopback (and not to the transit LSR). For this reason, the ingress PE sets the Router Alert (RA) option in the IPv4 header. This allows the transit LSRs (P1) to intercept and process the Path messages at the control plane, thereby creating dynamic local LSP state and updating both the ERO and the RRO on a hop-by-hop basis.

The Resv messages flow in the opposite direction (upstream) and contain label information. First, the egress PE (PE3) signals the implicit null label; then, the upstream LSRs assign a locally unique label bound to the LSP.

Because Resv messages are triggered by Path messages, RSVP-TE label distribution method is DoD, as compared to the default LDP mode (DU).

RSVP-TE LSPs are maintained by periodic Path/Resv message refresh. This per-LSP message exchange is often called an RSVP session. You can view an RSVP session as a control plane incarnation of an LSP. This is a subtle nuance, so in the RSVP world, the terms LSP and session are often used interchangeably (see Table 2-3).

After it is configured to do so, PE3 also signals a PE3→PE1 LSP by sending Path messages to PE1 and receiving Resv messages from PE1. This enables bidirectional end-to-end traffic.

LSRs send Path and Resv messages periodically in order to keep the RSVP-TE sessions alive. Chapter 16 covers some possible optimizations.

There is also a set of messages (PathErr, PathTear, ResvErr, and ResvTear) that signal LSP error conditions or tear down RSVP-TE LSPs.

RSVP-TE signaling and MPLS forwarding in the Junos plane

The first example (Example 2-38) is a loopback-to-loopback traceroute from CE1 to BR3 traversing the Junos plane (PE1, P1, PE3).

juniper@CE1> traceroute 192.168.20.3 source 192.168.10.1
traceroute to 192.168.20.3 (192.168.20.3) from 192.168.10.1 [...]
 1  PE1 (10.1.0.1)  21.468 ms  8.833 ms  4.311 ms
 2  P1 (10.0.0.3)  20.169 ms  33.771 ms  137.208 ms
     MPLS Label=300560 CoS=0 TTL=1 S=1
 3  PE3 (10.0.0.9)  14.305 ms  13.516 ms  12.845 ms
 4  BR3 (192.168.20.3)  23.651 ms  10.378 ms  11.674 ms

Let’s interpret the output step by step. As you saw in Chapter 1, PE1 has a BGP route toward BR3’s loopback, and the BGP next hop of this route is PE3. Then, PE1 resolves this BGP next hop by looking at the inet.3 auxiliary table, and this is how the Internet route (to BR3) gets a labeled forwarding next hop.

juniper@PE1> show route 192.168.20.3 active-path detail
[...]
                Protocol next hop: 172.16.0.33

juniper@PE1> show route table inet.3 172.16.0.33

inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.33/32     *[RSVP/7/1] 05:01:26, metric 20
        > to 10.0.0.3 via ge-2/0/4.0, label-switched-path PE1--->PE3

juniper@PE1> show route forwarding-table destination 192.168.20.3
Routing table: default.inet
Internet:
Destination      Type Next hop  Type       Index  NhRef Netif
192.168.20.3/32  user           indr     1048576  2
                      10.0.0.3  Push 300560  595  2     ge-2/0/4.0

PE1 pushes an MPLS header with label 300560 and sends the packet to the forwarding next hop P1. Why label 300560? The answer is in Figure 2-5, Figure 2-6, and Example 2-40: because this is the label that P1 maps to the LSP PE1→PE3.

juniper@PE1> show rsvp session
Ingress RSVP: 3 sessions
To           From         State Style Labelin Labelout LSPname
172.16.0.22  172.16.0.11  Up       SE       -        3 PE1--->PE2
172.16.0.33  172.16.0.11  Up       FF       -   300560 PE1--->PE3
172.16.0.44  172.16.0.11  Up       SE       -   300256 PE1--->PE4
Total 3 displayed, Up 3, Down 0

Egress RSVP: 3 sessions
To           From         State Style Labelin Labelout LSPname
172.16.0.11  172.16.0.22  Up       SE       3        - PE2--->PE1
172.16.0.11  172.16.0.44  Up       SE       3        - PE4--->PE1
172.16.0.11  172.16.0.33  Up       FF       3        - PE3--->PE1
Total 3 displayed, Up 3, Down 0

Transit RSVP: 2 sessions
To           From         State Style Labelin Labelout LSPname
172.16.0.22  172.16.0.33  Up       SE  299952        3 PE3--->PE2
172.16.0.33  172.16.0.22  Up       SE  299968   300144 PE2--->PE3
Total 2 displayed, Up 2, Down 0

juniper@PE1> show rsvp session name PE1--->PE3 detail
[...]
  PATH sentto: 10.0.0.3 (ge-2/0/4.0) 4226 pkts
  RESV rcvfrom: 10.0.0.3 (ge-2/0/4.0) 4235 pkts[...]
  Explct route: 10.0.0.3 10.0.0.9
  Record route: <self> 10.0.0.3 10.0.0.9

From the perspective of PE1, there are three types of RSVP sessions:

• Ingress RSVP sessions correspond to LSPs originated at PE1 (head-end). They have PE1’s router ID in the second column (From).

• Egress RSVP sessions correspond to LSPs that terminate at PE1 (tail-end). They have PE1’s router ID in the first column (To).

• Transit RSVP sessions correspond to LSPs that go through PE1, but whose two endpoints are both outside PE1.

The Style column can show two different values: Shared Explicit (SE) and Fixed Filter (FF). SE is the recommended mode because it makes sure that bandwidth reservations (if any) are not double counted. It is the default in IOS XR and requires explicit configuration in Junos, as you can see in Example 2-32, line 4 (adaptive keyword).

Now, let’s see how to interpret the Labelin and Labelout columns:

• If PE1 needs to send a packet through LSP PE1→PE3, PE1 pushes label 300560 to the packet before sending it out to the next hop.

• If PE1 receives an incoming packet with outermost label 299968, PE1 maps the packet to LSP PE2→PE3 and swaps its label to 300144.

• If PE1 receives an incoming packet with outermost label 299952, PE1 maps the packet to LSP PE3→PE2 and pops the label.

As you can see, RSVP’s Labelin and Labelout are forwarding-plane concepts. MPLS data packets are received by using Labelin and sent by using Labelout. In this sense, show rsvp session and show ldp database have an opposite interpretation of what input and output mean. Indeed, LDP’s input and output label database contain labels learned and advertised, respectively. But MPLS packets flow in the reverse direction!

Back to RSVP: let’s compare two similar (but not identical) commands in Junos.

juniper@PE1> show rsvp session ingress name PE1--->PE3
Ingress RSVP: 3 sessions
To           From         State Style Labelin Labelout LSPname
172.16.0.33  172.16.0.11  Up       FF       -   300560 PE1--->PE3

juniper@PE1> show mpls lsp ingress name PE1--->PE3
Ingress LSP: 3 sessions
To           From         State  P     ActivePath  LSPname
172.16.0.33  172.16.0.11  Up     *                 PE1--->PE3
Total 1 displayed, Up 1, Down 0

If the LSP is up and stable, the first command provides more information (namely, the labels). But, the second command is very useful in other situations: for example, if the LSP cannot be established due to a CSPF failure (no RSVP session), or if the LSP is being reoptimized or it has path protection (two RSVP sessions for the same LSP). These two commands are complementary.

Let’s move on to P1, a pure LSR or P-router (Example 2-42).

juniper@PE1> show rsvp session transit name PE1--->PE3
Transit RSVP: 6 sessions
To           From         State Style Labelin Labelout LSPname
172.16.0.33  172.16.0.11  Up       FF  300560        3 PE1--->PE3

juniper@P1> show route forwarding-table label 300560 table default
Routing table: default.mpls
MPLS:
Destination  Type RtRef Next hop         Index    NhRef Netif
300560       user     0 10.0.0.9   Pop     586     2    ge-2/0/6.0
300560(S=0)  user     0 10.0.0.9   Pop     588     2    ge-2/0/6.0

The forwarding table has two routes for label 300560, one for each value of the Bottom of Stack (BoS) bit in the external MPLS header. Which one is relevant for the CE1-to-BR3 traceroute packets? These arrive to P1 with just one MPLS label. In single-label stacks, the Top of Stack (ToS) label is at the same time the BoS label, so the BoS bit is set to 1 (S=1) and the first route applies.

As you saw in the LDP section, label 3 is a reserved label value called implicit null and it translates to pop the label. So, the IPv4 packet arrives unlabeled to PE3, and PE3 has the BGP route to reach BR3.

Let’s wrap up by looking at an RSVP-TE LSP traceroute.

juniper@PE1> traceroute mpls rsvp PE1--->PE3
  Probe options: retries 3, exp 7

  ttl    Label  Protocol  Address    Previous Hop   Probe Status
    1   300560  RSVP-TE   10.0.0.3   (null)         Success
  FEC-Stack-Sent: RSVP
  ttl    Label  Protocol  Address    Previous Hop   Probe Status
    2        3  RSVP-TE   10.0.0.9   10.0.0.3       Egress
  FEC-Stack-Sent: RSVP

  Path 1 via ge-2/0/4.0 destination 127.0.0.64

RSVP-TE signaling and MPLS forwarding in the IOS XR plane

Example 2-44 is an end-to-end traceroute from CE2 to BR4 that goes through the IOS XR plane (PE2, P2, PE4).

juniper@CE2> traceroute 192.168.20.4 source 192.168.10.2
traceroute to 192.168.20.4 (192.168.20.4) from 192.168.10.2 [...]
 1  PE2 (10.1.0.3)  2.833 ms  3.041 ms  2.441 ms
 2  P2 (10.0.0.5)  10.465 ms  8.480 ms  9.311 ms
     MPLS Label=24008 CoS=0 TTL=1 S=1
 3  PE4 (10.0.0.11)  8.461 ms  8.757 ms  7.982 ms
 4  BR4 (192.168.20.4)  9.109 ms  10.427 ms  9.248 ms

PE2 has a BGP route toward BR4’s loopback, and the BGP next hop of this route is PE4. The key here is the CEF entry for 172.16.0.44/32. Let’s have a look at it.

1     RP/0/0/CPU0:PE2#show cef 172.16.0.44
2     172.16.0.44/32, version 91, internal [...]
3      local adjacency 10.0.0.5
4      Prefix Len 32, traffic index 0, precedence n/a, priority 1
5        via 172.16.0.44, tunnel-te44, 4 dependencies [...]
6         path-idx 0 NHID 0x0 [0xa0db3250 0x0]
7         next hop 172.16.0.44
8         local adjacency
9          local label 24016      labels imposed {ImplNull}

The label operation for this LSP is as follows: push a real label, not implicit null. The real label does not show in line 9. Actually, seeing ImplNull there is a sign that everything is OK.

What is tunnel-te44? This is an explicitly configured interface, and it pushes an MPLS label with a value (24008) that matches traceroute’s output, as shown in Example 2-44 and in Example 2-46 (line 7):

1     RP/0/0/CPU0:PE2#show mpls traffic-eng tunnels name tunnel-te44 detail
2     Name: tunnel-te44  Destination: 172.16.0.44  Ifhandle:0x580
3       Signalled-Name: PE2--->PE4
4       Status:
5         Admin:    up Oper:   up   Path:  valid   Signalling: connected
6     [...]
7         Outgoing Interface: GigabitEthernet0/0/0/3, Outgoing Label: 24008
8         Path Info:
9           Outgoing:
10            Explicit Route:
11              Strict, 10.0.0.5
12              Strict, 10.0.0.11
13              Strict, 172.16.0.44
14        Resv Info:
15          Record Route:
16            IPv4 10.0.0.5, flags 0x0
17            IPv4 10.0.0.11, flags 0x0
18
19    RP/0/0/CPU0:PE2#show rsvp session tunnel-name PE2--->PE4
20    Type Destination Add DPort  Proto/ExtTunID  PSBs  RSBs  Reqs
21    ---- --------------- ----- --------------- ----- ----- -----
22    LSP4     172.16.0.44    44     172.16.0.22     1     1     0

Now, let’s look at the RSVP-TE session and forwarding entries on P2, the next hop LSR.

RP/0/0/CPU0:P2#show rsvp session tunnel-name PE2--->PE4 detail
SESSION: IPv4-LSP Addr: 172.16.0.44, TunID: 44, ExtID: 172.16.0.22
 Tunnel Name: PE2--->PE4 [...]
  RSVP Path Info:
   InLabel: GigabitEthernet0/0/0/0, 24008
   Incoming Address: 10.0.0.5
   Explicit Route:
     Strict, 10.0.0.5/32
     Strict, 10.0.0.11/32
     Strict, 172.16.0.44/32
   Record Route:
     IPv4 10.0.0.4, flags 0x0
   Tspec: avg rate=0, burst=1K, peak rate=0
  RSVP Resv Info:
   OutLabel: GigabitEthernet0/0/0/5, 3
   FRR OutLabel: No intf, No label
   Record Route:
     IPv4 10.0.0.11, flags 0x0

RP/0/0/CPU0:P2#show mpls forwarding labels 24008
Wed Nov 26 10:58:09.822 UTC
Local  Outgoing    Prefix     Outgoing     Next Hop   Bytes
Label  Label       or ID      Interface               Switched
------ ----------- ---------- ------------ ---------- --------
24008  Pop         44         Gi0/0/0/5    10.0.0.11  192900

And finally, following is an example of RSVP-TE LSP traceroute in IOS XR.

RP/0/0/CPU0:PE2#traceroute mpls traffic-eng tunnel-te 44

[...]
  0 10.0.0.4 MRU 1500 [Labels: 24008 Exp: 0]
L 1 10.0.0.5 MRU 1500 [Labels: implicit-null Exp: 0] 0 ms
! 2 10.0.0.11 1 ms     IPv4 10.0.0.4, flags 0x0

BGP-LU—policy and community scheme

Let’s clarify the usage of BGP communities in this example before jumping into the configuration details:

Servers

Advertise their own loopback addresses 172.16.3.11 and 172.16.3.22, respectively, as eBGP-LU routes with standard community CM-SERVER (65000:3).

Advertise the VM addresses 10.1.0.0/31 and 10.2.0.0/31, respectively, as vanilla eBGP routes with standard community CM-VM (65000:100).

Do not readvertise any eBGP route at all.

Leaf LSRs

Readvertise all of the eBGP-LU routes. Although they might advertise their local loopback, it is not required for the solution to work.

Spine LSRs

Advertise their own loopback addresses 172.16.1.1 and 172.16.1.2, respectively, as eBGP-LU routes with standard community CM-RR (65000:1).

Readvertise only those vanilla eBGP routes with community CM-VM.

Readvertise only those eBGP-LU routes with community CM-SERVER.

This careful community scheme is due to the fact that IOS XR keeps labeled and unlabeled IP routes in the same global table, so it is important not to readvertise labeled routes as unlabeled routes, or vice versa. Said differently, you need to pay special attention so that the SAFI=1 and SAFI=4 worlds remain independent.

As you can see, communities CM-VM and CM-SERVER play a key role in the route advertising flow. Conversely, the community CM-RR plays a subtler role that we’ll look at a bit later.

Junos—copying interface routes from inet.0 to inet.3

The role of the inet.0 and inet.3 routing tables in Junos has already been explained. Nonetheless, here’s a quick refresher:

• inet.0 is the global IPv4 routing table that populates the FIB and is typically populated by IP routing protocols (IGP, vanilla BGP, etc.).

• inet.3 is an auxiliary table for BGP next-hop resolution and is typically populated by MPLS signaling protocols (LDP, RSVP, SPRING-enabled IGP, etc.)

But is BGP-LU an IP routing protocol or an MPLS signaling protocol? Actually, it’s both. In Junos, you can configure it in two modes:

• BGP-LU installs prefixes in inet.0 and picks prefixes from inet.0 for further advertising. Optionally, explicit configuration might copy all (or a selection of) the prefixes into the inet.3 table, enabling BGP next-hop resolution for MPLS services.

• BGP-LU installs prefixes in inet.3 and picks prefixes from inet.3 for further advertising. Optionally, explicit configuration might copy all (or a selection of) the prefixes into the inet.0 table, enabling IPv4 forwarding with labeled next hop toward these prefixes.

This book uses the second method because it provides more flexibility. For example, with this model, a single BGP session can exchange prefixes from plain unicast and labeled unicast address families. Also, it is a good choice in terms of scalability because BGP-LU prefixes are not automatically installed in the FIB, relaxing the FIB load on low-end devices. The configuration is slightly more complex, though.

OK, let’s make BGP-LU work on the inet.3 routing table. The first step at Srv1 is to copy the local loopback address from inet.0 (where it resides by default) to inet.3, so BGP-LU can advertise it in later steps.

1     policy-options {
2         policy-statement PL-LOCAL-LOOPBACK {
3             term LOCAL-LOOPBACK {
4                 from interface lo0.0;
5                 then {
6                     metric 0;
7                     origin incomplete;
8                     community add CM-SERVER;
9                     accept;
10                }
11            }
12            term DIRECT {
13                from protocol direct;
14                then reject;
15            }
16        }
17        community CM-SERVER members 65000:3;
18    }
19    routing-options {
20        interface-routes {
21            rib-group inet RG-LOCAL-LOOPBACK;
22        }
23        rib-groups {
24            RG-LOCAL-LOOPBACK {
25                import-rib [ inet.0 inet.3 ];
26                import-policy PL-LOCAL-LOOPBACK;
27    }}}

A similar configuration is required on S1, just with a different community: CM-RR. The usage and relevance of all these communities is explained later. The service does not require L1’s loopback to be advertised; hence, this configuration is optional for L1 (you can use another community such as 65000:2 for L1 and L2 loopbacks).

A rib-group is like a template. It contains an ordered list of RIBs (line 25). The list begins with a single primary RIB (inet.0 here) where the to-be-copied prefixes originally reside, and then it lists one or more secondary RIBs (only inet.3 here) to which the prefixes must be copied. The rib-group is then applied to a protocol, or in this case, to the interface routes (lines 20 and 21) because the local loopback is one of them. As a result, the route 172.16.3.11/32 is copied from inet.0 to inet.3.

If no policy were specified, all the interface routes would be now in both inet.0 and inet.3. The policy (PL-LOCAL-LOOPBACK) performs a selective copy of the local loopback route only. Also, to provide consistency between Junos and IOS XR, the policy changes two route attributes:

• By default, there is no Multi Exit Discriminator (MED) in Junos, and in IOS XR it is set to zero. The policy sets the MED to zero for consistency across vendors.

• By default, the origin in Junos and IOS XR is igp and incomplete, respectively. The policy sets it to incomplete.

Note that the policy can only select or modify the routes installed in the secondary RIBs; it has no effect on the primary RIB. Also, if there were several secondary RIBs, the to rib knob makes the action specific to a subset of the secondary RIBs only.

Now that the configuration is applied, let’s have a look at the local loopback route.

juniper@Srv1> show route 172.16.3.11/32 detail

inet.0: 11 destinations, 12 routes (11 active, 0 holddown, 0 hidden)
172.16.3.11/32 (1 entry, 0 announced)
(...)
                Secondary Tables: inet.3

inet.3: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
172.16.3.11/32 (1 entry, 1 announced)
(...)
                Communities: 65000:3
                Primary Routing Table inet.0

The loopback IPv4 address is copied to inet.3 (secondary table), and the copied route has a new community. A similar check on other local link address (e.g., 10.0.10.8/31) only shows the route in inet.0, due to the policy constraints at the rib-group.

Junos—BGP-LU configuration

The next step is to assign a label to the route and advertise it with BGP-LU.

protocols {
    bgp {
        group eBGP-LU-65201 {
            family inet {
                labeled-unicast {
                    per-prefix-label;
                    rib {
                        inet.3;
                    }
                }
            }
            export PL-LOCAL-LOOPBACK;
            peer-as 65201;
            neighbor 10.0.0.1;
}}}

A similar configuration is required on S1. As for L1, it does not need any export policies, because Junos readvertises eBGP prefixes by default.

With the previous configuration (Example 2-71), Srv1 advertises its labeled loopback to L1, as you can see in the following example.

juniper@Srv1> show route advertising-protocol bgp 10.0.0.1
             172.16.3.11 detail

inet.3: 12 destinations, 12 routes (12 active, 0 holddown, ...)
* 172.16.3.11/32 (1 entry, 1 announced)
 BGP group eBGP-LU-65201 type External
     Route Label: 3
     Nexthop: Self
     Flags: Nexthop Change
     AS path: [65301] I
     Communities: 65000:3
     Entropy label capable

As expected, Srv1 assigns the implicit null label to enable PHP.

IOS XR—BGP-LU configuration

Following is the IOS XR configuration at Srv2.

1     route-policy PL-LOCAL-INTERFACES
2       if destination in (172.16.3.22/32) then
3         set community CM-SERVER
4         pass
5       endif
6     end-policy
7     !
8     route-policy PL-LOCAL-LOOPBACK
9       if community matches-any CM-SERVER then
10        pass
11      else
12        drop
13      endif
14    end-policy
15    !
16    route-policy PL-ALL
17      pass
18    end-policy
19    !
20    community-set CM-SERVER
21      65000:3
22    end-set
23    !
24    router bgp 65302
25     mpls activate
26      interface GigabitEthernet0/0/0/1
27     !
28     address-family ipv4 unicast
29      redistribute connected route-policy PL-LOCAL-INTERFACES
30      allocate-label all
31     !
32     neighbor 10.0.0.3
33      remote-as 65202
34      address-family ipv4 labeled-unicast
35       send-community-ebgp
36       route-policy PL-LOCAL-LOOPBACK out
37       route-policy PL-ALL in
38    !
39    router static
40     address-family ipv4 unicast
41      10.0.0.3/32 GigabitEthernet0/0/0/1

The local loopback is labeled and announced via eBGP-LU with the following actions:

• Redistribute the local loopback (lines 1 through 6, and 29) into BGP. The PL-LOCAL-INTERFACES policy is later extended during vanilla BGP configuration.

• Allocate labels to the unicast routes (line 30). It is recommended to apply a policy here in order to select the routes that require label allocation.

• Attach (line 36) a BGP outbound policy (lines 8 through 14) to only advertise the local loopback with the appropriate community. In this case, this community is set during route redistribution but it could have been set during route announcement, too.

A similar configuration is required on L2 and S2, just with different policies, as detailed in “BGP-LU—policy and community scheme”.

Sending communities over eBGP is turned on by default on Junos; you need to turn it on explicitly for IOS XR (line 35).

Furthermore, in IOS XR there is a default “reject all” inbound and outbound route policy applied to eBGP sessions. Explicit policies are required to accept and advertise eBGP routes (lines 16 through 18, 36 and 37).

When the previous configuration (Example 2-73) is applied, Srv2 advertises its loopback via eBGP-LU, as demonstrated in Example 2-74.

RP/0/0/CPU0:Srv2#show bgp ipv4 labeled-unicast advertised

172.16.3.22/32 is advertised to 10.0.0.3
[...]
  Attributes after outbound policy was applied:
    next hop: 10.0.0.2
    MET ORG AS COMM
    origin: incomplete  metric: 0
    aspath: 65302
    community: 65000:3

This is why a static route toward L1’s peer interface is configured (Example 2-73, lines 39 through 41). Thanks to that, Srv2 can resolve the eBGP-LU routes. Let’s see it.

RP/0/0/CPU0:Srv2#show cef 172.16.3.11
172.16.3.11/32, version 159, internal 0x1000001 [...]
 Prefix Len 32, traffic index 0, precedence n/a, priority 4
   via 10.0.0.3, 6 dependencies, recursive, bgp-ext [flags 0x6020]
    path-idx 0 NHID 0x0 [0xa1558774 0x0]
    recursion-via-/32
    next hop 10.0.0.3 via 24006/0/21
     local label 24004
     next hop 10.0.0.3/32 Gi0/0/0/1 labels imposed {ImplNull 24005}

Junos—copying eBGP-LU routes from inet.3 to inet.0

Back in Example 2-69 and Example 2-70, the local loopback route was copied from inet.0 to inet.3—this was so it could be advertised via eBGP-LU. Now, the process is the reverse: certain routes learned via eBGP-LU need to be copied from inet.3 to inet.0—so that they are reachable from a pure IPv4 forwarding perspective. The following example illustrates how to achieve it.

policy-options {
    policy-statement PL-RR-INET {
        term RR {
            from community CM-RR;
            then accept;
        }
        then reject;
    }
    community CM-RR members 65000:1;
}
routing-options {
    rib-groups {
        RG-RR-INET {
            import-rib [ inet.3 inet.0 ];
            import-policy PL-RR-INET;
        }
    }
}
protocols {
    bgp {
        group eBGP-LU-65201 {
            family inet {
                labeled-unicast {
                    rib-group RG-RR-INET;
}}}}}

As mentioned earlier, S1 and S2 are advertising their local loopback routes with community CM-RR. As a result of the copy in Example 2-76, these routes are installed in both inet.3 and inet.0 at Srv1, as shown in Example 2-77.

juniper@Srv1> show route 172.16.1.1/32 detail

inet.0: 11 destinations, 12 routes (11 active, 0 holddown, 0 hidden)
172.16.1.1/32 (1 entry, 0 announced)
(...)
                Communities: 65000:1
                Primary Routing Table inet.3

inet.3: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
172.16.3.11/32 (1 entry, 1 announced)
(...)
                Communities: 65000:1
                Secondary Tables: inet.0

Likewise, S1 is configured to copy all the routes with community CM-SERVER from inet.3 to inet.0. Here is a summary of the use of communities so far:

• Srv1 and Srv2 announce their loopbacks with community CM-SERVER.

• S1 and S2 announce their loopbacks with community CM-RR.

• Srv1 copies eBGP-LU routes with community CM-RR from inet.3 to inet.0.

• S1 copies eBGP-LU routes with community CM-SERVER from inet.3 to inet.0.

• Srv2 and S2 run IOS XR, which does not have a resolution RIB, so no route copy is needed.

At this point, IPv4 connectivity between PEs (Srv1, Srv2) and RRs (S1, S2) is guaranteed and multihop vanilla eBGP sessions can be established as in Figure 2-11’s dotted lines.

Junos—Vanilla eBGP configuration in IGP-less topology

Following is the service configuration at Srv1.

policy-options {
    policy-statement PL-eBGP-INET-OUT {
        term VM-INTERFACE {
            from interface ge-2/0/1.0;
            then {
                community add CM-VM;
                accept;
            }
        }
        then reject;
    }
    community CM-VM members 65000:100;
}
protocols {
    bgp {
        group eBGP-INET {
            multihop;
            local-address 172.16.3.11;
            export PL-eBGP-INET-OUT;
            neighbor 172.16.1.1 { peer-as 65101; }
            neighbor 172.16.1.2 { peer-as 65102; }
}}}

And Example 2-79 shows the service configuration at S1.

1     policy-options {
2         policy-statement PL-eBGP-INET-OUT {
3             term VM {
4                 from community CM-VM;
5                 then accept;
6             }
7             then reject;
8         }
9     }
10    protocols {
11        bgp {
12            group eBGP-INET {
13                multihop {
14                    no-nexthop-change;
15                }
16                local-address 172.16.1.1;
17                export PL-eBGP-INET-OUT;
18                neighbor 172.16.3.11 { peer-as 65301; }
19                neighbor 172.16.3.22 { peer-as 65302; }
20    }}}

Here is the logic behind this configuration: MPLS-enabled servers advertise the VM prefixes with community CM-VM. Service RRs—S1 and S2, which in this example happen to be spine LSRs, too—only reflect routes with the community CM-VM. This prevents any unexpected leaking between labeled and unlabeled routes. A very important piece of configuration is in line 14. By default, announcing a prefix to a different AS triggers a BGP next-hop attribute rewrite. This is not desired for service route reflection, and this is why no-nexthop-change is configured.

IOS XR—vanilla eBGP configuration in IGP-less topology

Following is the service configuration at Srv2.

route-policy PL-LOCAL-INTERFACES
  if destination in (172.16.3.22/32) then
    set community CM-SERVER
    pass
  endif
  if destination in (10.2.0.0/31) then
    set community CM-VM
    pass
  endif
end-policy
!
route-policy PL-eBGP-INET
  if community matches-any CM-VM then
    pass
  else
    drop
  endif
end-policy
!
community-set CM-VM
  65000:100
end-set
!
router bgp 65302
 neighbor-group eBGP-INET
  ebgp-multihop 255
  update-source Loopback0
  address-family ipv4 unicast
   send-community-ebgp
   redistribute connected route-policy PL-LOCAL-INTERFACES
   route-policy PL-eBGP-INET out
   route-policy PL-eBGP-INET in
 !
 neighbor 172.16.1.1
  remote-as 65101
  use neighbor-group eBGP-INET
 !
 neighbor 172.16.1.2
  remote-as 65102
  use neighbor-group eBGP-INET

The logic is similar to Junos. Only the VM routes, flagged with community CM-VM, are advertised as unlabeled IPv4 prefixes toward the service RRs.

Remember that the PL-LOCAL-INTERFACES policy is in control of the redistribution of interface routes into BGP (Example 2-73, line 29). The local loopback and the VM route are flagged with community CM-SERVER and CM-VM, respectively. With this entire configuration in place, the local loopback is only distributed via eBGP-LU, whereas the VM route is only distributed via vanilla BGP.

The configuration at S2 is similar, with two differences. First, it reflects only remote VM routes, so it does not have any local VM route to announce. Second, it must not change the BGP next-hop attribute, so the extra configuration is required.

router bgp 65302
 neighbor-group eBGP-INET
  address-family ipv4 unicast
   next-hop-unchanged

SPRING Anycast

Anycast segments allow sharing the same SID (for a given so-called anycast prefix) among a group of devices. Several drafts cover two possible scenarios: if all of the anycast nodes advertising a given SID use the same SRGB, or if they use different SRGBs. Further details are beyond the scope of this book.