Controlling Route and Cache Invalidation
A traditional requirement of BGP was resetting the neighbors' TCP connection in order for a policy change to take effect (clear ip bgp [* | address | peer-group]). Clearing the sessions in this manner restarts the neighbor negotiations from scratch, invalidates the associated portions of the IP forwarding cache, and causes a major impact on the operation of live networks.
The reasoning for this is that, as discussed in Chapter 6, routes learned from a peer are initially populated in the peer's Adj-RIB-In. They are then passed to the Input Policy Engine, modified accordingly, and presented to the BGP decision process. Because an unmodified copy of the route (what was initially stored in the Adj-RIB-In) is usually not available, but is required in order to implement the new policy, one of a few things can happen:
-
The source of the BGP route could manually trigger readvertisement of the route.
-
The entire TCP session could be reset.
-
BGP soft reconfiguration can be used to store the peer's Adj-RIB-In in memory.
-
BGP Route Refresh could be used to ask the peer to readvertise its respective Adj-RIB-Out.
As for 1, it's unlikely in the real world that you'll have access to the source router. As for 2, this is the brute-force approach, and it would trigger a great deal of unnecessary instability. As for soft reconfiguration, it's a very nice approach, but it consumes a significant amount of memory. BGP Route Refresh, a very new solution, is by far the most elegant. The following sections discuss 3 and 4.
BGP Soft Reconfiguration
BGP soft reconfiguration is one approach that lets policies be configured and activated without resetting the BGP TCP session. This allows new policies to take effect without significantly affecting the network. Soft configuration can be applied in two different ways—inbound and outbound. Use the following EXEC command:
clear ip bgp [⋆ | address | peer-group][soft [in|out]]
Outbound Soft Reconfiguration
Whenever outbound soft reconfiguration is applied, the new policies are automatically triggered, and appropriate updates (from the Adj-RIB-Out) are generated to enforce the new policy. Outbound soft reconfiguration requires no additional memory. Use the following EXEC command:
clear ip bgp [⋆ | address | peer-group] soft out
Inbound Soft Reconfiguration
Inbound soft reconfiguration is a bit more involved. All inbound updates (the Adj-RIB-In) from the specified peer are stored in memory unmodified. When a new policy is applied and activated, the stored Adj-RIB-In is again presented to the Input Policy Engine. An additional router subcommand is needed to configure inbound soft reconfiguration:
neighbor {address | peer-group} soft-reconfiguration inbound
This command is required to start storing the received updates for the specified peer or peer group. The EXEC command used to do the inbound reconfiguration is as follows:
clear ip bgp [* | address | peer-group] soft in
To avoid the memory overhead needed for the inbound soft reconfiguration, the same outcome could be achieved by doing an outbound soft reconfiguration at the other end of the connection, which would force readvertisement of the subsequent Adj-RIB-Out.
If the in/out option is not specified (clear ip bgp [* | address | peer-group] soft), both inbound and outbound soft reconfiguration will be applied.
The following example demonstrates the difference between clearing a BGP session between two routers without and with the soft configuration BGP feature. While the session is cleared, an output log will be displayed to show the actual session being established and the route updates being exchanged.
Referring still to Figure 12-14, Example 12-83 shows how RTA is configured to send its updates to RTC with a metric of 5000.
Example 12-83. Inbound Soft Reconfiguration: RTA Configuration
router bgp 65050 no synchronization bgp confederation identifier 3 network 172.16.220.0 mask 255.255.255.0 network 172.16.70.0 mask 255.255.255.0 neighbor 172.16.20.1 remote-as 1 neighbor 172.16.20.1 soft-reconfiguration inbound neighbor 172.16.20.1 filter-list 10 out neighbor 172.16.20.1 route-map setmetric out neighbor 172.16.70.2 remote-as 65050 no auto-summary ip as-path access-list 10 permit ^$ route-map setmetric permit 10 set metric 5000 |
Note the neighbor 172.16.20.1 soft-reconfiguration inbound command in Example 12-83. This is needed only if clearing the session needs to take effect on the inbound—that is, in case you have no control over the neighbor router to clear the session on the outbound.
For this information to take effect, the BGP session would have to be cleared between the two routers, as demonstrated in Example 12-84.
Example 12-84. Inbound Soft-Reconfiguration: Clearing the BGP Session Between Two Routers
RTA#clear ip bgp 172.16.20.1
BGP: 172.16.20.1 reset requested
BGP: no valid path for 192.68.11.0/24
BGP: 172.16.20.1 reset by 0x27B740
BGP: 172.16.20.1 went from Established to Idle
BGP: nettable_walker 192.68.11.0/255.255.255.0 no best path selected
BGP: 172.16.20.1 went from Idle to Active
BGP: 172.16.70.2 computing updates, neighbor version 21, table version 23,
starting at 0.0.0.0
BGP: 172.16.70.2 send UPDATE 192.68.11.0/24 -- unreachable
BGP: 172.16.70.2 1 updates enqueued (average=27, maximum=27)
BGP: 172.16.70.2 update run completed, ran for 0ms, neighbor version 21,
start version 23, throttled to 23, check point net 0.0.0.0
BGP: scanning routing tables
BGP: 172.16.20.1 went from Active to OpenSent
BGP: 172.16.20.1 went from OpenSent to OpenConfirm
BGP: 172.16.20.1 went from OpenConfirm to Established
BGP: 172.16.20.1 computing updates, neighbor version 0, table version 23,
starting at 0.0.0.0
BGP: 172.16.20.1 send UPDATE 172.16.25.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.30.0/24, next 172.16.20.2, path (65060)
BGP: 172.16.20.1 send UPDATE 172.16.50.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.60.0/24, next 172.16.20.2, path (65060)
BGP: 172.16.20.1 send UPDATE 172.16.70.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.90.0/24, next 172.16.20.2, path (65060)
BGP: 172.16.20.1 send UPDATE 172.16.65.0/26, next 172.16.20.2, path (65060)
BGP: 172.16.20.1 send UPDATE 172.16.112.0/24, next 172.16.20.2, path
BGP: 172.16.20.1 send UPDATE 172.16.220.0/24, next 172.16.20.2, path
BGP: 172.16.20.1 send UPDATE 192.68.222.0/24, next 172.16.20.2, metric 5000,
path 3 2
BGP: 172.16.20.1 4 updates enqueued (average=58, maximum=68)
BGP: 172.16.20.1 update run completed, ran for 24ms, neighbor version 0,
start version 23, throttled to 23, check point net 0.0.0.0
BGP: 172.16.20.1 rcv UPDATE about 192.68.11.0/24, next hop 172.16.20.1,
path 1 metric 2000
BGP: 172.16.20.1 rcv UPDATE about 192.68.222.0/24, next hop 172.16.20.1,
path 1 2 metric 2000
BGP: 172.16.20.1 rcv UPDATE about 172.16.25.0/24 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.30.0/24 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.50.0/24 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.60.0/24 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.70.0/24 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.90.0/24 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.65.0/26 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.112.0/24 -- denied
BGP: 172.16.20.1 rcv UPDATE about 172.16.220.0/24 -- denied
BGP: nettable_walker 192.68.11.0/255.255.255.0 calling revise_route
BGP: revise route installing 192.68.11.0/255.255.255.0 -> 172.16.20.1
BGP: 172.16.70.2 computing updates, neighbor version 23, table version 24,
starting at 0.0.0.0
BGP: NEXT_HOP part 1 net 192.68.11.0/24, neigh 172.16.70.2, next 172.16.20.1
BGP: 172.16.70.2 send UPDATE 192.68.11.0/24, next 172.16.20.1, metric 2000, path 1
BGP: 172.16.70.2 1 updates enqueued (average=59, maximum=59)
BGP: 172.16.70.2 update run completed, ran for 4ms, neighbor version 23,
start version 24, throttled to 24, check point net 0.0.0.0
BGP: 172.16.20.1 rcv UPDATE about 172.16.25.0/24 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.30.0/24 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.50.0/24 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.60.0/24 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.70.0/24 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.90.0/24 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.65.0/26 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.112.0/24 -- withdrawn
BGP: 172.16.20.1 rcv UPDATE about 172.16.220.0/24 -- withdrawn
BGP: 172.16.20.1 computing updates, neighbor version 23, table version 24,
starting at 0.0.0.0
BGP: 172.16.20.1 update run completed, ran for 0ms, neighbor version 23,
start version 24, throttled to 24, check point net 0.0.0.0
BGP: scanning routing tables
|
Note in the output in Example 12-84 how much overhead is caused by actually killing the TCP session between the two routers and starting over from scratch.
Note
The overhead introduced would also likely be apparent if you viewed show process cpu or show ip bgp sum output, which would reveal a high CPU load and a significant change in the number of routes in the BGP table. You would also notice a large number of BGP routes sent and received during this time to all the peers.
The log will show that the BGP peer session is reset, and then the neighbor election goes from Idle to Established, and then the actual routing updates will flow.
In Example 12-85, the same session is cleared by using the soft configuration feature. Note how the metric 5000 was sent without killing the BGP session, and the overhead is much smaller.
Example 12-85. Inbound Soft Reconfiguration: Clearing the BGP Session Between Two Routers with Soft Configuration
RTA#clear ip bgp 172.16.20.1 soft out
BGP: start outbound soft reconfiguration for 172.16.20.1
BGP: 172.16.20.1 computing updates, neighbor version 0, table version 24,
starting at 0.0.0.0
BGP: 172.16.20.1 send UPDATE 172.16.25.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.30.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.50.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.60.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.70.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.90.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.65.0/26, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.112.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 172.16.220.0/24, next 172.16.20.2, metric 5000, path 3
BGP: 172.16.20.1 send UPDATE 192.68.11.0/24 -- unreachable
BGP: 172.16.20.1 send UPDATE 192.68.222.0/24, next 172.16.20.2, metric 5000,
path 3 2
BGP: 172.16.20.1 5 updates enqueued (average=52, maximum=68)
BGP: 172.16.20.1 update run completed, ran for 24ms, neighbor version 0,
start version 24, throttled to 24, check point net 0.0.0.0
BGP: scanning routing tables
|
BGP Route Refresh
BGP Route Refresh capability is enabled by BGP Capabilities (discussed in Chapter 5). During the establishment of the BGP connection, the peers advertise support of the Route Refresh capability. If they support it, they can dynamically request that the peer readvertise its Adj-RIB-Out (which, in reality, is what happens when outbound soft reconfiguration is performed). Because this requires no additional memory for storing the information, it's more efficient than using soft reconfiguration, and it doesn't cause the extra route flapping that a BGP session reset would. When the requestor receives the Adj-RIB-Out, it presents it to the Input Policy Engine (and the new policy).
In the output in Example 12-86, support of the route refresh capability can be verified by examining the Neighbor capabilities portion of the show ip bgp neighbor x.x.x.x output.
Example 12-86. BGP Route Refresh Verification
r1#show ip bgp n 1.1.2.2
BGP neighbor is 1.1.2.2, remote AS 2, external link
BGP version 4, remote router ID 3.3.3.1
BGP state = Established, up for 2w0d
Last read 00:00:15, hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received
Address family IPv4 Unicast: advertised and received
Received 20674 messages, 0 notifications, 0 in queue
Sent 20675 messages, 0 notifications, 0 in queue
Route refresh request: received 1, sent 2
Minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 6, neighbor version 6
Index 1, Offset 0, Mask 0x2
NEXT_HOP is always this router
Community attribute sent to this neighbor
1 accepted prefixes consume 36 bytes
Prefix advertised 4, suppressed 0, withdrawn 0
Connections established 1; dropped 0
Last reset never
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Local host: 1.1.2.1, Local port: 179
Foreign host: 1.1.2.2, Foreign port: 11000
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x49ED7420):
Timer Starts Wakeups Next
Retrans 20675 0 0x0
TimeWait 0 0 0x0
AckHold 20674 19530 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
iss: 1081723559 snduna: 1082116474 sndnxt: 1082116474 sndwnd: 15567
irs: 1087514066 rcvnxt: 1087906928 rcvwnd: 15605 delrcvwnd: 779
SRTT: 301 ms, RTTO: 621 ms, RTV: 9 ms, KRTT: 0 ms
minRTT: 4 ms, maxRTT: 600 ms, ACK hold: 200 ms
Flags: passive open, nagle, gen tcbs
Datagrams (max data segment is 1460 bytes):
Rcvd: 39791 (out of order: 0), with data: 20674, total data bytes: 392861
Sent: 40473 (retransmit: 0), with data: 20674, total data bytes: 392914
|
Note that route refresh can be supported by only one side of the connection. You can see this by viewing the advertised and/or received output following the Route Refresh: entry.
Example 12-87 shows what you would do to request that a peer readvertise its Adj-RIB-Out.
Example 12-87. Forcing a Peer to Readvertise Adj-RIB-Out
r1# clear ip bgp 1.1.2.2 soft in
r1#
2w0d: BGP: 1.1.2.2 sending REFRESH_REQ for afi/safi: 1/1
2w0d: BGP: 1.1.2.2 send message type 128, length (incl. header) 23
2w0d: BGP: 1.1.2.2 send message type 4, length (incl. header) 19
2w0d: BGP: 1.1.2.2 rcv message type 4, length (excl. header) 0
|
As you can see from the associated debug output in Example 12-87, this triggers a route refresh request to the peer and a subsequent readvertisement of the peer's Adj-RIB-Out.
As displayed in the show ip bgp neighbor output of Example 12-86, there are also counters available to track the number of route refresh requests sent to and received from the peer.