STP

Lots of things in networking are based on trees, branches, and leaves. STP is no exception. My intent is not to give you a book or chapter that tells you everything you need to know about STP, but rather to review the critical components and analyze it from the practical perspective.

You, like many other network engineers, might consider STP to be one of those theory things that you never have to worry with in the practical world. However, the truth is STP accounts for more than half of the issues with configuring, maintaining, and supporting campus networks.

STP is so critical because it is a dynamic loop prevention protocol for Layer 2. Time to live (TTL) does not exist at Layer 2; STP is what prevents frames from looping endlessly around the network. I know you would never intentionally create a bridge loop at Layer 2, but redundancy is a good thing to build in to your environment to eliminate single points of failure. Redundancy may lead to bridge loops and bridge loops may lead to broadcast storms. This is not to say redundancy is bad, but a mechanism such as STP keeps it under control.

Broadcast storms are feedback loops that occur in both directions. Want to see one? Just create a physical loop in your topology and turn off STP with a command such as set spantree disable [vlan#]. Do not do this in a production environment.

There is a DEC version and IEEE version of STP that, although similar, are not compatible. This chapter focuses on only the IEEE version because that is what many of the Layer 2 Catalyst switches support. However, Cisco routers and Layer 3 switches support multiple versions. For example, the following command output shows you how to change this parameter on an IOS-based switch if required to do so:

sw3512XL(config)#spanning-tree protocol ?
  dec   Digital spanning-tree protocol
  ibm   IBM spanning-tree protocol
  ieee  IEEE Ethernet spanning-tree protocol

Bridges dynamically form a tree of the physical topology by exchanging bridge protocol data units (BPDUs). At first all ports send BPDUs out every two seconds to build the tree. The best ports are forwarding and the alternate ports are blocking as to not form bridge loops. See the STP decision criteria in the following section. STP convergence requires the election of a root bridge, root ports, and designated ports.

STP Decision Criteria

Many decisions are made in STP:

• Who is the root bridge?— Lowest BID. (I bet many of you can relate to that terminology.)

• How far away is the root bridge?— Lowest path cost (highest bandwidth) to root bridge.

• Who sent this BPDU?— Lowest sender BID.

• What port did this BPDU come from?— Lowest port ID.

It is helpful to think of the root bridge as a wagon wheel with spokes going out to the other switches. Just like many other things in life, the lowest BID wins again. BID in the sense of STP is a concatenation of the bridge priority and the 6-byte MAC address. By default the 2-byte priority field is 32,768 decimal or 8000 hex, so the MAC address is really the tiebreaker.

Every non-root bridge must select a root port, which is the port closest to the root bridge. This closeness is measured by the root path cost, which is the cumulative cost of all links leading to the root bridge. The STP costs are incremented as BPDUs are received on a port.

Costs are associated with various LAN segments. The original IEEE 802.1D specification didn't really take into consideration Gigabit Ethernet, ATM, and 10 Gigabit Ethernet technologies. The original specification was a linear value derived from the formula 1000 Mbps divided by the bandwidth of a segment. An example for FDDI or Fast Ethernet is as follows: 1000/100 = 10. Table 6-5 illustrates the latest IEEE path costs.

^[*] Changes in bandwidth dictate software changes. This only ever happened once to go from a linear to a nonlinear scale to account for the old and new schemes to interoperate.

Every segment elects a designated port based on the lowest cumulative root path cost (best) to the root bridge. Now that I have discussed root bridge, non-root bridge, root port, and designated port, I want to mention nondesignated ports or ports that are in a blocking state. Blocking is not truly blocking because the port must still listen for BPDUs unless of course you take advantage of the new improved Rapid Spanning Tree Protocol (RSTP). Figure 6-6 displays the five STP states and associated timers.

All ports start out in either a blocking or a disabled state and work their way to a forwarding state. The disabled state is the one most people forget; it is when the port is administratively shut down. Ports are listening for BPDUs while they are in the blocking state in case there is a change in topology. The port transitions to a listening state where it is sending and receiving BPDUs to determine the current active topology. By default, a 15-second forward delay timer is associated with the listening state. The designated and root ports then progress to the learning state. There is then another 15-second forward delay time when tables are being built but still no user data has passed. Next the port transitions to the most wanted forwarding state, where it actually sends and receives user data. All ports that are not in a forwarding state at this point are in a blocking state. This entire process might sound slow and bad and make you want to turn STP off. Do not do it! (However, there are ways to tweak STP that I mention at the end of the section.) From a troubleshooting perspective, the issues are whether anything breaks STP and causes a forwarding loop, and the impact of the delays on user traffic. Techniques such as RSTP, portfast, uplinkfast, and so on improve on the 20-year-old approach. See the practical STP and RSTP sections.

As you have seen in previous chapters, tools such as Sniffer Pro are quite helpful to understand the interworkings of things. However, switches are not quite as free flow as hubs are when it comes to monitoring ports with protocol analyzers. Port monitoring, therefore, is a topic of necessary discussion.

Port Monitoring

Unlike hubs flooding to all ports, switches learn Layer 2 addresses so that they can filter and send only unicast traffic to the correct recipient. From a Sniffer standpoint, you will capture the broadcast-type frames but miss a lot of unicast frames unless you intentionally monitor the right ports. Essentially you must select a port to receive or mirror the traffic you want to capture. This is called the SPAN port and has nothing to do with STP. The source SPAN port(s) or ingress ports are where traffic enters the switch, and the destination SPAN port or egress port is where the network analyzer is connected.

Configuring port monitoring on an IOS-based switch is as simple as going to the destination interface where the Sniffer is located and getting into interface configuration mode. Turn on port monitoring for each individual port you want to act as a source. If you just issue the port monitor command, all ports in the same VLAN are monitored. Use show port monitor to see the results.

On a CatOS-based switch, you need to turn on SPAN, the Catalyst switched port analyzer. It enables you to mirror the data from one port, trunk, or VLAN to another port so that you can monitor it. This command turns STP off on the mirrored port. Use set span help for assistance. For example, set span 1/1 2/1 both takes the transmit (Tx) and receive (Rx) traffic source of port 1/1 and mirrors it to port 2/1 for monitoring. The inpkts option determines whether frames emitted by the SPAN collector are processed or ignored. Use show span to see the results. Use commas and dashes to specify multiple ports.

Port monitoring and SPAN are one and the same. It just depends whether you are on an IOS-based switch or a CatOS-based switch. Look at the Figure 6-3 and contemplate how you can monitor the traffic using a tool such as Sniffer Pro. Although I wanted to briefly mention port monitoring, the real goal at this point in time is to analyze BPDUs.

Analyzing BPDUs

Analyze the configuration BPDU Sniffer capture in Figure 6-7. You will capture your own data in the Trouble Tickets.

Figure 6-7. Configuration BPDU

[View full size image]

Configuration BPDUs flow from the root bridge to others. A BPDU is an IEEE 802.3 frame with 802.2 LLC SAP header. The Type is 0 because it is a configuration BPDU rather than a topology change BPDU. The root bridge at the time of this capture was the CatOS sw2900 that has a MAC that ends in 1400. The default priority is 8000 hex. The root path cost is 10. The sending BID is from the sw1900 that has a MAC that ends in 7680. The max age is 20 seconds, the hellos are every 2 seconds, and the forward delay is 15 seconds as discussed earlier. Compare Figure 6-7 to the following headers:

• 802.3 header:

  - Source MAC is the Catalyst port that sent the BPDU.

  - Destination MAC is the IEEE 802.1D STP multicast address 0180c2000000.

• LLC header:

  - DSAP and SSAP are both 0x42 for STP.

• The configuration BPDU:

  - Protocol ID Always 0.
  
  

  - Version Always 0.
  
  

  - Type 0 for configuration BPDU; 0x80 for topology change notification (TCN BPDU).
  
  

  - Flags Topology change or acknowledgement.
  
  

  - Root BID Concatenation of priority and MAC to form the bridge ID for the root bridge.
  
  

  - Root Path Cost Cumulative cost of links toward the root bridge.
  
  

  - Sender BID Bridge ID of the sender of the BPDU.
  
  

  - Port ID Unique value for port that sent the BPDU.
  
  

  - Message Age Time since the root bridge first created this BPDU. This is not max age. Think of it like a reverse TTL where 1 is added at every bridge hop.
  
  

  - Max Age Time period to save BPDU information.
  
  

  - Hello Time Time interval between BPDUs.
  
  

  - Forward Delay For listening and learning states.
  
  

STP convergence time is 20 max age + 30 for the two 15-second forward delays, for a total of 50 seconds. Tools such as Sniffer Pro and the CatOS or IOS spanning-tree commands in Table 6-7 or even more automated tools such as the CiscoWorks Campus Manager can lend a hand in drawing a very informative Layer 2 STP diagram to assist you in supporting STP issues.

TCN BPDUs are one way to speed up this convergence. They flow from the spoke bridges to the center of the wheel root bridge to let the root bridge know that the topology has changed. You analyze TCNs in more detail in the Trouble Tickets.

Tuning STP in a Practical Environment

As you can see, STP is quite complex, but necessary to avoid Layer 2 broadcast storms. Think of it as the Layer 2 “TTL.” The following items and commands are suggestions for tuning STP in a practical environment:

• Create a design hierarchy including Layer 3 switches rather than a flat network.

• Understand how STP works, particularly on your network. Draw pictures of what you have and what you will have after a failure. Use tools such as the Campus Manager CWSI STP mapping tool to get a good Layer 2 picture.

• Plan root bridge placement. Centrally locate them close to heavy traffic destinations and don't choose your lowest caliber switch. The set spantree priority priority [vlan] command enables you to adjust the priority of a switch on a VLAN basis, and with Cisco there is one instance of STP per VLAN. Alternatively, set spantree root is a macro that places its results in the configuration file.

• Set more than one root bridge in case of failure. The backup acts like a hot standby until needed. For example, type set spantree priority 100 1 on one switch to make it the primary. Use set spantree priority 200 1 on another switch to configure it as the backup root bridge. VLAN 1 is the default, and therefore is really not necessary in this example. However, you must specify the VLAN if different from VLAN 1.

• Do not use set spantree portvlanpri for back-to-back switch load balancing. Use Fast or Gigabit EtherChannel instead.

• Use port/VLAN cost load balancing in situations where you can't use the root bridge form of load balancing. For example: set spantree portvlancost 2/1 cost 2000 10 assigns a path cost of 2000 to port 2/1 for VLAN 10 on the trunk.

• Turn on portfast and disable Port Aggregation Protocol (PAgP) for hosts. Some PCs boot faster than the time it takes STP to go through its blocking, listening, learning, and forwarding states. However, you can't turn off STP for an individual port. Portfast essentially skips the steps prior to the forwarding and immediately starts the port in a forwarding state when it first initializes. Practical places to implement portfast include Microsoft hosts looking for a domain controller or DHCP server or Novell clients that never seem to find the login screen upon boot up. In most cases it is not necessary to enable portfast on servers because they rarely reboot. The set spantree portfast 2/1-12 enable command sets portfast on for ports 1 through 12 on module 2. If port 1 connects this switch to another, however, you do not want portfast on for that port, so you can disable it with set spantree portfast 2/1 disable. Use show spantree 1 to see the results (assuming this is VLAN 1).

  CAUTION

  Use portfast only when connecting a single host to an access port, otherwise you are asking for a network loop.

  
  

• BPDUguard assists with helping you find invalid configurations of portfast. In a valid configuration, the portfast ports do not receive BPDUs. In an invalid configuration, they do and STP places the port in a blocking state. Enable BPDUguard with set spantree portfast bpdu-guard enable so that STP automatically shuts the portfast port down if it receives a BPDU to recalculate STP.

• Enable uplinkfast on wiring closet switches (STP leaves) but not in the core for root port optimization for directly connected failed links. Uplinkfast is for dual-homed access layer switches or any similar topology. Don't do this on root bridges or transit switches. Enable uplinkfast with set spantree uplinkfast enable and view your changes with show spantree uplinkfast.

• Enable backbonefast using set spantree backbone fast enable on every switch in your network to assist with indirect failures and optimize max age.

• Verify such features as BPDUguard, uplinkfast, and backbonefast with show spantree summary.

• With EtherChannel it is normally best practice to set the ports to desirable. The following command sets port 2/1 to desirable: set port channel 2/1 desirable.

• Take advantage of the IEEE RSTP/Multiple STP (MSTP) where possible. See the “Rapid STP (RSTP)” section.

The commands for tuning STP in a practical environment are all given in the CatOS syntax. However, you can compare the IOS equivalent commands in Table 6-7.

Rapid STP (RSTP)

Although VLANs and trunking are the topic of the next chapter, they are certainly a relative topic when it comes to STP. Cisco enhanced the original IEEE 802.1Q standard with features such as portfast, uplinkfast, and backbonefast, per VLAN STP (PVST+), and Multiple Instance STP (MISTP), but they are proprietary. Although not a standard yet as of this writing, RSTP IEEE 802.1w is available in newer versions of the IOS and is backward compatible with 802.1D STP. It was first implemented in CatOS 7.1 and IOS 12.1(11)EX as Multiple STP (MSTP). CatOS 7.5 and IOS 12.1(13)E offer RSTP where the switch runs an RSTP instance on each VLAN, like the Cisco PVST. Compare the 802.1D and 802.1w port states in Table 6-6.

The end result is the same for STP or RSTP, but the overall STP convergence is much improved with the latter (50 seconds to 1 second). The main reason is the way topology changes are detected and propagated. The initiator of the topology change directly forwards the change throughout the network instead of waiting for the root bridge to do so. Although the inherent fast convergence benefits are lost when your network includes legacy bridges, STP and RSTP are compatible.

MSTP IEEE 802.1s uses RSTP to provide very fast convergence, as well as to group VLANs into an instance of STP to provide multiple forwarding paths and load balancing. A big advantage is that a failure in one instance doesn't affect another STP instance. Although not a standard yet as of this writing, MSTP was released in CatOS version 7.1 via the following command: set spantree mode MST. By default, all VLANs are in instance 0 unless you use the set spantree MST 20 vlan 1-6,12 command. This puts ports 1 through 6 and 12 in MST instance 20. To commit the changes, use set spantree MST config commit. In IOS there is a separate mode for MSTP configuration. When you are in global config mode, enter MSTP config with spanning-tree mst configuration, where you enter the instance ID and VLAN range.

Cisco switches have certainly evolved over the years. They are easy to configure, maintain, and support when you maintain a good balance of hierarchical routing and switching, not flat networks. As discussed in the “STP” and “Rapid STP (RSTP)” sections, Cisco has their own improvements to the current STP standards, and IEEE has a few in the works. You should also consider the idea of using point-to-point links as Layer 3 networks for fast failover rather than the 50-second STP convergence.

As mentioned in the early chapters of this book, the Core, Distribution, and Access Layers describe Cisco's three-layer tiered hierarchy in which switches and routers are both important components. Many times you will see the following acronyms in use for the layers:

• IDF— Intermediate distribution frame (Access Layer)

• MDF— Main distribution frame (Distribution Layer)

• MDF to MDF— Core Layer

The layered hierarchy is more of a design focus, and obviously a good design helps things run well and lessens support issues. However, next I want to focus more on the evolution of the software on Cisco switches to assist you with analyzing common issues.