10.3. SYSTEM BRINGUP EXAMPLE

This example shows the process employed by a system that offers two host processors, a single switch and two resource board subsystems. The hosts (A and B) are redundant with each other. Host B is assigned a higher priority than Host A. To better demonstrate the enumeration interaction between the host devices, Host A begins the enumeration process first. Figure 10.1 provides a representation of the system connectivity and topology.

The following additional assumptions are made about the example system: system Host A is preloaded with device ID 0×00 and system Host B is preloaded with device ID 0×01. System Bringup advances through a series of time steps. The time steps are not meant to be of equal length, but they would occur in the specified order. The following time steps would occur as the enumeration process proceeded:

• T+0: Host A begins RapidIO enumeration.

• T+1: Host B begins RapidIO enumeration and Host A continues RapidIO enumeration.

• T+2: Host B discovers another host in the system (Host A) and waits.

• T+3: Host A discovers a higher-priority host in the system (Host B) and retreats.

• T+4: Host B assumes sole enumeration of the system.

• T+5: Host B enumerates the PE on switch port 1.

• T+6: Host B enumerates the PEs on switch ports 2, 3 and 4.

• T+7: System enumeration is complete.

The following describes the actions taken during each time slot in more detail.

Time T: Host A attempts to acquire the lock from its host base device ID lock CSR by writing 0×00 to the CSR. Host A confirms it has acquired the lock when it reads the value of 0×00 (the host device ID) from the lock CSR. Host A continues by reading the processing element features CAR and adding the information from the CAR to its RapidIO device database. Host A updates its base device ID CSR with the host device ID (0×00).

Time T+1: Host B attempts to acquire the lock from its host base device ID lock CSR by writing 0×01 to the CSR. Host B confirms it has acquired the lock when it reads the value of 0×01 (the host device ID) from the lock CSR. Host B continues by reading the processing element features CAR and adding the information from the CAR to its RapidIO device database. Host B updates its base device ID CSR with the host device ID (0×01).

Host A begins neighbor enumeration. It attempts to acquire the lock from the host base device ID lock CSR of the board interconnect switch. A maintenance write of the host device ID (0×00), the destination device ID (0×FF), and the hop count (0) is issued for the lock CSR. Host A confirms it has acquired the lock when it reads the value of 0×00 (the host device ID) from the lock CSR.

Time T+2: Host B begins neighbor enumeration. It attempts to acquire the lock from the host base device ID lock CSR of the board interconnect switch. A maintenance write of the host device ID (0×01), the destination device ID (0×FF), and the hop count (0) is issued for the lock CSR. However, after Host B issues a maintenance read from the lock CSR it finds that the device was already locked by host device ID 0×00. Because Host B has a higher priority than the current lock holder (0×01 is greater than 0×00), Host B spins in a delay loop and repeatedly attempts to acquire the lock.

Time T+3: Host A continues neighbor enumeration. It issues a maintenance read cycle to the device identity CAR of the board interconnect switch and looks for a matching entry in the device database. Device configuration continues because no match is found (Host A has not enumerated the device). Host A reads the source operations and destination operations CARs for the device. It is determined that the device does not support read/write/atomic operations and does not require a device ID. Host A reads the processing element feature CAR for the device and determines that it is a switch element.

Because the device is a switch, Host A reads the switch port Information CAR and records the device identity in the switch database. Next, Host A adds a set of entries to the switch's routing table. For each previously discovered device ID, an entry is created containing a target ID (0×FF), hop count (0), and the route port number (from the switch port information CAR). The switch database is updated with the same routing information. Host A reads the port error status CSR for switch port 0, verifying that it is possible for the port to have a neighbor PE. An entry is created in the switch's routing table containing target ID (0×FF), hop count (0), and the route port number (0).

Host A continues neighbor enumeration, using a hop count of 1. It attempts to acquire the lock from the host base device ID lock CSR of the neighbor PE on port 0. A maintenance write of the host device ID (0×00), the destination device ID (0×FF), and the hop count (1) is issued for the lock CSR. However, after Host B issues a maintenance read from the lock CSR it finds that the device was already locked by host device ID 0×01. Because Host A has a lower priority than the current lock holder (0×00 is less than 0×01), Host A retreats. It begins the process of backing out all enumeration and configuration changes it has made.

Host A checks its device and switch databases to find all host locks it obtained within the system (system Host A and the board interconnect switch). It issues a maintenance write transaction to their host base device ID lock CSRs to release the locks.

Time T+4: As Host B spins in its delay loop, it attempts to acquire the lock from the host base device ID Lock CSR of the board interconnect switch. A maintenance write of the host device ID (0×01), the destination device ID (0×FF), and the hop count (0) is issued for the lock CSR. Because Host A released the lock, Host B is able to confirm it has acquired the lock when it reads the value of 0×01 from the lock CSR.

Host B continues neighbor enumeration. It issues a maintenance read cycle to the device identity CAR of the board interconnect switch and looks for a matching entry in the device database. Device configuration continues because no match is found (Host B has not enumerated the device). Host B reads the source operations and destination operations CARs for the device. It is determined that the device does not support read/write/atomic operations and does not require a device ID. Host B reads the processing element feature CAR for the device and determines that it is a switch element.

Because the device is a switch, Host B reads the switch port information CAR and records the device identity in the switch database. Next, Host B adds a set of entries to the switch's routing table. For each previously discovered device ID, an entry is created containing a target ID (0×FF), hop count (0), and the route port number (from the switch port information CAR). The switch database is updated with the same routing information. Host B reads the port error status CSR for switch port 0, verifying that it is possible for the port to have a neighbor PE. An entry is created in the switch's routing table containing target ID (0×FF), hop count (0), and the route port number (0). Host B detects that it is attached to port 0. Because Host B has already been enumerated, neighbor enumeration continues on the next port.

Time T+5: Host B reads the port error status CSR for switch port 1, verifying that it is possible for the port to have a neighbor PE. An entry is created in the switch's routing table containing target ID (0×FF), hop count (0), and the route port number (1).

Host B continues neighbor enumeration using a hop count of 1. It attempts to acquire the lock from the host base device ID lock CSR of the neighbor PE on port 1. A maintenance write of the host device ID (0×01), the destination device ID (0×FF), and the hop count (1) is issued for the lock CSR. Host B confirms it has acquired the lock when it reads the value of 0×01 from the lock CSR.

Host B issues a maintenance read cycle to the device identity CAR of the DSP Farm board as shown in Figure 10.1 and looks for a matching entry in the device database. Device configuration continues because no match is found (Host B has not enumerated the device). Host B reads the source operations and destination operations CARs for the device. It is determined that the device supports read/write/atomic operations. A maintenance write is used to update the base device ID CSR with the value of 0×00 (the first available device ID). Device ID is incremented and compared with the Host B device ID. Because they are equal, device ID is assigned the next available device ID.

Time T+6: The process described in the previous step (time T+5) is repeated on switch ports 2–4. Device IDs 0×02, 0×03, and 0×04 are assigned to the PEs on switch ports 2, 3 and 4, respectively.

Time T+7: Host A detects that its host base device lock CSR has been acquired by another host device, indicating it has been enumerated. Host A can initiate passive discovery to build a local system database.