6.1. SYSTEM TOPOLOGY
By virtue of the use of device IDs as the system level addressing scheme, RapidIO can support virtually any system topology. The device IDs do not convey any intrinsic information about where they are located. It is the responsibility of the interconnection fabric to discover where the devices are located and to forward packets to them based on the target device ID only.
Similar to a switched Ethernet network, which makes no assumptions about the a priori location of any given Ethernet device, a RapidIO network learns, during the system discovery phase of system bringup where devices are located in the system. Switches are programmed to understand the direction, although not the exact location of all devices in a system. When device locations change, as might occur during a hot swap or failover situation, only the switches need to be reconfigured to understand the new system topology.
In RapidIO, devices communicate with other devices by sending packets that contain source and destination device IDs. These device IDs, 8- or 16-bit fields in the packet header, are used as keys by the switch fabric devices to forward the packet towards its final destination. In RapidIO, switch devices will typically use lookup tables to hold the associations between device IDs and proper output ports. RapidIO switches will be programmed to understand which of multiple output ports should be used to forward a packet with any given destination device ID. Owing to the relatively small size of the device ID field and the common use of hashing keys to reduce table size, this lookup can be done quickly and need not add to the latency of a RapidIO packet traversing a switch device.
Also, because the source device ID is also included in the packet header, switches and end points are always aware of not only where a packet is going to, but also where it came from. The source device ID can be used by the system to send back a response packet indicating proper operation completion or perhaps an error condition. System level addressing in RapidIO is very similar to, although much simpler, than that of switched Ethernet. Ethernet relies on extended protocol stacks such as TCP/IP to create reliable communications channels between devices. RapidIO achieves these reliable channels solely through hardware constructs, which are more efficient for moving data around within a limited trusted system. RapidIO is not a replacement for Ethernet in a LAN environment. It is far too insecure for such an application. In like manner, Ethernet is not a replacement for RapidIO as an in-system communications technology. At higher speeds it requires far too much software overhead and introduces significant latency into the communications path between adjacent processing devices.
This switch-based approach to packet forwarding is much simpler to manage than alternative path-based routing schemes. In path-based schemes each end point must keep track of the proper path to every other end point in a system. This will be accomplished through the use of a table in each end point, which maps device IDs to ports for every other device in the system. If the system topology changes due to an event such as a path failover, every end point must be made aware of the failover and must have its path mapping tables updated appropriately. All of these tables, of which there might be several thousand, must be properly updated before the system can resume normal operation. A master processor will generally manage the updating of these path tables. The master processor must then keep a master copy of all the tables in all devices, and must be responsible for the topology changeover of the system both before and after the failover event; perhaps also interacting, post-event with all of the devices in the system to test that all of the connectivity paths are properly configured.