Brief Layer Descriptions

Upper Layer Protocols (ULP—i.e., software)

A software application or driver builds the message data and transfer request in system memory. It then posts a message send or receive request to a local QP's SQ or RQ. It subsequently handles notice of successful completion or any errors that may be reported back by the verbs or by the transport layer.

Verb Layer

A software application calls the verbs to perform operations such as:

• HCA management.

• QP management.

• EEC management.

• Memory Region and Window management.

• Post work requests to send and receive queues.

• Handle message transfer completions.

• Handle unexpected events.

Transport Layer

The Transport Layer is responsible for sending and receiving messages across the fabric between QPs. When necessary, it segments a message into multiple packets (when the message's total length is greater than the PMTU of the path between the source and destination ports). The QP on the receiving end reassembles the message data into the specified data buffer(s) in its memory.

Network Layer

The Network Layer in a router handles the routing of a packet between subnets (using the packet's GRH). A packet contains a GRH when the source and destination are in different subnets (or if it's a multicast UD packet). In a CA, the Network Layer performs the following functions:

• On packet transmission: When a request packet is received from the Transport Layer for transmission and the request packet's destination resides in a different subnet, the CA's Network Layer inserts the GRH in the packet. Otherwise, it just passes the packet to the source port's Link Layer for transmission.

• On packet receipt: When a request or a response packet is received from the Link Layer, the Network Layer silently discards (with the exception of incrementing any applicable performance counters) any packet with a GRH that does not meet either of the following conditions:

  - The GRH:IPVer field must contain 6 (indicating Internet Protocol Version 6).

  - The packet's GRH:DGID must be one of the GID addresses assigned to the port that received the packet.

  If both of these conditions are met, the Network Layer presents the packet to the Transport Layer for processing. If a packet does not contain a GRH, the Network Layer just passes the packet to the Transport Layer (i.e., to the destination QP or EEC).

Link Layer

The Link Layer is responsible for sending and receiving data across the fabric at the packet, rather than the message, level. It has the following responsibilities:

• Addressing:

  - On transmission, the Link Layer receives the packet from the CA's Network Layer and uses the Source Path Bits supplied by the Transport Layer to insert the correct port LID address into the packet's LRH:SLID field.

  - On receipt of a packet from the port's Physical Layer, the port's Link Layer performs an address decode of the packet's DLID field to determine if it is the target port. If it is, the packet is passed to the CA's Transport Layer for processing.

• Buffering. A port's Link Layer contains transmit and receive buffer pairs (referred to as Virtual Lane buffer pairs).

• Flow Control. The port's Link Layer handles the issuance of link-level Flow Control packets from its data VL receive buffers. Its data VL transmit buffers receive and process link-level Flow Control packets sent by the data VL receive buffers on the other end of the link (a detailed description of link-level Flow Control can be found in “Link-Level Flow Control” on page 637).

• Error detection.

• Packet switching. In a switch, each port's Link Layer performs a lookup in the switch's Forwarding Table attribute using the packet's DLID field to determine through which of its ports the packet must be forwarded.

Physical Layer

The physical layer is responsible for sending and receiving data across the fabric at the character- and bit-level (rather than the message- or packet-level). It defines how bits are placed on the wire to form symbols (i.e., characters) and defines the symbols used for framing (i.e., the start-of-packet and end-of-packet symbols), for data characters, and the fill characters used between packets (Idles). It defines the symbol encoding, the proper alignment of the packet framing symbols, disallows invalid and non-data symbols between the start- and end-packet delimiters, and defines disparity errors, as well as the methods used to sync up the transmitter and receiver on a link.