SQ Drain (SQD) State
Under Software Control, SQD Is Entered From...
...the RTS state. Software transitions a QP from the SQD state to the RTS state using the Modify QP verb call.
Used When Altering Operational QP's Characteristics
Software may need to alter the operational characteristics of a QP that is already fully operational (i.e., it is in the RTS state). It would be dangerous to do so while the QP still has one or more messages in flight to the remote QP. Commanding an operational QP to enter the SQD state effectively causes the QP:
- To complete the transmission of all in-progress messages,
- wait until all expected responses have been received,
- retire all in-progress WQEs,
- create their respective CQEs,
- and then pause.
No new message transmissions are initiated. Software may then safely use the Modify QP verb call to alter the QP's operational characteristics and then return the QP to the fully operational state. At this point, the QP's SQ Logic effectively unpauses and resumes message processing with the next WQE on the SQ.
SQD State Operational Characteristics
When a QP enters the SQD state, it has the following operational characteristics:
New WRs can be posted to both queues while the QP remains in the SQD state.
When the QP enters the SQD state, the QP:
- Must not start processing any WRs on its SQ for which processing (i.e., message transmission) hadn't already been initiated.
- Must complete any previously initiated message transmissions.
- Must process any incoming Acks associated with in-progress message transmissions (RC and RD).
An incoming message (a Send or an RDMA Write With Immediate) targeting the QP is processed normally by the next RQ WQE. If RC or RD, this includes generating an Ack (or a Nak) for the request packet received.
When all expected Acks have been received and processing of all in-progress WRs (i.e., message transfers) is done, and if Event Notification was requested, an Affiliated Asynchronous Event is generated. Software can use the Affiliated Asynchronous Event to determine when a state transition (back to RTS from SQD) is possible. See “Event Notification Signals SQ Quiescence” on page 251.
The QP's characteristics may be modified during the transition from SQD back to RTS (via the Modify QP verb), but the software entities associated with both the local and remote QPs must have received the Affiliated Asynchronous Event in order to safely change the QP's characteristics.
If the HCA supports changing the port that is associated with the QP when transitioning from SQD to RTS, it associates the new port with the QP before making the transition back to the RTS state. Specification of a new port is an optional input parameter for the Modify QP verb when switching from the SQD state to the RTS state.
Software can cause the QP to exit SQD by executing the Destroy QP verb or the Modify QP verb (to change back to the RTS state, or to go to the Reset or Error state).
While a RD QP is in the SQD state, WRs may be posted to its SQ. If the WR specifies an EEC that is currently in the SQD state, then the WR isn't processed but remains posted.
Event Notification Signals SQ Quiescence
Earlier in time, software would have executed the Set Asynchronous Event Handler verb call to tell the Verb Layer the start address of a handler to be called whenever an Affiliated Asynchronous Event occurs.
When software later uses the Modify QP verb to transition a QP to the SQD state, it may optionally specify that an Affiliated Asynchronous Event be generated once the QP's SQ has finished all in-progress message transmissions.
Upon completion of all in-progress message transmissions, the HCA would then inform the Verb Layer (typically via an interrupt) that the Asynchronous Event Handler is to be called and the reason why. The handler is called, determines that the QP's SQ is now quiescent, and informs the software entity that is waiting for this notification before changing the QP's operational characteristics. Software can then use a single Modify QP verb call to alter the QP's operational characteristics and change it back to the RTS state.
Input Parameters on SQD-to-RTS Transition
Table 12-6 on this page lists the input parameters that may be supplied when using the Modify QP verb to transition the QP from SQD back to RTS. This would cause the QP to assume the altered operational characteristic(s) and then switch back to the RTS state with them in effect.
| Input Parameter(s) | Applicable QP Types | Required? | Description |
|---|---|---|---|
| HCA Handle | All | Yes | Returned by earlier Open HCA verb call. |
| QP handle | All | Yes | Returned by earlier Create QP verb call. |
| Next State | All | Yes | In this case, the RTS state. |
| Remote Node Address Vector | RC and UC | No | See the description of this parameter in Table 12-3 on page 241. |
| Alternate path address information | RC and UC | No | See the description of this parameter in Table 12-3 on page 241. |
| Path migration state | No | For a complete description, refer to “Automatic Path Migration” on page 575. | |
| Depth of remote QP's special queue for RDMA Read/Atomic operations | RC and RD | No | Depth of remote QP's special queue for posting of inbound RDMA Read/Atomic operations. |
| Queue depth for inbound RDMA Read/Atomic operations | RC and RD | No | Depth of the special queue in the RQ Logic into which inbound RDMA Read and atomic operation requests are posted. |
| Q_Key | RD and UD | No | See the description of the Q_Key parameter in Table 12-2 on page 238. |
| P_Key index | RC, UC, UD | No | See the description of this parameter in Table 12-2 on page 238. |
| Local Ack Timeout | RC | No | Time QP's SQ Logic waits for Ack before it retries transmission of corresponding request packet. |
| Retry Count | RC | No | Number of times QP's SQ Logic retries transmission of request packet due either to an Ack timeout, or to the receipt of a Nak from the remote QP's RQ indicating it detected a missing packet (i.e., a request packet was received with a PSN > the ePSN). |
| RNR Retry Count | RC | No | Supplied by remote CA's CM when two QP's first set up. Defines how many times this QP's SQ Logic should retry transmission of request packet that receives RNR Nak from remote QP's RQ Logic. Remote QP's RQ Logic responds to request with RNR Nak if temporarily unable to handle request. Classic example: Request packet received and no WQEs posted to receiver's RQ to handle inbound request. |
| New SQ and/or RQ size | All | No | If HCA supports resizing of the SQ and RQ:
|
| Minimum RNR NAK Timer Field | RC and RD | No | See the description of this parameter in Table 12-3 on page 241. |
| HCA port number | RC | No | Physical HCA port the QP will use to send and receive messages. This can only be altered if the HCA supports this capability. |