MP Systems Need a Better Interrupt Distribution Mechanism
Introduction
As just described, the legacy interrupt delivery mechanism interrupts the processor by asserting the processor's INTR input signal. The processor recognizes the interrupt on the next instruction boundary and must then perform an Interrupt Acknowledge transaction on its FSB to obtain the interrupt vector from the interrupt controller. This method is inefficient in the following ways:
Refer to Figure 61-3 on page 1502. Using the INTR signal to deliver interrupts to the processors in a multiprocessor (MP) system is a poor approach. All of the interrupts would be delivered to the processor that is connected to the output of the master 8259A PIC and that processor would have the burden of servicing all hardware interrupts. In an MP system, any processor should be capable of executing any program and of servicing any interrupt initiated by any device adapter. This is referred to as symmetric multiprocessing. Delivering all of the interrupts to one processor would yield a system that supports asymmetric multiprocessing.
Figure 61-3. Legacy Interrupt Delivery Mechanism Is Not Good Enough
The legacy interrupt delivery mechanism does not deliver the interrupt vector to the processor. The processor is forced to perform a transaction on the FSB to obtain the vector. This consumes FSB bandwidth and also increases the latency (i.e., the delay) in servicing an interrupt.
The APIC Interrupt Distribution Mechanism
Introduction
Refer to Figure 61-4 on page 1505. Starting with the P54C version of the Pentium® processor, a Local APIC module is incorporated within all IA32 processors. In addition, the chipset incorporates an IO APIC module. When a device adapter generates an interrupt to the IO APIC, the IO APIC in turn transmits an interrupt message to all of the Local APICS within the processors. The interrupt message contains the identity of the target Local APIC as well as the interrupt vector. An interrupt message can target one specific processor, multiple processors, or the processor that is currently executing the lowest-priority program (from the perspective of the OS).
Figure 61-4. The APIC Bus
The method used to transfer the message from the IO APIC to the processors depends on the processor family (refer to the next two subsections).
Message Transfer Mechanism Prior to the Pentium® 4
In the Pentium® P54C and the P6 processor family, the IO APIC is connected to the Local APICs within the processors via a 3-wire APIC bus. The APIC bus consists of two data lines and an APIC clock line. During each clock cycle, two bits of a message can be transmitted. Since all of the interrupt messages are considerably longer than two bits, it takes multiple APIC clock cycles to transmit a message.
Message Transfer Mechanism Starting with the Pentium® 4
Starting with the Pentium® 4 processor, the APIC bus was eliminated. When the IO APIC within the chipset must send an interrupt message to the processors, it arbitrates for ownership of the FSB and performs a memory write transaction to write the interrupt message to the Local APICS within the processor. The Local APICs are memory-mapped IO devices and accessed via memory transactions. Likewise, when a message must be sent from the Local APIC of one processor to the Local APIC of another processor, it is sent by performing a memory write transaction on the FSB.
Inter-Processor Interrupt Messages
In addition to the IO APIC sending device adapter interrupt messages to the Local APICs within the processors, software executing on one processor can send an interrupt message to one or more of the other processors (or even to itself). It does so by writing a command into the Local APIC's ICR (Interrupt Command Register).
Local Interrupts
As well as receiving interrupt messages, a processor's Local APIC can receive interrupts from a number of sources that are local to that specific processor:
The processor's Local Interrupt pin 0 (LINT0). When the Local APIC is disabled, this processor input acts as the INTR input from the legacy 8259A interrupt controllers (see Figure 61-1 on page 1500). Alternatively, the LINT0 input can be used to generate an interrupt when an event local to the processor occurs. For example, a thermal sensor in the machine (outside of the processor) might monitor the thermal envelope immediately around the processor and generate an interrupt to the processor if the temperature in the processor's immediate vicinity rises too much.
The APIC can generate an interrupt if it encounters an error while performing its duties.
NMI, SMI and Init Messages
The chipset can generate an NMI, an SMI or an INIT to a processor by sending an NMI, SMI, or INIT message to a processor.
The Cluster and APIC ID
Refer to Figure 61-6 on page 1506. The processors that reside on a FSB are referred to as a processor cluster. An MP system may incorporate one or more processor clusters.
Figure 61-6. The Big Picture
When the IO APIC sends a message to one or more of the processors (specifically, to their Local APICs), the interrupt message contains the ID of the target Local APIC within the target cluster. Each processor is automatically assigned a cluster ID and an APIC ID on the trailing edge of reset (see “Assignment of IDs to the Processor” on page 860). If need be, the OS can change the automatically assigned APIC ID, but each of the Local APICs in the system must be assigned a unique ID.
Physical Destination Mode
When an interrupt message specifies Physical Destination Mode, the message is received by all Local APICs but is only accepted by the one whose APIC ID matches the one specified in the message. Alternatively, the message can be broadcast to the Local APICs in all of the system processors.
Logical Destination Mode
When an interrupt message specifies Logical Destination Mode, the message is received by all Local APICs but is only accepted by the one or more Local APICs that belong to the logical group specified in the message.