Addition and Subtraction

The first file in Listing 13-1, Ch13_01.h, contains the function declarations for this example. Note that functions AddI16_Aavx() and SubI16_Aavx() both require pointer arguments of type XmmVal . This is the same C++ SIMD data structure that was introduced in Chapter 2. The file Ch13_01.cpp contains code that performs test case initialization and streams results to std::cout.

The first function in file Ch13_01_fasm.asm, AddI16_Aavx(), illustrates packed integer addition using 16-bit wide elements. Function AddI16_Aavx() begins with a vmovdqa xmm0,xmmword ptr [r8] that loads argument value a into register XMM0. The text xmmword ptr is an assembler operator that conveys the size (128 bits) of the source operand pointed to by R8. The next instruction, vmovdqa xmm1,xmmword ptr [r9], loads argument value b into register XMM1. The ensuing instruction pair, vpaddw xmm2,xmm0,xmm1 (Add Packed Integers) and vpaddsw xmm3,xmm0,xmm1 (Add Packed Integers with Signed Saturation), performs packed integer addition of word elements using wraparound and saturated arithmetic, respectively. The final two AVX instructions of AddI16_Aavx(), vmovdqa xmmword ptr [rcx],xmm2 and vmovdqa xmmword ptr [rdx],xmm3, save the calculated results to the XmmVal buffers pointed to by c1 and c2.

Recall that source code example Ch02_01 included a C++ function named AddI16_Iavx(). This function employed _mm_load_si128() and _mm_store_si128() to perform SIMD load and store operations. In the current example, the assembly language function AddI16_Aavx() (which performs the same operations as AddI16_Iavx()) uses the vmovdqa instruction to perform SIMD loads and stores. Function AddI16_Iavx() also used _mm_add_epi16() and _mm_adds_epi16() to carry out packed integer addition using 16-bit wide integer elements. These C++ SIMD intrinsic functions are the counterparts of the AVX instructions vpaddw and vpaddsw . Most of the C++ SIMD intrinsic functions that you learned about in the first half of this book are essentially wrapper functions for x86-AVX instructions.

Multiplication

The file Ch13_02_fasm.asm contains three functions that perform packed integer multiplication. The first function, MulI16_Aavx(), begins its execution with two vmovdqa instructions that load argument values a and b into registers XMM0 and XMM1, respectively. This is followed by a vpmullw xmm2,xmm0,xmm1 (Multiply Packed Signed Integers and Store Low Result) instruction that performs packed signed integer multiplication using the 16-bit wide elements of XMM0 and XMM1. The vpmullw instruction saves the low-order 16 bits of each 32-bit product in register XMM2. The vpmulhw xmm3,xmm0,xmm1 (Multiply Packed Signed Integers and Store High Result) that follows calculates and saves the high-order 16 bits of each 32-bit product in register XMM3. The ensuing instruction pair, vpunpcklwd xmm4,xmm2,xmm3 (Unpack Low Data) and vpunpckhwd xmm5,xmm2,xmm3 (Unpack High Data), interleaves the low- and high-order word elements of their respective source operands to form the final doubleword products as shown in Figure 13-1. The last two AVX instructions of MulI16_Aavx(), vmovdqa xmmword ptr [rcx],xmm4 and vmovdqa xmmword ptr [rcx+16],xmm5, save the calculated products to c[0] and c[1]. Note that the second vmovdqa instruction uses a displacement value of 16 since each XmmVal structure instance in array c is 16 bytes wide.

The next function in Ch13_02_fasm.asm, MulI32a_Aavx(), performs packed signed integer multiplication using 32-bit wide elements. Note that this function only saves the low-order 32 bits of each 64-bit product. The final function in Listing 13-2, MulI32b_Aavx(), performs packed signed integer multiplication using 32-bit wide elements and saves complete 64-bit products. Function MulI32b_Aavx() begins its execution with two vmovdqa instructions that load argument values a and b into registers XMM0 and XMM1. The next instruction, vpmuldq xmm2,xmm0,xmm1 (Multiple Packed Doubleword Integers), performs packed 32-bit signed integer multiplication using the even-numbered elements of XMM0 and XMM1 and saves the resultant 64-bit products in register XMM2. The ensuing vpsrldq xmm3,xmm0,4 (Shift Double Quadword Right) and vpsrldq xmm4,xmm1,4 instructions right shift registers XMM0 (a) and XMM1 (b) by 4 bytes. This facilitates the use of the next instruction, vpmuldq xmm5,xmm3,xmm4, which calculates products using the odd-numbered elements of a and b as illustrated in Figure 13-2.

Bitwise Logical Operations

Arithmetic and Logical Shifts

Function SllU16_Aavx() begins its execution with a vmovdqa xmm0,xmmword ptr [rdx] instruction that loads argument value a into register XMM0. The next instruction, vmovd xmm1,r8d (Move Doubleword), copies the doubleword value in register R8D (argument value count) to XMM1[31:0]. Execution of this instruction also zeros bits YMM1[255:32]; bits ZMM1[511:256] are likewise zeroed if the processor supports AVX-512. The ensuing vpsllw xmm2,xmm0,xmm1 instruction left shifts each word element in XMM0 using the shift count in XMM1[31:0].

Pixel Minimum and Maximum

Near the top of Listing 13-5, function CalcMinMaxU8_Aavx() employs two test instructions to confirm that argument value n is not equal to zero and an integral multiple of 16. The third test instruction verifies that pixel buffer x is aligned on a 16-byte boundary. Following argument validation, CalcMinMaxU8_Aavx() uses a vpcmpeqb xmm4,xmm4,xmm4 (Compare Packed Data for Equal) instruction to load 0xFF into each byte element of register XMM4. More specifically, vpcmpeqb performs byte element compares using its two source operands and sets the corresponding byte element in the destination operand to 0xFF if source operand elements are equal. Function CalcMinMaxU8_Aavx() uses vpcmpeqb xmm4,xmm4,xmm4 to set each byte element of XMM4 to 0xFF since this is faster than using vmovdqa instruction to load a 128-bit constant of all ones from memory. The ensuing vpxor xmm5,xmm5,xmm5 instruction sets each byte element in register XMM5 to 0x00.

The next instruction, mov rax,-NSE, initializes loop index variable i. Register RAX is loaded with -NSE since each iteration of Loop1 begins with an add RAX,NSE instruction that calculates i += NSE. This is followed by the instruction pair cmp rax,r9 and jae @F, which terminates Loop1 when i >= n is true. Note that the order of instructions used to initialize and update i in Loop1 precludes a loop-carried dependency condition from occurring. A loop-carried dependency condition arises when calculations in a for-loop are dependent on values computed during a prior iteration. Having a loop-carried dependency in a for-loop sometimes results in slower performance. A for-loop sans any loop-carried dependencies provides better opportunities for the processor to perform calculations of successive iterations simultaneously.

During execution of Loop1, function CalcMinMaxU8_Aavx() maintains packed minimums and maximums in registers XMM4 and XMM5, respectively. The first AVX instruction of Loop1, vmovdqa xmm0,xmmword ptr [r8+rax], loads a block of 16 pixels (x[i:i+15]) into register XMM0. This is followed by a vpminub xmm4,xmm4,xmm0 (Minimum of Packed Unsigned Integers) instruction that updates the packed minimum pixel values in XMM4. The ensuing vpmaxub xmm5,xmm5,xmm0 (Maximum of Packed Unsigned Integers) instruction updates the packed maximum values in XMM5.

Following execution of Loop1, CalcMinMaxU8_Aavx() reduces the 16 minimum values in XMM4 to a single scalar value. The code block that performs this operation uses a series of vpsrldq and vpminub instructions as shown in Figure 13-3.

As a reminder, it is important to keep in mind that the benchmark timing measurements reported in this and subsequent chapters are intended to provide some helpful insights regarding potential performance gains of an x86-AVX assembly language coded function compared to one coded using standard C++ statements. It is also important to reiterate that this book is an introductory primer about x86 SIMD programming and not benchmarking. Many of the x86 SIMD calculating functions, both C++ and assembly language, are coded to hasten learning and yield significant but not necessarily optimal performance. Chapter 2 contains additional information about the benchmark timing measurements published in this book.

Pixel Mean Intensity

Near the top of file Ch13_06_fasm.asm is the statement extern g_NumElementsMax:qword, which declares g_NumElementsMax as an external quadword variable (the definition of g_NumElementsMax is located in the file Ch13_06_Misc.cpp). The first code block of CalcMeanU8_Aavx() uses a test r9,r9 and jz BadArg to ensure that argument value n is not equal to zero. The next instruction pair, cmp r9,[g_NumElementsMax] and ja BadArg, bypasses the calculating code if n > g_NumElementsMax is true. This is followed by the instruction pair test r9,3fh and jnz BadArg, which confirms that n is an even multiple of 64 (in later examples, you will learn how to process residual pixels). The final check of the first code block, test r8,0fh and jnz BadArg, confirms that pixel buffer x is aligned on a 16-byte boundary.

Following argument validation, CalcMeanU8_Aavx() sums the elements of pixel buffer x using SIMD arithmetic. The technique used in for-loop Loop1 is identical to the one used in source code example Ch02_07 and begins with a vpxor xmm3,xmm3,xmm3 instruction that initializes eight word sums to zero. The vmovdqa xmm0,xmmword ptr [r8+rax] instruction that follows loads pixel values x[i:i+15] into register XMM0. The ensuing instruction pair, vpunpcklbw xmm1,xmm0,xmm4 and vpunpckhbw xmm2,xmm0,xmm4, size-promotes the pixel values to 16 bits. These values are then added to the intermediate word sums in register XMM3 using the instructions vpaddw xmm3,xmm3,xmm1 and vpaddw xmm3,xmm3,xmm2 as shown in Figure 13-4. The next three code blocks in Loop1 add pixel values x[i+16:i+31], x[i+32:i+47], and x[i+48:i+63] to the intermediate sums in register XMM3. The final code block in Loop1 employs the instruction pair vpunpcklwd xmm0,xmm3,xmm4 and vpunpckhwd xmm1,xmm3,xmm4 to size-promote the packed word sums in XMM3 to doublewords. It then exercises two vpaddd instructions to add the current iteration sums to the intermediate packed doubleword sums maintained in register XMM5.