Calling conventions describe how you transfer variables to and from functions. If you will be using only functions that you have built yourself, you do not have to care about calling conventions. But when you are using C functions from the C library, you need to know in which registers you have to put the values to be used by that function. Also, if you write assembly functions for building a library that will be used by other developers, you’d better follow some convention for which registers to use for which function arguments. Otherwise, you will have lots of conflicts with arguments.
You already noticed that with the function printf, we put an argument in rdi, another in rsi, and yet another argument in xmm0. We were using a calling convention.
To avoid conflicts and the resulting crashes, smart developers designed calling conventions, a standardized way to call functions. It is a nice idea, but as you may expect, not everybody agrees with everybody else, so there are several different calling conventions. Up until now in this book we have used the System V AMD64 ABI calling convention, which is the standard on Linux platforms. But there is also another calling convention worth knowing: the Microsoft x64 calling convention to be used in Windows programming.
These calling conventions allow you to use external functions built with assembly, as well as functions compiled from languages such as C, without having access to the source code. Just put the correct arguments in the registers specified in the calling convention.
You can find out more about the System V AMD64 ABI calling convention at https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf . This Intel document has an overwhelming amount of detailed information about the System V application binary interface. In this chapter, we will show what you have to know to start calling functions in the standard way.
Function Arguments
Look back at the previous source files: for the circle calculations, we used xmm0 to transfer floating-point values from the main program to the circle function, and we used xmm0 to return the floating-point result of the function to the main program. For the rectangle calculation, we used rdi and rsi to transfer integer values to the function, and the integer result was returned in rax. This way of passing arguments and results is dictated by the calling convention.
The 1st argument goes into rdi.
The 2nd argument goes into rsi.
The 3rd argument goes into rdx.
The 4th argument goes into rcx.
The 5th argument goes into r8.
The 6th argument goes into r9.
The 10th argument is pushed first.
Then the 9th argument is pushed.
Then the 8th argument is pushed.
The 7th argument is pushed.
When you push the 10th argument, you decrease the stack pointer rsp by 8 bytes.
When you push the 9th argument, rsp decreases by 8 bytes.
When you push the 8th argument, rsp decreases by 8 bytes.
With the 7th argument, rsp decreases by 8 bytes.
Then the function is called; rip is pushed on the stack, and rsp decreases by 8 bytes.
Then rbp is pushed at the beginning of the function; as part of the prologue, rsp decreases by 8 bytes.
Then align the stack on a 16-byte boundary, so maybe another push is needed to decrease rsp by 8 bytes.
Thus, after we pushed the function’s arguments, at least two additional registers are pushed on the stack, i.e., 16 additional bytes. So, when you are in the function, to access the arguments, you have to skip the first 16 bytes on the stack, maybe more if you had to align the stack.
The 1st argument goes into xmm0.
The 2nd argument goes into xmm1.
The 3rd argument goes into xmm2.
The 4th argument goes into xmm3.
The 5th argument goes into xmm4.
The 6th argument goes into xmm5.
The 7th argument goes into xmm6.
The 8th argument goes into xmm7.
Additional arguments are passed via the stack; this is not accomplished with a push instruction as you might expect. We will show later how to do that, in the more advanced SIMD chapters.
A function returns a floating-point result in xmm0, and an integer number or address is returned in rax.
function5.asm
In this example, we pass all arguments in the correct order to printf. Note the reverse order of pushing the arguments.
This instruction leaves all the bytes in rsp intact, except the last one: the last four bits in rsp are changed to 0, thus decreasing the number in rsp and aligning rsp on a 16-byte boundary. If the stack had been aligned to start with, the and instruction would do nothing. Be careful, though. If you want to pop values from the stack after this and instruction, you have a problem: you have to find out if the and instruction changed rsp and eventually adjust rsp again to its value before the execution of the and instruction.
Output of function5
Stack Layout
function6.asm
Here, instead of printing with printf immediately after we provide all the arguments, as we did in the previous section, we call the function lfunc. This function takes all the arguments and builds a string in memory (flist); that string will be printed after returning to main.
We store these characters one by one in memory, starting at the address in rdi, which is the address of flist, with the instruction: mov [rdi], al. Using the byte keyword is not necessary, but it improves the readability of the code.
The other variables are each 8 bytes higher than the previous one. We used rbx as a temporary register for building the string in flist. Before using rbx, we saved the content of rbx to the stack. You never know if rbx is used in main for other purposes, so we preserve rbx and restore it before leaving the function.
Output of function6
Preserving Registers
It should be clear that you have to keep track of with happens with the registers during a function call. Some registers will be altered during the execution of a function, and some will be kept intact. You need to take precautions in order to avoid unexpected results caused by functions modifying registers you are using in the main (calling) program.
Calling Conventions
Register | Usage | Save |
|---|---|---|
rax | Return value | Caller |
rbx | Callee saved | Callee |
rcx | 4th argument | Caller |
rdx | 3rd argument | Caller |
rsi | 2nd argument | Caller |
rdi | 1st argument | Caller |
rbp | Callee saved | Callee |
rsp | Stack pointer | Callee |
r8 | 5th argument | Caller |
r9 | 6th argument | Caller |
r10 | Temporary | Caller |
r11 | Temporary | Caller |
r12 | Callee saved | Callee |
r13 | Callee saved | Callee |
r14 | Callee saved | Callee |
r15 | Callee saved | Callee |
xmm0 | First arg and return | Caller |
xmm1 | Second arg and return | Caller |
xmm2-7 | Arguments | Caller |
xmm8-15 | Temporary | Caller |
The function called is the callee. When a function uses a callee-saved register, the function needs to push that register on the stack before using it and pop it in the right order afterward. The caller expects that a callee-saved register should remain intact after the function call. The argument registers can be changed during execution of a function, so it is the responsibility of the caller to push/pop them if they have to be preserved. Similarly, the temporary registers can be changed in the function, so they need to be pushed/popped by the caller if needed. Needless to say, rax, the returning value, needs to be pushed/popped by the caller!
Problems can start popping up when you modify an existing function and start using a caller-saved register. If you do not add a push/pop of that register in the caller, you will have unexpected results.
Registers that are callee saved are also called nonvolatile. Registers that the caller has to save are also called volatile.
The xmm registers can all be changed by a function; the caller will be responsible for preserving them if necessary.
Of course, if you are sure you are not going to use the changed registers, you can skip the saving of these registers. However, if you change the code in the future, you may get in trouble if you start using these registers without saving them. Believe it or not, after a couple of weeks or months, assembly code is difficult to read, even if you coded everything yourself.
One last note: syscall is also a function and will modify registers, so keep an eye on what a syscall is doing.
Summary
Calling conventions
Stack alignment
Callee/caller-saved registers