System V ABI AMD64

This document describes concisely the subset of the amd64 ABI as it is implemented in QBE. The subset can handle correctly arbitrary standard C-like structs containing float and integer types. Structs that have unaligned members are also supported through opaque types, see the IL description document for more information about them.

ABI Subset Implemented

Data classes of interest as defined by the ABI:

• INTEGER
• SSE
• MEMORY

Classification

1. The size of each argument gets rounded up to eightbytes. (It keeps the stack always 8 bytes aligned.)
2. _Bool, char, short, int, long, long long and pointers are in the INTEGER class. In the context of QBE, it means that 'l' and 'w' are in the INTEGER class.
3. float and double are in the SSE class. In the context of QBE, it means that 's' and 'd' are in the SSE class.
4. If the size of an object is larger than two eightbytes or if contains unaligned fields, it has class MEMORY. In the context of QBE, those are big aggregate types and opaque types.
5. Otherwise, recursively classify fields and determine the class of the two eightbytes using the classes of their components. If any is INTEGER the result is INTEGER, otherwise the result is SSE.

Passing

• Classify arguments in order.
• INTEGER arguments use in order %rdi %rsi %rdx %rcx %r8 %r9.
• SSE arguments use in order %xmm0 - %xmm7.
• MEMORY gets passed on the stack. They are "pushed" in the right-to-left order, so from the callee's point of view, the left-most argument appears first on the stack.
• When we run out of registers for an aggregate, revert the assignment for the first eightbytes and pass it on the stack.
• When all registers are taken, write arguments on the stack from right to left.
• When calling a variadic function, %al stores the number of vector registers used to pass arguments (it must be an upper bound and does not have to be exact).
• Registers %rbx, %r12 - %r15 are callee-save.

Returning

• Classify the return type.
• Use %rax and %rdx in order for INTEGER return values.
• Use %xmm0 and %xmm1 in order for SSE return values.
• If the return value's class is MEMORY, the first argument of the function %rdi was a pointer to an area big enough to fit the return value. The function writes the return value there and returns the address (that was in %rdi) in %rax.

Alignment on the Stack

The ABI is unclear on the alignment requirement of the stack. What must be ensured is that, right before executing a 'call' instruction, the stack pointer %rsp is aligned on 16 bytes. On entry of the called function, the stack pointer is 8 modulo 16. Since most functions will have a prelude pushing %rbp, the frame pointer, upon entry of the body code of the function is also aligned on 16 bytes (== 0 mod 16).

Here is a diagram of the stack layout after a call from g() to f().

               |             |
               | g() locals  |
               +-------------+
          ^    |             | \
          |    | stack arg 2 |  '
          |    |xxxxxxxxxxxxx|  | f()'s MEMORY
 growing  |    +-------------+  |  arguments
addresses |    | stack arg 1 |  ,
          |    |xxxxxxxxxxxxx| /
          |    +-------------+ -> 0 mod 16
          |    |   ret addr  |
               +-------------+
               |  saved %rbp |
               +-------------+ -> f()'s %rbp
               | f() locals  |    0 mod 16
               |   ...       |
                               -> %rsp

Legend:

• xxxxx Optional padding.

Remarks

• A struct can be returned in registers in one of three ways. Either %rax, %rdx are used, or %xmm0, %xmm1, or finally %rax, %xmm0. The last case happens when a struct is returned with one half classified as INTEGER and the other as SSE. This is a consequence of the Returning section above.

• The size of the arguments area of the stack needs to be computed first, then arguments are packed starting from the bottom of the argument area, respecting alignment constraints. The ABI mentions "pushing" arguments in right-to-left order, but I think it's a mistaken view because of the alignment constraints.

  Example: If three 8 bytes MEMORY arguments are passed to the callee and the caller's stack pointer is 16 bytes algined, the layout will be like this.

             +-------------+
             |xxxxxxxxxxxxx|    padding
             | stack arg 3 |
             | stack arg 2 |
             | stack arg 1 |
             +-------------+ -> 0 mod 16
  

  The padding must not be at the end of the stack area. A "pushing" logic would put it at the end.