M68k Application Binary Interface (ABI)

Variants of ABIs

In this document, we’re covering two variants of M68k ABIs:

  1. Linux/GCC ABI: The default ABI used by GCC on Linux. It is backward-compatible with many legacy compilers’ ABIs, such as Sun PCC. Yet it lacks formal documentations. This is currently the only ABI implemented by M68k LLVM.
  2. SystemV ABI: The default ABI used by UNIX systems. There is a nice book documenting related details. Most of the contents in this page are also adapted from it.

Data Representations

Scalar Types

Here are the sizes and alignments of all scalar types.

Type C Type sizeof Alignment (bytes)
GCC ABI SysV ABI
Integral char / signed char / unsigned char 1 1 1
short / signed short / unsigned short 2 2 2
int / signed int / unsigned int 4 2 4
long / signed long / unsigned long 4 2 4
Pointer any-type * / any function pointer type 4 2 4
Floating Point float 4 2 4
double 8 2 8
long double 16 2 8

The biggest difference between GCC and SysV ABIs is, of course, the alignment. While SysV adopts natural alignment, GCC by default aligns larger integer types on 16-bit boundaries.

Note that by using the -malign-int flag, which is not enabled by default, GCC aligns int, long, float, double, and long double on 32-bit boundaries.

Another GCC-specific trick is the -mshort flag, which considers int to be 16 bits.

Aggregate Types

Here are the rules to determine the layout and alignment of an aggregate type (for example, struct or union):

  • The alignment of an aggregate type is the maximum alignment of its element types.
  • Each member will be placed at the lowest offset while respecting to its member type alignment. Internal paddings will be added if needed.
  • The size of an aggregate-type instance should be the multiple of its alignment. Tail paddings will be added if needed.

Stack Alignment

There is a difference between GCC and SysV ABI on stack alignment: By default, GCC aligns stack data on 16-bit boundaries, whereas SysV uses 32-bit alignment.

Calling Convention

In this section, we’re going to talk about the standard calling convention used by M68k. It is splitted into three sub-sections: Stack frame layout, passing function arguments, and handling return values.

Stack Frame

The diagram below shows a typical M68k stack frame: Usually, people use special instructions to setup a new stack frame. For instance, JSR pushes return address to the stack before jumping to the destination while RTS helps you to restore the return address upon returning from a function; LINK and UNLNK instructions can help you to save and restore the frame pointer.

In addition to special registers like %fp and %sp, other register usages also follow a certain convention.

Register Names Usage
%d0,%d1
%a0,%a1		 Scratch registers. Caller-save
%d2 ~ %d7
%a2 ~ %a5		 Local (variable) registers. Callee-save
%a6 (%fp) Frame pointer (if implemented)
%a7 (%sp) Stack pointer
%pc Program counter
%ccr Condition code register

For floating point unit, which is available after 68040 / 68881, here is a list of floating point register usages:

Register Names Usage
%fp0, %fp1 Scratch registers. Caller-save
%fp2 ~ %fp7 Local (variable) registers. Callee-save
%fpcr Floating point control register
%fpsr Floating point status register
%fpiar Floating point instruction address register

Function Arguments

As illustrated in the previous diagram, incoming function arguments are passed by stack. Since arguments are always aligned on 32-bit boundaries, they can be accessed at the following memory address (assuming frame pointer is implemented):

%fp + 8 + N * 4

where N is the index of the argument. For instance, the first argument is at %fp + 8 and the second is at %fp + 12.

When passing an aggregate-type object, regardless of their original type alignment, the object will be aligned on 32-bit boundaries.

Return Values

An integral return value is put in %d0, whereas a pointer return value is put in %a0.

When it comes to returning an aggregate-type object, the object should be stored in memory and its address will be put in:

  • SysV ABI: %a0
  • GCC ABI: %a1

Whether it’s caller or callee’s responsibility to allocate the space remains unspecified.