To see the information contained in the object file’s ELF header, enter the following command at the command line:

readelf -h HelloWorld.o

The -h flag tells readelf to display only the ELF header. Some of its important fields are listed here:

Magic:   7f 45 4c 46 02 02 01 00 00 00 00 00 00 00 00 00
Class:   ELF64
Data:    2's complement, big endian
...
Type:    REL (Relocatable File)
Machine: PowerPC64
...
Start of program headers:   0 (bytes into the file)
Start of section headers: 304 (bytes into the file)
...
Size of program headers:    0 (bytes)
Number of program headers:  0
Size of section headers:   64 (bytes)
Number of section headers: 14

The first 3 bytes of the Magic number identify the file as an ELF file (0x45, 0x4c, 0x46 form “ELF” in ASCII). The Class field states that the file was created for 64-bit systems. The ELF type, REL, stands for relocatable, which means this file contains data that can be linked into an executable. The Machine field specifies the target as PowerPC64, a category that includes the PPU.

Below the Machine field, the ELF header states the location and content of the program headers and section headers. In this example, there are no program headers and 14 section headers. The section headers can be found 304 bytes into the ELF file.

The ELF header states the size of the section headers and where they are located in memory. To see the section header data, enter the following command:

readelf -S HelloWorld.o

The output presents information about the ELF file’s sections in row-column format. This may look confusing because each section header occupies two rows. To clarify, Figure A.1 shows an example header for an Official Procedure Descriptor (.opd) section. This PowerPC-specific section stores the addresses of functions contained in the object file.

Figure A.1. Example section header in an ELF object file

The section header fields are as follows:

With this information, you can better understand Figure A.1. The .opd section takes up 0x18 bytes, is aligned on an 8-byte boundary, and its offset is 0xB0 bytes. This section is writable (W flag) and its data is incorporated into a program (PROGBITS). It occupies memory during the load process (A flag), but there’s no address yet for where it’s going to be placed. It contains no entries (it’s not a table) and doesn’t point to the symbol table.

Figure A.2 shows the memory structure of the HelloWorld.o object file, including the ELF header, the section headers, and the sections themselves.

Figure A.2. Structure of the HelloWorld.o object file

Sections listed with the A flag, called allocable sections, are loaded into memory when the executable starts. These include the machine code in the .text section and the data in such sections as .toc, .bss, .data, and .rodata. Sections without the A flag, called nonallocable sections, provide information to the linker and loader to create the executable. When the strip() function reduces the size of an executable, it removes the ELF file’s nonallocable sections.

Table A.1 lists many of the different types of sections found in ELF files and the information they provide. Some appear frequently in object files, whereas others are found only in executables.

The first of the listed sections, .text, is the most important. It contains the machine code that executes when the program runs. When the executable needs data, it accesses the .toc section. This contains global and local variables, and assembly instructions access them with offsets to the .toc location.

The section containing the symbol table, .symtab, lists all the symbols used within the object file, including function names and the name of the source file. To see the symbol table of HelloWorld.o, enter the following command:

readelf -s HelloWorld.o

This prints output similar to the following:

Symbol table '.symtab' contains 11 entries:

 Num:    Value          Size Type    Bind   Vis      Ndx Name
  0: 0000000000000000     0 NOTYPE   LOCAL  DEFAULT  UND
  1: 0000000000000000     0 FILE     LOCAL  DEFAULT  ABS HelloWorld.c
  2: 0000000000000000     0 SECTION  LOCAL  DEFAULT    1
  3: 0000000000000000     0 SECTION  LOCAL  DEFAULT    3
  4: 0000000000000000     0 SECTION  LOCAL  DEFAULT    4
  5: 0000000000000000     0 SECTION  LOCAL  DEFAULT    5
  6: 0000000000000000     0 SECTION  LOCAL  DEFAULT    7
  7: 0000000000000000     0 SECTION  LOCAL  DEFAULT    8
  8: 0000000000000000     0 SECTION  LOCAL  DEFAULT   10
  9: 0000000000000000    88 FUNC     GLOBAL DEFAULT    8 main
 10: 0000000000000000     0 NOTYPE   GLOBAL DEFAULT  UND puts

TheValue column identifies the symbols’ addresses. Because HelloWorld.o is a relocatable file, its symbols don’t have addresses yet and all the values equal zero. The Type field specifies whether a symbol refers to a function, file, data object, or special type. If a symbol’s type equals SECTION, the symbol represents one of the sections listed in the section header.

The Bind and Vis columns state the symbol’s visibility with respect to other files, executables, and libraries. A symbol’s Bind value is LOCAL, GLOBAL, or WEAK, depending on its usage and scope within the code. A WEAK symbol is assumed to be global, but is set to zero if its name conflicts with a GLOBAL symbol. If the Vis value equals DEFAULT, the symbol’s external visibility is determined by its Bind value.

If a symbol has type SECTION, the Ndx value identifies its index within the list of section headers. If a symbol doesn’t represent a section, its Ndx field can take one of three values:

The last symbol, puts, refers the printf function declared in stdio.h. This symbol is undefined (UNDEF) in HelloWorld.o, so the linker will look through its search path to find a matching symbol in an available library.

An object file lists its strings in a string table whose section name is .strtab. There is no readelf flag that displays the string table, but —hex-dump=N displays the content of Section N in hexadecimal and ASCII. On my system, the string table in HelloWorld.o is Section 13, so its content can be printed with the following command:

readelf —hex-dump=13 HelloWorld.o

The output is as follows:

Hex dump of section '.strtab':
0x0000 0048656c 6c6f576f 726c642e 63006d61 .HelloWorld.c.ma
0x0010 696e0070 75747300                   in.puts.

As shown, the string table contains the names of the functions (main, puts) and the name of the file, HelloWorld.c. When the linker needs to find these strings, it accesses the string table with the appropriate index.

The compiler and assembler don’t know how much memory is available on your system or how the linker will allocate it. Therefore, instead of assigning addresses to symbols, the machine code creates offsets from a hypothetical memory location.

Three of the sections in the object file (rela.toc, rela.text, rela.opd) contain information needed to assign effective addresses to three other sections (.toc, .text, .opd). To see the contents of these sections in the HelloWorld object file, enter the following command:

readelf -r HelloWorld.o

This displays the structure of all three of the relocation sections. The structure of the rela.txt section is shown below (abridged to fit on the page):

'.rela.text' at offset 0x5d0 contains 2 entries:

Offset   Info           Type         Sym. Value  Name+Addend
0022  00050000003f  R_PPC64_TOC16_DS 00000000000  .toc + 0
0024  000a0000000a  R_PPC64_REL24    00000000000  puts + 0

These table entries tell the linker that the .text section requires specific addresses for .toc (the program data) and puts (the output function). The Offset value shows where the specific address needs to be placed in the .text section. Info contains the index into the symbol table of the data that needs to be relocated. For example, puts is the tenth entry in the symbol table, and the first four digits in the corresponding Info field are 0x000a.

The addend in the last column specifies an added value that should be used when calculating an effective address for the symbol. All relocation sections that start with .rela have addends for their entries. Sections starting with .rel do not.

The INTERP segment in the executable names an interpreter to process the executable. This segment consists of the .interp section, which is Section 1 in HelloWorld. You can examine the section’s content with the following command:

readelf —hex-dump=1 HelloWorld

The printed result is /lib64/ld64.so.1. This shared-object file is GNU’s 64-bit dynamic linker, and integrates code from dynamic libraries into the process image at runtime.

To find the information it needs, the dynamic linker searches through the .dynamic section in the DYNAMIC segment of the executable. It also searches for the dynamic string table, stored in the .dynstr section. On my system, the ASCII content of this section is given as follows:

.libc.so.6.puts.__libc_start_main.GLIBC_2.3.

This tells the dynamic linker which function is needed (puts) and which function needs it (main). Using this and the other information in the .dynamic section, the linker can make sure Hello World! is displayed for all to see.

This discussion has briefly touched on dynamic libraries, but now it’s important to take a closer look. The next section shows how static and dynamic libraries are structured, and how to create them using GNU tools.

A static library (*.a) isn’t a new kind of ELF file; it’s just an archived collection of object files (*.o) whose structure was investigated earlier. To see this, change to the lib_static folder and run make. The two object files are created with the following commands:

ppu-gcc -c -o hello.o hello.c
ppu-gcc -c -o goodbye.o goodbye.c

Then the static library (libsimple.a) is built with the following command:

ppu-ar rcs libsimple.a hello.o goodbye.o

If you analyze libsimple.a with readelf, you’ll see that it doesn’t have an ELF structure of its own; readelf simply lists the contents of the two archived object files, hello.o and goodbye.o. Finally, the makefile creates an executable from libcall.c and libsimple.a with the following command:

ppu-gcc -o libcall libcall.c -L. -lsimple

After linking, the object code from the static library is completely incorporated into the executable. This self-containedness makes it simple to copy and distribute the application, but there’s no easy way to update the executable when libsimple.a is modified. For this reason, most *nix libraries are kept separate from the application and are integrated only during execution. These are shared libraries.

A shared library, also called a dynamic library or a shared object file, contains object code that is linked into the executable during runtime. This modularity keeps executables small and allows developers to update libraries without altering applications that require them. However, when an application depends on multiple shared libraries, version compatibility becomes a crucial issue.

Section A.2, “ELF Executable Files,” described how an executable ELF file specifies a dynamic linker and the libraries it needs. When the application executes, the linker searches through the list of directories in /etc/ld.so.cache. The cache entries are defined in /etc/ld.so.conf.

The makefile in the lib_shared project creates object files for a shared library with the following command:

ppu-gcc -c -fpic hello.c goodbye.c

The -fpic option tells the compiler to create position-independent object code. This is different from the relocatable code we looked at earlier. The addresses in relocatable code are resolved after the linking process while addresses in position-independent code are resolved during program execution.

The makefile creates libsimple.so from the position-independent object files with the following command:

ppu-gcc -shared -Wl,-soname,libsimple.so.1 
   -o libsimple.so.1.0.1 hello.o goodbye.o

This is markedly different from any other usage of ppu-gcc in this book. The compiler/linker options are given by the following:

To create a link between libsimple.so.1.0.1 and a name the linker will be able to recognize (*.so), the makefile executes two link commands:

ln -sf libsimple.so.1.0.1 libsimple.so.1
ln -sf libsimple.so.1 libsimple.so

Now libsimple.so is available for dynamic linking. You can view the ELF structure of this library with the following command:

readelf -e libsimple.so

You can see that the library has its own file type (DYN). Like an executable, it consists of segments that are loaded into memory at runtime. But it also has sections that executables don’t have, such as the Symbol Table (.symtab) and String Table (.strtab) sections. These sections contain references to the hello and goodbye functions provided by the library.

The last command of the makefile creates an executable that dynamically loads libsimple.so:

ppu-gcc -o libcall libcall.c -L. -R. -lsimple

The -R option identifies the executable’s rpath. This location is coded into the executable so that the dynamic linker knows where to find its library dependencies.

If libsimple.a and libsimple.so are placed in the same directory, the shared library will take precedence in the build. But you can ensure that the static library is linked by adding the -static option to the build command.

PPU-ELF files and SPU-ELF files have similar structures, but files of the latter type can contain an additional section called .toe. This stands for Table of EARs (Effective Address References). The purpose of this section is to store symbols mapped to effective addresses. By initializing the data in .toe, the PPU can pass address information to SPU applications.

At the time of this writing, developers must create .toe sections in assembly. Listing A.5 shows an example of how this is done.

.section .toe, "a", @nobits
.align 4

.global _EAR_prime
_EAR_prime:
.octa 0x0

This code creates an allocable section, .toe, containing a global symbol called _EAR_prime. The _EAR_ prefix is required to prevent mangling. This symbol refers to a value of 0x0 in toe.s, but the PPU code assigns the _EAR_prime symbol to a memory address. The SPU code reads this memory address as an unsigned long long called _EAR_prime.

The toe.s source file can be found in the AppendixA/prime project directory. The makefile in this directory incorporates the code into the SPU executable spu_prime with the following command:

spu-gcc -o spu_prime spu_prime.c toe.s

When this command is executed, spu_prime can access _EAR_prime as if it was a regular external symbol. This is shown in Listing A.6. In this application, the SPU retrieves an array of ints from the _EAR_prime address and displays both the address and the elements in the array.

#include <stdio.h>
#include <spu_mfcio.h>

#define CESOF_TAG 27

/* The .toe symbol defined in the assembly code */
extern unsigned long long _EAR_prime;

/* The array in the SPU's Local Store */
int local_prime[16] __attribute((aligned(128)));

int main(long long spuid, char** argp, char** envp) {

   /* Display the value of _EAR_prime */
   printf("SPU Array Location: %#llx
", _EAR_prime);

   /* Transfer the array at _EAR_prime to memory */
   mfc_get(local_prime, _EAR_prime, sizeof(local_prime),
      CESOF_TAG, 0, 0);
   mfc_write_tag_mask(1<<CESOF_TAG);
   mfc_read_tag_status_all();

   /* Display the array's contents */
   int i;
   for (i=0; i<16; i++)
      printf("%d ",local_prime[i]);
   printf("
");

   return 0;
}

To examine the sections and segments in the spu_prime executable, enter the following command:

spu-readelf -e spu_prime

The ELF header identifies this file as an SPU executable. The section header table contains an entry for the .toe section. This section doesn’t hold any meaningful data yet, but the PPU code will use this section to transfer a memory address to the SPU.

To be accessed by the PPU, SPU executable code must be embedded into a PPU object file. The resulting object file is called a CESOF (CBE Embedded SPE Object Format) file. The ppu-embedspu command performs the embedding. The following makefile command converts the spu_prime executable into a CESOF called spu_prime-embed.o:

ppu-embedspu -m64 spu_prime_handle spu_prime spu_prime-embed.o

The ppu-embedspu arguments in this command are as follows:

The spu_prime-embed.o object file is a CESOF file. If you examine its structure you’ll see that it’s a relocatable file targeting the PPU. You’ll also see a new section, .rodata.elf, that takes up a great deal of the ELF file. As shown in Figure A.5, this section contains the allocable sections of the SPU executable, including .text, .data, and .toe.

Figure A.5. CESOF sections

The format of the CESOF is similar to that of regular PPU relocatable object files but there are a few important differences. The .data section contains a program handle, spu_prime_handle, created by ppu-embedspu. This structure holds three pieces of information: a pointer to the SPU executable, a pointer to the TOE information, and the structure’s size.

The SPU’s .toe section is mirrored in the CESOF’s .data.spetoe section. This is where the PPU code will access the TOE data when the executable runs. Like any relocatable object file, the CESOF can be converted into a static or shared library using the methods described in Section A.3, “ELF Libraries.”

The PPU code in Listing A.7 accesses the spu_prime_handle created earlier and loads it into a context representing the SPU. Then it runs and deallocates the context.

#include <stdio.h>
#include <stdlib.h>
#include <libspe2.h>

/* Initializes the memory location contained in .toe */

int prime[16] __attribute((aligned(128)))
   = {2, 3, 5, 7, 11, 13, 17, 19,
      23, 29, 31, 37, 41, 43, 47, 53};

/* Accesses the handle in the CESOF file */
extern spe_program_handle_t spu_prime_handle;

int main(int argc, char **argv) {
   int ret;
   spe_context_ptr_t ctx;
   unsigned int entry_point;
   spe_stop_info_t stop_info;

   /* Display the EA of the array */
   printf("PPU array location: %#llx
",
      (unsigned long long)prime);

   /* Create the SPE Context */
   ctx = spe_context_create(0, NULL);
   if (!ctx) {
      perror("spe_context_create");
      exit(1);
   }

   /* Load the program into the context */
   ret = spe_program_load(ctx, &spu_prime_handle);
   if (ret) {
      perror("spe_program_load");
      exit(1);
   }

   /* Run the program */
   entry_point = SPE_DEFAULT_ENTRY;
   ret = spe_context_run(ctx, &entry_point, 0,
      NULL, NULL, &stop_info);
   if (ret < 0) {
      perror("spe_context_run");
      exit(1);
   }

   /* Deallocate the context */
   ret = spe_context_destroy(ctx);
   if (ret) {
      perror("spe_context_destroy");
      exit(1);
   }

   return 0;
}

The makefile in the prime project compiles the full application with the following command:

ppu-gcc -o ppu_prime ppu_prime.c spu_prime-embed.o -lspe2

The result of the execution should look something like this:

PPU array location: 10019480
SPU Array Location: 10019480
2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53

As shown, the .toe section in the CESOF enables the SPU to access its _EAR_prime array at the same address as the PPU’s prime array.