All the multiprocessor applications in this book have relied on libspe to coordinate PPU/SPU processing, and they all work the same way: The PPU assigns object code to SPUs using contexts, runs the contexts as threads, and waits for the threads to complete. Mailbox messaging transfers small data payloads and DMA transfers large blocks of data. All data transfer and memory buffering is performed at a low level using pointers to memory.

An ALF application accomplishes the same result, but the development process is simpler thanks to the ALF runtime. This processing environment handles many of the low-level details of SPU management. For example, the code still has to identify SPU program handles, but the ALF assigns the program handles to SPUs. The code still has to identify what data the SPUs should have, but the ALF runtime manages the communication mechanisms like mailboxes and DMA.

In libspe, the central data structure is the context. With ALF, there are three central structures: tasks, task instances, and work blocks:

An example will clarify how this works. Let’s say you want to perform eight matrix multiplications using four accelerators. The ALF task identifies the accelerator object code needed to multiply the matrices, the type of input data to be transferred to the accelerators, and the type of output data to be transferred back to host memory. A task instance is assigned to each of the accelerators, as shown in Figure C.1.

Figure C.1. ALF tasks and work blocks

Eight work blocks are created, and each is configured with the information it needs to perform a specific matrix multiplication. That is, the first work block receives the addresses of the first two input matrices and the address where the output should be stored. The computational task doesn’t change from work block to work block; the specific data needed for the computation does. Once defined, these eight work blocks are enqueued on the task for execution.

Figure C.1 shows the eight work blocks evenly distributed among the task instances, but this isn’t always the case. By default, the ALF runtime sends each work block to whichever accelerator is available to process it. This behavior can be changed when the task is defined.

Chapter 12, “SPU Communication, Part 1: Direct Memory Access (DMA),” explains how DMA and multibuffering can be combined to create applications that process and transfer data at the same time. It’s like an on-chip assembly line: While the input buffer is filled, the data in the computation buffer is processed, and data in the output buffer is delivered to host memory. Then the buffers switch roles.

This is fine in theory, but writing Cell code to make multiple DMA transfers into multiple buffers can be difficult. Not only do you have to know what each buffer is doing for each iteration, you also have to keep track of DMA tags, array indices, and scatter/gather operations. As the amount of data increases, it becomes easier to make mistakes and harder to debug them.

The ALF runtime manages data transfers and multibuffering by dividing up accelerator memory into six main buffers of configurable size. These are shown in Figure C.2.

Figure C.2. ALF accelerator memory buffers

These buffers are differentiated by the type of data they hold and when/where the data is transferred:

The stack buffer operates like a regular stack, and its maximum size must be configured in code. The two context buffers hold initialization data for the task and work blocks. The last three buffers contain preprocessed and postprocessed data. As a general rule of thumb, if you’d normally send data using mailboxes, send the data using an ALF context buffer. If you’d normally transfer data using DMA, use ALF’s input, output, or inout buffers.

The ALF library, libalf, provides three basic functions that initialize ALF’s operation and provide general information. alf_init creates the ALF environment, alf_query_system_info returns information about the environment, and alf_num_instances tells the environment how many accelerators will be used by the application. In many ALF applications, these are the first three functions to be invoked.

int alf_init(sys_config_info_CBEA_t* library_path, alf_handle_t* handle);

typedef struct {
   char* library_path;
} alf_sys_config_t_CBEA_t;

int alf_query_system_info(alf_handle_t alf_handle,
ALF_QUERY_SYS_INFO_T info_type,
ALF_ACCEL_TYPE_T accel_type, unsigned int* result);

#include <stdio.h>
#include "alf.h"

int main(int argc, char **argv) {

   /* Declare variables - info vars must be unsigned int */
   alf_handle_t alf_handle;
   unsigned int num_accel, host_mem, accel_align;

   /* Initialize the alf handle and access system info */
   alf_init(NULL, &alf_handle);
   alf_query_system_info(alf_handle, ALF_QUERY_NUM_ACCEL,
      ALF_ACCEL_TYPE_SPE, &num_accel);
   alf_query_system_info(alf_handle, ALF_QUERY_HOST_MEM_SIZE,
      ALF_ACCEL_TYPE_SPE, &host_mem);
   alf_query_system_info(alf_handle,
      ALF_QUERY_ACCEL_ADDR_ALIGN,
      ALF_ACCEL_TYPE_SPE, &accel_align);

   /* Display the results */
   printf("Number of accelerators: %d
", num_accel);
   printf("Host memory: %d
", host_mem);
   printf("Accelerator memory alignment: %d
", accel_align);
   return 0;
}

Number of accelerators: 6
Host memory: 217068
Accelerator memory alignment: 4

int alf_num_instances_set(alf_handle_t handle, unsigned int n);

alf_init(NULL, &alf_handle);

alf_query_system_info(alf_handle, ALF_QUERY_NUM_ACCEL,
   ALF_ACCEL_TYPE_SPE, &num_accel);

alf_num_instances_set(alf_handle, num_accel);

You don’t configure tasks directly in ALF. Instead, you create and configure a task descriptor, and use it to create a new task. The function that creates task descriptors is alf_task_desc_create. Its signature is given by the following:

int alf_task_desc_create(alf_handle_t alf_handle, ALF_ACCEL_TYPE_T accel_type, alf_task_desc_handle_t* descriptor_handle);

The second parameter is the type of accelerator (ALF_ACCEL_TYPE_SPE on the Cell), and the last parameter must be a pointer to an alf_task_desc_handle_t structure. This represents the task descriptor.

int alf_task_desc_set_int32(alf_task_desc_handle_t task_desc_handle,
ALF_TASK_DESC_FIELD_T field, unsigned int value);
int alf_task_desc_set_int64(alf_task_desc_handle_t task_desc_handle,
ALF_TASK_DESC_FIELD_T field, unsigned long long value);

char library_name[] = "lib_name.so";
char spu_image_name[] = "spu_alf_text";

alf_task_desc_set_int64(task_desc, ALF_TASK_DESC_ACCEL_IMAGE_REF_L,
   (unsigned long long)spu_image_name);

alf_task_desc_set_int64(task_desc, ALF_TASK_DESC_ACCEL_LIBRARY_REF_L,
   (unsigned long long)library_name);

int alf_task_desc_ctx_entry_add(alf_task_desc_handle_t task_desc_handle, ALF_DATA_TYPE_T data_type, unsigned int size);

alf_task_desc_ctx_entry_add(desc_handle, ALF_DATA_DOUBLE, 4);

alf_task_desc_ctx_entry_add(desc_handle, ALF_DATA_FLOAT, 3);
alf_task_desc_ctx_entry_add(desc_handle, ALF_DATA_ADDR64, 1);

After the task descriptor has been configured, you can use it to create a task. This is accomplished with a call to alf_task_create, which accepts six parameters:

The tsk_context parameter points to the task context structure whose composition was defined by alf_task_desc_ctx_entry_add. It must have the same size as the value of the ALF_TASK_DESC_TSK_CTX_SIZE field in the task descriptor.

The next two parameters, num_inst and tsk_attr, are more complicated. As mentioned earlier, you tell the ALF runtime the total number of task instances you want with alf_num_instances_set. If tsk_attr is set to ALF_TASK_ATTR_SCHED_FIXED and num_inst is set to a lower number, however, this lower number controls how many task instances are created.

By default, the ALF runtime assigns each work block to the first task instance that can process it. But if tsk_attr is set to ALF_TASK_ATTR_SCHED_FIXED | ALF_TASK_ATTR_WB_CYCLIC, the work blocks are distributed in a round-robin manner, regardless of how time intensive they are.

For example, if a task contains nine work blocks (0–8), num_instances equals 3, and tsk_attr is set to ALF_TASK_ATTR_SCHED_FIXED | ALF_TASK_ATTR_WB_CYCLIC, blocks {0, 3, 6} are assigned to task instance 0, blocks {1, 4, 7} are assigned to task instance 1, and blocks {2, 5, 8} are assigned to task instance 2. This is shown in Figure C.3.

Figure C.3. Cyclic work block distribution

The last two fields of alf_task_create are straightforward: wb_dist_size tells the runtime how many work blocks are bundled together with each distribution, and task_handle represents the task itself. You probably won’t access this task structure directly. Instead, you need to create and enqueue one or more work blocks.

An ALF task serves no purpose without work blocks. Each work block represents one invocation of the task on the accelerator, and each work block executes the accelerator code identified in the task descriptor. In addition, each work block receives the same task context data.

int alf_wb_create(alf_task_handle_t task_handle, ALF_WORK_BLOCK_TYPE_T block_type, unsigned int repeat_count, alf_wb_handle_t *wb_handle);

alf_wb_create(task_handle, ALF_WB_MULTI, 20, &wb_handle);

int alf_wb_parm_add(alf_wb_handle_t wb_handle, void *pdata, unsigned int data_num, ALF_DATA_TYPE_T data_type, unsigned int alignment)

Work Blocks and Data Transfer Lists

When the ALF runtime needs to transfer large amounts of data, it creates data transfer lists, or DTLs. A DTL is similar to the DMA list discussed in Chapter 12, but in addition to transferring data, a DTL also manages how the data is buffered on the accelerator. This buffer management, in my opinion, is the most important advantage of using ALF.

The process of generating DTLs is called data partitioning, and it can be performed by the host application or the accelerator application. To tell the ALF runtime which partitioning method to use, you need to set the ALF_TASK_DESC_PARTITION_ON_ACCEL field in the task descriptor. If this is set to 1, the runtime calls data partitioning routines on the accelerator. If it’s set to 0, the host must invoke its own DTL functions.

Each DTL provides two pieces of information:

1. The data structure in host memory that needs to be transferred

2. How the data should be buffered in accelerator memory

The ALF library provides three host functions that create DTLs. Table C.4 lists each and its purpose.

Table C.4. ALF Functions for Creating DTLs on the Host

Function Name	Purpose
alf_wb_dtl_begin	Specify data buffering scheme
alf_wb_dtl_entry_add	Identify data to be transferred
alf_wb_dtl_end	End the data transfer list

The first function, alf_wb_dtl_begin, tells the runtime how and where the data should be stored in accelerator memory. Its signature is given by the following:

int alf_wb_dtl_begin(alf_wb_handle_t wb_handle, ALF_BUF_TYPE_T buffer_type, unsigned int buffer_offset);

The second parameter identifies how the data should be buffered in accelerator memory. DTL data can be transferred into an input or inout buffer, or transferred out of an inout or output buffer. Table C.5 lists the possible values for the second parameter.

Table C.5. Buffer Storage for Work Block Data (ALF_BUF_TYPE_T)

Buffer Type	Purpose
ALF_BUF_IN	Store data in dedicated input buffer
ALF_BUF_OUT	Store data in dedicated output buffer
ALF_BUF_OVL_IN	Store data in input section of overlap buffer
ALF_BUF_OVL_OUT	Store data in output section of overlap buffer
ALF_BUF_OVL_INOUT	Store data in inout section of overlap buffer

Which buffer type to use depends on what you intend to do with the data. For example, if the first work block of a sequence takes data into an accelerator but doesn’t transfer it out, ALF_BUF_IN is appropriate. If the last work block of a sequence delivers output from an accelerator but doesn’t bring any in, ALF_BUF_OUT is the type to use. If the work block transfers data into and out of an accelerator, ALF_BUF_OVL_INOUT should be used.

Note

Make sure to manage the buffer sizes so that they don’t take up all the accelerator memory. Buffer sizes are configured in the task descriptor using the

ALF_TASK_DESC_WB_IN_BUF_SIZE, ALF_TASK_DESC_WB_OUT_BUF_SIZE, and ALF_TASK_DESC_WB_INOUT_BUF_SIZE fields.

ALF’s method of multibuffering depends on which storage scheme you choose. If you store data in an overlap buffer and your combined buffer size (input + output + inout) is less than 120KB, the ALF runtime uses two sets of buffers to perform input-computation-output tasks. The first set of buffers starts first: It acquires data from host memory and begins computation. The second set acquires data while the first is computing, and the two sets of buffers alternate between computation and data transfer. Figure C.4 shows how this double-buffering mechanism works.

Figure C.4. Overlap buffer operation

In addition to specifying the buffer type, you need to tell the ALF runtime where data should be placed inside the buffer. The last parameter of alf_wb_dtl_begin specifies how many bytes into the buffer the data should be placed. For example, to position matrix_b after matrix_a in the input section of an overlap buffer, you could use the following:

int alf_wb_dtl_begin(wb_handle, ALF_BUF_OVL_IN, sizeof(matrix_a));

The second function in the table, alf_wb_dtl_entry_add, adds entries to the DTL. Each entry usually corresponds to a single data structure. DTL entries are like DMA list elements, but simpler. The function’s signature is given by the following:

int alf_wb_dtl_entry_add (alf_wb_handle_t wb_handle, void* pdata, unsigned int data_num, ALF_DATA_TYPE_T data_type);

This is similar to the alf_wb_parm_add function described previously. pdata points to the data to be transferred, data_type identifies the datatype, and data_num tells how many of the datatype should be transferred.

The last function, alf_wb_dtl_end, completes the DTL creation process, and its only parameter is the work block handle. The following code shows how all three functions are used to transfer two large float arrays, vec_a and vec_b, into a dedicated input buffer:

alf_wb_dtl_begin(wb_handle, ALF_BUF_ IN, 0);
alf_wb_dtl_entry_add(wb_handle, vec_a, 1000, ALF_DATA_FLOAT);
alf_wb_dtl_entry_add(wb_handle, vec_b, 1000, ALF_DATA_FLOAT);
alf_wb_dtl_end(wb_handle);

Multiple DTLs can be added to a work block as needed.

int alf_wb_enqueue(alf_wb_handle_t wb_handle)

After the work blocks are enqueued, the task can be started. This requires two function calls: alf_task_finalize(task_handle) and alf_task_wait(task_handle, timeout). The first function prevents further work blocks from being enqueued, and the second starts the task and waits until it completes its processing.

The timeout parameter of alf_task_wait identifies how many milliseconds the host should wait before an error occurs. The function returns immediately if the value equals 0. If the value is negative, it waits for all the accelerators to finish.

The last required function is alf_exit. This deallocates all ALF resources and ends the operation of the ALF runtime. Its signature is given by the following:

int alf_exit(alf_handle_t alf_handle, ALF_EXIT_POLICY_T policy, int timeout);

The policy parameter specifies what should happen when the function executes. The three possible options are as follows:

The timeout parameter of alf_exit is similar to the timeout parameter of alf_task_wait. If the value is positive, the runtime waits at most the specified number of milliseconds before exiting. If the value is 0, the function returns immediately. If timeout is less than 0, the function waits indefinitely if the policy parameter is set to ALF_EXIT_POLICY_WAIT.

In most applications, the execution environment starts processing at the function called main. The ALF runtime also invokes functions according to their names, but the actual names aren’t important: It’s the processing stage corresponding to the name. The five ALF stages are as follows:

Not all the stages are necessary, but if your accelerator code performs a stage, it must have a corresponding function. The function that performs Stage 1 is executed first, and the function whose name corresponds to Stage 5 is executed last. Stage 3, the computational kernel stage, is the most important and is the only stage required by the ALF runtime. Put simply, it’s the stage where input data is processed into output data.

The rest of the stages manage data before and after the computational kernel does its work. As described in the preceding section, the host can create an optional task context that holds data to be shared between the host and accelerator. If task contexts are used, the ALF runtime expects to find accelerator functions representing Stages 1 and 5, which manage the task context before and after processing.

ALF applications use DTLs to transfer large amounts of data. As described earlier, these are similar to DMA transfer lists, but provide additional buffer management. You can generate DTLs in the host application or in the accelerator application, and the preceding section explained how this is done on the host. Stage 2 creates DTLs that bring data into the accelerator, and Stage 4 creates DTLs that send data out of the accelerator.

Stages 1 and 5, if available, are processed only once for each task. Stages 2 and 4, if available, are processed once for single-use work blocks and multiple times for multiple-use work blocks. Similarly, Stage 3 is processed once for single-use work blocks and multiple times for multiple-use work blocks. Figure C.5 displays these processing stages for single-use work blocks.

Figure C.5. Stage processing on the accelerator

The rest of this section further explains what these stages actually do.

Before the ALF runtime can process these stages, it needs to know which functions represent them. To provide this information, you need to perform three steps:

The first step was briefly mentioned in the previous section. Of the 64-bit fields used by alf_task_desc_set_int64 to configure task descriptors, five of them identify function names on the accelerator. These are, in order

For example, to configure a task descriptor (task_desc_handle) to associate the merge function with Stage 5 (task context merge), you’d use the following code:

char ctx_setup_name[] = "merge";

alf_task_desc_set_int64(task_desc_handle, ALF_TASK_DESC_ACCEL_CTX_MERGE_REF_L, ctx_setup_name);

The second of the three steps is more involved. The ALF runtime presents each function representing a stage with a specific list of parameters. The number and types of these parameters differ from stage to stage.

int stage1_function(void* task_context);

int stage2_function(void* task_context, void* param_context, void* dtl, unsigned int current_iter, unsigned int num_iter);

ALF_ACCEL_DTL_BEGIN(dtl, ALF_BUF_IN, 0);
ALF_ACCEL_DTL_ENTRY_ADD(dtl, 100, ALF_DATA_FLOAT, host_addr_1);
ALF_ACCEL_DTL_ENTRY_ADD (dtl, 100, ALF_DATA_FLOAT, host_addr_2);
ALF_ACCEL_DTL_END(dtl);

int stage3_function (void* task_context, void *parameter_context, void *input_buffer, void *output_buffer, void* inout_buffer, unsigned int current_iter, unsigned int num_iter);

int stage4_function(void* task_context, void* param_context, void* dtl, unsigned int current_iter, unsigned int num_iter);

ALF_ACCEL_DTL_BEGIN(dtl, ALF_BUF_ OUT, 0);
ALF_ACCEL_DTL_ENTRY_ADD(dtl, 500, ALF_DATA_INT32, ea_1);
ALF_ACCEL_DTL_ENTRY_ADD (dtl, 100, ALF_DATA_INT32, ea_2);
ALF_ACCEL_DTL_END(dtl);

int stage5_function(void* source_context, void* target_context);

You need to take one last step to tell the ALF runtime what functions correspond to the accelerator stages. In your accelerator code, you need to insert a series of macros that export your function names to the ALF runtime. The section starts with ALF_ACCEL_EXPORT_API_LIST_BEGIN and ends with ALF_ACCEL_EXPORT_API_LIST_END. Neither of these macros takes any arguments.

Inside the section, ALF_ACCEL_EXPORT_API must be inserted for each function name corresponding to a stage. This macro accepts two parameters, both of type char*. The first identifies the API as a whole, and the second contains the name of a function that needs to be exported. The first argument is usually set blank, or "".

For example, suppose your application uses a task context but doesn’t create any data transfer lists. In this case, Stages 1, 3, and 5 must be implemented. If the corresponding functions are named stage1_function, stage3_function, and stage5_function, an acceptable export definition section would be this:

ALF_ACCEL_EXPORT_API_LIST_BEGIN
  ALF_ACCEL_EXPORT_API("", stage1_function);
  ALF_ACCEL_EXPORT_API("", stage3_function);
  ALF_ACCEL_EXPORT_API("", stage5_function);
ALF_ACCEL_EXPORT_API_LIST_END

This section is commonly placed at the end of the accelerator code. Keep in mind that these function names must be the same as those identified in the task descriptor.

Before I finish discussing the accelerator API, I need to explain two last functions: alf_num_instances and alf_instance_id. These provide accelerators with information about themselves and the processing environment.

The first, alf_num_instances, tells the accelerator how many task instances are being created. The second function, alf_instance_id, provides the accelerator with its rank among the task instances. This is a number between 0 and alf_num_instances - 1.

The code in Listing C.2 creates a task containing four work blocks and distributes them to two accelerators. It allocates memory for a task context and a parameter context, and then sets the contexts’ contents equal to char arrays. It tells the ALF runtime that the Stage 1 function is called context_setup, the Stage 3 function is called kernel, and the Stage 5 function is called context_merge.

#include <stdio.h>
#include <string.h>
#include "alf.h"

#define NUM_ACCEL 2

int main(int argc, char **argv) {

   /* ALF data structures */
   alf_handle_t alf_handle;
   alf_task_handle_t task_handle;
   alf_task_desc_handle_t td_handle;
   alf_wb_handle_t wb_handle;

   /* Names of SPU objects and functions */
   char library_name[] = "spu_alf_text.so";
   char spu_image_name[] = "spu_alf_text";
   char kernel_name[] = "kernel";
   char ctx_setup_name[] = "context_setup";
   char ctx_merge_name[] = "context_merge";

   /* Create messages to send to accelerator */
   char task_msg[] = "This is the task context";
   char param_msg[32];

   /* Initialize the ALF handle and # of accelerators */
   alf_init(NULL, &alf_handle);
   alf_num_instances_set(alf_handle, NUM_ACCEL);

   /* Create and configure the task descriptor */
   alf_task_desc_create(alf_handle, 0, &td_handle);
   alf_task_desc_set_int64(td_handle,
      ALF_TASK_DESC_ACCEL_IMAGE_REF_L,
      (unsigned long long)spu_image_name);
   alf_task_desc_set_int64(td_handle,
      ALF_TASK_DESC_ACCEL_LIBRARY_REF_L,
      (unsigned long long)library_name);
   alf_task_desc_set_int64(td_handle,
      ALF_TASK_DESC_ACCEL_KERNEL_REF_L,
      (unsigned long long)kernel_name);
   alf_task_desc_set_int64(td_handle,
      ALF_TASK_DESC_ACCEL_CTX_SETUP_REF_L,
      (unsigned long long)ctx_setup_name);
   alf_task_desc_set_int64(td_handle,
      ALF_TASK_DESC_ACCEL_CTX_MERGE_REF_L,
      (unsigned long long)ctx_merge_name);

   /* Configure the memory requirements
      and add the task context */
   alf_task_desc_set_int32(td_handle,
      ALF_TASK_DESC_MAX_STACK_SIZE, 4096);
   alf_task_desc_set_int32(td_handle,
      ALF_TASK_DESC_TSK_CTX_SIZE, sizeof(task_msg));
   alf_task_desc_set_int32(td_handle,
      ALF_TASK_DESC_WB_PARM_CTX_BUF_SIZE, sizeof(param_msg));
   alf_task_desc_ctx_entry_add(td_handle, ALF_DATA_BYTE,
      sizeof(task_msg));

   /* Create the task for cyclic work block distribution */
   int ret;
   if ((ret = alf_task_create(td_handle, (void*)task_msg,
      NUM_ACCEL, ALF_TASK_ATTR_SCHED_FIXED |
      ALF_TASK_ATTR_WB_CYCLIC, 1, &task_handle)) < 0) {
      fprintf(stderr, "alf_task_create failed, ret=%d
", ret);
      return 1;
   }

   /* Create work blocks and add the parameter context */
   int i;
   for (i=0; i<NUM_ACCEL*2; i++) {
      sprintf(param_msg, "This is parameter context %d", i);
      alf_wb_create(task_handle, ALF_WB_SINGLE, 1, &wb_handle);
      alf_wb_parm_add(wb_handle, (void*)param_msg,
         sizeof(param_msg), ALF_DATA_BYTE, 3);
      alf_wb_enqueue(wb_handle);
   }

   /* Finalize the task, wait for it to finish, and exit */
   alf_task_finalize(task_handle);
   alf_task_wait(task_handle, -1);
   alf_task_desc_destroy(td_handle);
   alf_task_destroy(task_handle);
   alf_exit(alf_handle, ALF_EXIT_POLICY_FORCE, 0);

   /* Display the changed task context */
   printf("Host: %s
",task_msg);
   return 0;
}

The alf_task_create function is particularly important. It tells the runtime to distribute the work blocks cyclically across all the accelerators. Because of the many possible errors involved in creating a task, this is the only function ALF whose return value is checked.

The application creates twice as many work blocks as there are accelerators. Each work block receives a parameter context that contains a message specific to the work block. This parameter context is aligned on a 2³-byte boundary.

Listing C.3 presents code to run on the accelerator. The operation is simple: The context_setup function prints a message identifying the task instance, the kernel function prints the content of the task context and the parameter context, and context_merge prints a message and changes the content of the task context.

#include <stdio.h>
#include <string.h>
#include <spu_mfcio.h>
#include <alf_accel.h>

/* Stage 1: Preprocess */
int context_setup(void *task_ctx) {
   printf("Accelerator %d of %d: Running context_setup.
",
      alf_accel_instance_id(), alf_accel_num_instances());
   return 0;
}

/* Stage 3: Display the task and parameter contexts */
int kernel(void* task_context, void* param_context,
   void* in_buffer, void* out_buffer, void* inout_buffer,
   unsigned int current_count, unsigned int total_count) {

   /* Determine the ID of the accelerator and their number */
   int id = alf_accel_instance_id();
   int num_tasks = alf_accel_num_instances();
   printf("Accelerator %d of %d: %s
",
      alf_accel_instance_id(), alf_accel_num_instances(),
      (char*)task_context);
   printf("Accelerator %d of %d: %s
", id, num_tasks,
      (char*)param_context);
   return 0;
}

/* Stage 5: Postprocess, update the task context */
int context_merge(void* source_context, void* target_context) {

   printf("Accelerator %d of %d: Running context_merge.
",
      alf_accel_instance_id(), alf_accel_num_instances());
   strcpy(target_context, "New task context.");
   return 0;
}

/* Use macros to export function names */
ALF_ACCEL_EXPORT_API_LIST_BEGIN
   ALF_ACCEL_EXPORT_API("", context_setup);
   ALF_ACCEL_EXPORT_API("", kernel);
   ALF_ACCEL_EXPORT_API("", context_merge);
ALF_ACCEL_EXPORT_API_LIST_END

The displayed results are as follows:

Accelerator 0 of 2: Running context_setup
Accelerator 1 of 2: Running context_setup
Accelerator 0 of 2: This is the task context
Accelerator 0 of 2: This is parameter context 0
Accelerator 0 of 2: This is the task context
Accelerator 0 of 2: This is parameter context 2
Accelerator 1 of 2: This is the task context
Accelerator 1 of 2: This is parameter context 1
Accelerator 1 of 2: This is the task context
Accelerator 1 of 2: This is parameter context 3
Accelerator 0 of 2: Running context_merge
Host: New task context

The results may differ slightly from execution to execution, but Accelerator 0 always receives Work Blocks 0 and 2, and Accelerator 1 always receives Work Blocks 1 and 3. This is because the ALF runtime distributes the work blocks cyclically. If alf_task_create is configured to use a different method of distribution, the work blocks are not necessarily sent to all the accelerators in order.

The context_setup and kernel functions simply provide output. The kernel function displays the parameter context, which changes from work block to work block, and the task context, which always remains the same. But context_merge does more than send text to standard output; it accesses the target_context pointer and uses strcpy to transfer a new char array into the task context. When the host prints the context after the task completes, it now reads "New task context" rather than "This is the task context".

The previous application transferred information to the accelerator through task and parameter context structures, but this isn’t suitable for data-intensive applications. To transfer large amounts of data, you need DTLs. The ALF runtime uses DTLs to form DMA lists, which transfer data to and from buffers in accelerator memory.

The host_matrix and acc_matrix projects both demonstrate how DTLs work. They take two 512x512 float matrices (matrix_one and matrix_two) as input and perform the same tasks:

The difference between the two projects is that the host_matrix application partitions data on the host, and acc_matrix partitions data on the accelerator.

/* Allocate memory for the input buffer (matrix_one) */
alf_task_desc_set_int32(td_handle, ALF_TASK_DESC_WB_IN_BUF_SIZE,
   N*N*sizeof(float)/NUM_BLOCKS);

/* Allocate memory for the inout buffer (matrix_two) */
alf_task_desc_set_int32(td_handle, ALF_TASK_DESC_WB_INOUT_BUF_SIZE,
   N*N*sizeof(float)/NUM_BLOCKS);

/* Allocate memory for the output buffer (matrix_three) */
alf_task_desc_set_int32(td_handle, ALF_TASK_DESC_WB_OUT_BUF_SIZE,
   N*N*sizeof(float)/NUM_BLOCKS);

for (i=0; i<NUM_BLOCKS; i++) {

   /* Create the work block */
   alf_wb_create(task_handle, ALF_WB_SINGLE, 0, &wb_handle);

   /* DTL: store matrix_one in an input buffer */
   alf_wb_dtl_begin(wb_handle, ALF_BUF_IN, 0);
   alf_wb_dtl_entry_add(wb_handle,
      &matrix_one[i*N/NUM_BLOCKS][0],
      N*N/NUM_BLOCKS, ALF_DATA_FLOAT);
   alf_wb_dtl_end(wb_handle);

   /* DTL: store matrix_two in an inout buffer */
   alf_wb_dtl_begin(wb_handle, ALF_BUF_OVL_INOUT, 0);
   alf_wb_dtl_entry_add(wb_handle,
      &matrix_two[i*N/NUM_BLOCKS][0],
      N*N/NUM_BLOCKS, ALF_DATA_FLOAT);
   alf_wb_dtl_end(wb_handle);

   /* DTL: store matrix_three in an output buffer */
   alf_wb_dtl_begin(wb_handle, ALF_BUF_OUT, 0);
   alf_wb_dtl_entry_add(wb_handle,
      &matrix_three[i*N/NUM_BLOCKS][0],
      N*N/NUM_BLOCKS, ALF_DATA_FLOAT);
   alf_wb_dtl_end(wb_handle);

   /* Enqueue the work block in the task */
   alf_wb_enqueue(wb_handle);
}

alf_task_desc_set_int32(td_handle, ALF_TASK_DESC_PARTITION_ON_ACCEL, 1);

/* creating wb and adding param & io buffer */
param_ctx.num_floats = num_floats;
for (i=0; i<N; i+=N/NUM_BLOCKS) {

   /* Create the work block */
   alf_wb_create(task_handle, ALF_WB_SINGLE, 0, &wb_handle);

   /* Configure and add parameter to the work block */
   param_ctx.matrix_one = (unsigned long long)&matrix_one[i][0];
   param_ctx.matrix_two = (unsigned long long)&matrix_two[i][0];
   param_ctx.matrix_three = (unsigned long long)&matrix_three[i][0];
   alf_wb_parm_add(wb_handle, (void *)(&param_ctx),
      sizeof(param_ctx), ALF_DATA_BYTE, 0);

   /* Enqueue the work block in the task */
   alf_wb_enqueue(wb_handle);
}

int create_in_dtl(void *task_ctx,
                  void *param_ctx,
                  void *dtl,
                  unsigned int current_count,
                  unsigned int total_count)
{
   /* Access the parameter context */
   ctx_struct* ctx = (ctx_struct *) param_ctx;

   /* Create input DTL for matrix_one */
   ALF_ACCEL_DTL_BEGIN(dtl, ALF_BUF_IN, 0);
   ALF_ACCEL_DTL_ENTRY_ADD(dtl, ctx->num_floats,
      ALF_DATA_FLOAT, ctx->matrix_one);
   ALF_ACCEL_DTL_END(dtl);

   /* Create inout DTL for matrix_two */
   ALF_ACCEL_DTL_BEGIN(dtl, ALF_BUF_OVL_INOUT, 0);
   ALF_ACCEL_DTL_ENTRY_ADD(dtl, ctx->num_floats,
      ALF_DATA_FLOAT, ctx->matrix_two);
   ALF_ACCEL_DTL_END(dtl);

   return 0;
}

The ALF API provides the alf_task_depends_on function to ensure that one task starts only when a second finishes. For example, if task1, task2, and task3 should be performed in strict sequence, you’d use the following code:

alf_task_depends_on (task2, task1);
alf_task_depends_on (task3, task2);

The first line makes task2 dependent on task1, and the second makes task3 dependent on task2. If alf_task_wait(task3) is called, the ALF runtime waits for task1, task2, and task3 to finish in order. Note that alf_task_depends_on must be called after the tasks are created but before any of their work blocks are enqueued.

It’s important to understand that this capability can create task sequences, but not task pipelining. The difference is that, in a multistage pipeline, each stage needs to repeat itself after finishing. But ALF provides no way to make a task dependent on itself, and no way to make the work blocks of a task dependent on the work blocks of another task.

Chapter 7 explains how libspe makes it possible to handle SPU events in PPU code: Create an event handler, create events, and then register the events with the handler. Handling events in ALF is even simpler. The alf_task_event_handler_register function matches an event-handling function to an event type and a task. When the task produces the specified event, the ALF runtime invokes the event handler. alf_task_event_handler_register accepts five parameters:

The event-handling function pointed to by the second parameter is called when an event of the appropriate type occurs. An event handler must have a specific signature:

int task_event_handler(alf_task_handle_t task_handle, ALF_TASK_EVENT_TYPE_T event, void* p_data)

The first parameter is the handle to the task being monitored, and the third parameter points to supplemental data supplied in alf_task_event_handler_register. The second parameter identifies what type of event or events the handler responds to. This event mask can take the values listed in Table C.7.

It’s important to see the distinction between a finalized task and a finished task. A task is finalized when all work blocks have been enqueued and no further alteration is possible. It is finished when all of its work blocks have completed their execution on the accelerators. These events deal with the status of tasks and work blocks, not the status of the accelerators.

As an example, let’s say function handler should be called just before a task (task_handle) is ready to be executed. No supplemental data is needed. First, you’d register the event handler with the following:

alf_task_event_handler_register(task_handle, &handler, NULL, 0, ALF_TASK_EVENT_READY);

Then you’d write code for handler, whose signature should be similar to this:

int handler(alf_task_handle_t task_handle, unsigned int event, void* data)

A task’s event handler must be registered before the task is finalized. In the alf_events project, alf_task_event_handler_register is called immediately after the task is created. It tells the ALF runtime that the example_handler function should be called if any of the ALF_TASK_EVENT_FINALIZED, ALF_TASK_EVENT_READY, or ALF_TASK_EVENT_FINISHED events occur.

When example_handler is invoked, it displays output depending on which event has fired. This is shown in the following code:

int example_handler(alf_task_handle_t tk_handle, unsigned int event, void* data) {

   /* select event type and respond */
   switch(event) {
      case ALF_TASK_EVENT_FINALIZED:
         printf("The task is finalized.
"); break;
      case ALF_TASK_EVENT_READY:
         printf("The task is ready.
"); break;
      case ALF_TASK_EVENT_FINISHED:
         printf("The task is finished.
"); break;
      default:
         printf("Invalid event.
");
   }
   return 0;
}

The ALF runtime calls the event handler after each of the three events, and then returns to regular processing.