Privilege Levels

All modern embedded processors provide a mechanism to allow portions of the software to operate with differing levels of privilege. The current privilege level is used by the system to control access to resources and execution of certain instructions. The number and specific use of privilege levels are architecture specific, but most architectures support a minimum of two privilege levels. The highest privilege level is usually reserved for the operating system. User programs and applications typically run with a lower privilege level. The use of privilege levels increases the robustness of the system by reducing the ability of an application to interfere with system-wide resources. For example, the ability to disable interrupts is a privileged instruction and ideally not accessible directly by an application.

Intel processors provide four privilege levels, although in practice level zero and level three are predominantly used. The current privilege level (CPL) of the processor is stored in the lowest 2 bits of the code segment selector (CS). The highest privilege level is number zero. This level is commonly known as Kernel Mode for Linux and Ring 0 for Windows-based operating systems. A CPL of three is used for user space programs in both Linux and Windows.

Many processors simply grant privileges to execute system level instructions or access system level resources by being at a privileged supervisor level. However, on Intel architecture processors some further details are required to understand how an Intel processor decides whether an operation is allowed. Whether an instruction has sufficient privilege to perform a specific operation is established by comparing the CPL to the active I/O Privilege Level (IOPL) The IOPL is stored in bits 12 and 13 of the FLAGS register and its value is controlled by the operating system. If the CPL is less than the current IOPL, then the privileged operation is allowed; if greater than or equal to IOPL, then a privileged operation will fail. On an Intel processor the CPL value is stored in the low 2 bits of the Code Segment register. Most operating systems set the IOPL value to three, thus having a CPL value of three corresponds to the lowest privilege level allowed in the system, and a CPL value of zero is the highest privilege level. As an embedded systems programmer, you might often require direct access to a hardware resource from your application. Most operating systems provide a mechanism to alter the privilege level, and a root privileged application, for example, can call the IOPL() function on Linux to altar the IOPL flags. To increase the security of your application you should minimize the time where the code executes with higher privilege level, and you should not run the entire application with increased privileges.

Floating-Point Units

Floating-point mathematical operations are required in a wide range of embedded applications, such as control systems and digital signal analysis. Where an application does require floating point, the performance targets often dictate the need for a hardware-based floating-point unit. Most embedded processor SOCs offer a version that provides hardware-based floating point. A key attribute of a floating point acceleration function associated with the processor is whether the floating-point unit is compliant with the IEEE Standard 754 for Binary Floating-Point Arithmetic. The precision of the floating-point unit (single/double) is also an important attribute in developing the floating-point algorithms.

Intel processors have two floating-point units. The first and probably best known is the x87 Floating-Point Unit (FPU). The x87 FPU instructions operate on floating-point, integer, and binary-coded decimal (BCD) operands. It supports 80-bit precision, double extended floating-point. The FPU operates as a coprocessor to the instruction unit. The FPU instructions are submitted to the FPU and the scalar (main processor) pipeline continues to run in parallel. To maximize overall application performance it is important to ensure that the processor’s main execution flow can perform useful work while the FPU performs the floating-point operations. To that end it is usually best to use a compiler to generate the target code to ensure efficient instruction scheduling.

Not all floating-point operations can be completed; for example, dividing a number by zero results in a floating-point fault. The following floating-point operations result in faults: all invalid operations, for example, square root of a negative number, overflow, underflow, and inexact result. The operating system provides an exception handler to handle the floating-point fault exceptions. In the case of Linux, the kernel catches the fault and sends a user space signal (SIGFPE, signal floating-point exception). The application will be terminated unless the application has chosen to handle the exceptions. The C language specification as defined by the ISO C99 (ISO/IEC 9899:1999) standardizes a number of functions to control the behavior of floating-point rounding and exception handling. One such function is fegetexceptflag().

Intel processors also have a Single Instruction Multiple Data (SIMD) execution engine. The Intel Atom processor supports the Supplemental Streaming SIMD Extensions 3 (SSSE3) version of the SIMD instructions, which support integer, single, and double precession floating-point units.

The Intel Atom processor has a rich set of floating-point coprocessor capabilities. As a result, a particular algorithm could be implemented in a number of ways. The trade-offs for the use of each floating-point unit for a particular operation are described in Chapter 11, “Digital Signal Processing.”

The floating-point units are resources that contain a number of registers and some current state information. When different software threads wish to use the resources, the kernel software may have to save and restore the registers and all state information during operating system context switches. The Intel processor provides FXSAVE and FXRSTOR to save and restore the required state and register information of FP and SSE units. These operations can be costly if performed on every task transition, so Intel processors provide a mechanism to help the kernel identify whether an FPU was actually used by a particular process. The TS flag in the control register zero (CS0.TS) provides an indication that the floating point unit has been used. The kernel can clear the value when it performs a context switch, and check if the bit has been set during the execution of the process (indicating the process used the FP unit). The operating system can be configured to save the registers and state on transition from a thread that used the resource or alternatively raise an exception when a new thread attempts to use the resource after a previous thread has used it. If the real-time behavior of your FP/SSE code is important, you should look into the detailed operating system behavior. You may have to take special steps if you want to use floating-point or SSE units from within the kernel. For example, in Linux you have to call kernel_fpu_begin() and kernel_fpu_end() around the code that uses the FP/SSE units. This will save and restore the required state.

Processor Specifics

Practically all embedded processors have been in existence for a number of generations, though none of the existing embedded processors quite match the longevity of the Intel architecture. As each product generation evolves, new capabilities are introduced, and as an embedded systems developer you will need to establish exactly which features are supported on the particular version you are working with. Having mechanisms to identity product capabilities at runtime facilitates the development of code that will run on a number of different generations of the part without modification. The information is typically provided by special registers that are accessed via dedicated instructions. For example, on ARM platforms, there are a number of co-processor registers one of which is the System Control co-Processor. The system control coprocessor is known as CP15 and provides information and controls the processor behavior. On Intel platforms, a number of control/information registers, CR0, CR1, CR2, CR3, and CR4, and a special instruction (CPUID) are available. The CPUID instruction and registers describe capabilities of the processor and also provide some current processor state information. The CPUID instruction provides detailed information on the executing processor. The output from the CPUID instruction is dependent upon the contents of the EAX register; by placing different values in the EAX register and then executing the CPUID instruction, the CPUID instruction performs the function defined by the EAX value. The CPUID instruction returns the requested values in the EAX/EBX/EDX and ECX registers. The code segment below uses GCC compiler inline assembly code to dump all the CPUID values available on an Intel Atom platform.

long maxCpuId=0 , long cpuid;

long eax = 0, ebx = 0, ecx=0, edx=0;

eax = 0;  /∗ Get the maximum CPUID range ∗/

/∗

 ∗ Input:Loads eax input register with eax variable value

 ∗ Call CPUID instrucion

 ∗ Output:Ensure EAX/EBX/EXC and EDX registers are mapped

 ∗ the variables with the corresponding name.

 ∗/

__asm__ __volatile__

 ("CPUID"

  :"=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx)

  :"a"(eax)

 );

 maxCpuId=eax; /∗ Get the maximum value for EAX∗/

 for ( cpuid=0; cpuid <= maxCpuId ; ++cpuid)

 {

   eax = cpuid;

   __asm__ __volatile__("CPUID"

           :"=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx)

           :"a"(eax)

    );

   printf("CPUID [%2X] = %08X, %08X, %08X, %08X ",cpuid, eax,ebx,ecx,edx);

 }

The code above displays the various CPUID values, which can be decoded to provide detailed information about the processor. A good example of the type of information provided is the processor feature list. This is provided by calling CPUID with an EAX value of 01h. The processor features capabilities listed on an Atom processor returned the following values: EDX[31:0] = 0xBFE9FBFF and ECX[31:0] = 0x0040C3BD. Decoding the bits set in EDX/ECX indicates the following feature set on the Intel Atom processor: Floating-Point Unit, Time Stamp Counter, Physical Address Extension, Local Interrupt Controller (APIC), Cache Line Flush, Model Specific Registers, FXSAVE/FXSTOR, MMX, SSE, SSE2, SSE3, SSSE3, Thermal Management, Multi-Threading – (Hyper thread). The support for 64-bit operation (EM64T) is also indicated through the CPUID feature.

It is often useful to have an awareness of the cache structures on your platform, especially when it comes to performance tuning or at least understanding the behavior of your application and the correlation between application data set size and overall cache sizes. Chapter 18, “Platform Tuning,” provides more specific details on tuning your system. The CPUID provides cache structure details of the processor. The decoded values returned from the current Intel Atom processor are as follows:

The Linux command cat/proc/cpuinfo provides similar details relating to the CPU. A portion of the details presented is obtained by issuing using the CPUID instruction.

The provision of free running counters and hardware timers is required by most operating systems to provide the operating system “Tick,” timed delay loops, code performance measurements, and the like. Intel Atom platforms provide a wealth of timers and counters on the platform. A very useful free running time stamp counter is provided on Intel platforms. It is simply accessed by using the RDTSC instruction. The instruction returns the number of ticks at maximum processor frequency (resolved at boot up) since reset. The tick represents the length of time for one duty cycle of the CPU clock, so for a processor running at 1.6 GHz, a tick would represent 0.625 ns. The value is very reliable, but you should be aware that in some system interactions may skew some measurements, such as when the processor’s actual speed is slowed due to power-managed controls, although this is not the case for Atom processors. The code segment below shows how the time stamp counter can be read from a C program. The embedded assembly is written to be compiled by the GCC compiler.

long long timeStamp;

__asm__ __volatile__

 ("RDTSC"          ; Mnemonic to read counter

  :"=A"(timeStamp)  ; Map 64bit timeStamp to EDX/EAX

  :

 );

In addition to the free running counter, embedded platforms will make use of hardware-based timers. The timers on the Intel Atom platform are described in Chapter 4, “Embedded Platform Architecture.”

Immediate Operands

Some instructions use data encoded in the instruction itself as a source operand. The operands are called immediate operands. For example, the following instruction loads the EAX register with zero.

MOV EAX, 00

The maximum value of an immediate operand varies among instructions, but it can never be greater than 2³². The maximum size of an immediate on RISC architecture is much lower; for example, on the ARM architecture the maximum size of an immediate is 12 bits as the instruction size is fixed at 32 bits. The concept of a literal pool is commonly used on RISC processors to get around this limitation. In this case the 32-bit value to be stored into a register is a data value held as part of the code section (in an area set aside for literals, often at the end of the object file). The RISC instruction loads the register with a load program counter relative operation to read the 32-bit data value into the register.

Register Operands

Source and destination operands can be any of the follow registers depending on the instruction being executed:

On RISC embedded processors, there are generally fewer limitations in the registers that can be used by instructions. IA-32 often reduces the registers that can be used as operands for certain instructions.

Memory Operands

Source and destination operands in memory are referenced by means of a segment selector and an offset. On embedded operating systems, the segment selector often results in a base address of zero, particularly if virtual memory is used, so the memory address specified by the operand degenerates to being the offset value. The segment selector is automatically chosen by the processor, but can be overridden if needed. The following instruction moves the value in EAX to the address pointed by EBX, assuming the data segment selector contains zero. It is the simplest memory operand form.

MOV [EBX], EAX

The memory operand can also specify offsets to the base address specified in the memory operand. The offset is added to the base address (the general-purpose register) and can be made up from one or more of the following components:

So we have a memory operand that can consist of

Since the segment selector usually returns zero, the memory operand effective address becomes the following:

The compiler will make best use of these modes to de-reference data structures in memory or on the stack. The components of the offsets can be either positive or negative (two’s complement values), providing excellent flexibility in the memory operand address generation.

Data Transfer Instructions

All processes provide instructions to move data between registers, memory, and registers, and in some architectures between memory locations. Table 5.3 shows some instruction combinations.

Table 5.3. Data Transfer Instructions

There is also a set of instructions that provides hints to the underlying hardware to help manage the cache more efficiently. The MOVNTI (store double word using non-temporal hint) instruction is designed to minimize cache pollution; by writing a double word to memory without writing to the cache hierarchy, it also prevents allocation in the cache line. There are also PREFETCH instructions that perform memory reads and bring the result data closer to the processor core. The instruction includes a temporal hint to specify how close the data should be brought to it. These instructions are used when you are aggressively tuning your software and require some skill to use effectively. More details are provided in Chapter 18, “Platform Tuning.” These hints are optional; the processor may ignore them.

Arithmetic Instructions

The arithmetic instructions define the set of operations performed by the processor Arithmetic Logic Unit (ALU). The arithmetic instructions are further classified into binary, decimal, logical, shift/rotate, and bit/byte manipulation instructions.

Branch and Control Flow Instructions

We clearly need instructions to control the flow of a program’s execution. The branch and control flow instructions fall into two primary categories. The first is unconditional changes of program flow to a new program counter address. This occurs when a jump or call instruction is encountered. The second category of branch or control flow instructions are conditional branches or conditional execution of an instruction. The conditional execution of an instruction is dictated by the contents of bits within the EFLAGS register, or for some instructions the value in the ECX register.

Jump operations transfer control to a different point in the program stream without recording any return information. The destination operand specifies the address of the instruction we wish to execute next. The operand can be an immediate value, a register, or a memory location. Intel processors have several different jump modes that have evolved over time, but a number of modes are no longer used. The near jump is a jump within the current code segment. As we mentioned earlier, the current code segment often spans the entire linear memory range (such as zero to 4 GB). So all jumps are effective within the current code segment. The target operand specifies either an absolute offset (that is, an offset from the base of the code segment) or a relative offset (a signed displacement relative to the current value of the instruction pointer in the EIP register). A near jump to a relative offset of 8 bits is referred to as a short jump. The CS register is not changed on near and short jumps. An absolute offset is specified indirectly in a general-purpose register or a memory location. Absolute offsets are loaded directly into the EIP register. A relative offset is generally specified as a label in assembly code, but at the machine code level it is encoded as a signed 8-, 16-, or 32-bit immediate value. This value is added to the value in the EIP register. (Here, the EIP register contains the address of the instruction following the JMP instruction). Although this looks complicated, in practice the near jump is a simple branch with flexibility in specifying the target destination address. Intel processors also includes FAR jumps, which allow the program to jump to a different code segment, jump through a call gate with privilege checks, or a task switch (task in the IA-32 processor context). Table 5.9 shows the different instructions with examples.

Table 5.9. Program Flow—No Saved Return State

Call gates are sometimes used to support calls to operating system services; for instance, this is a configuration available in VxWorks for real-time tasks when calling operating system services. However, operating system calls are more usually provided via the software interrupt call.

Calling subroutines, functions, or procedures require the return address to be saved before the control is transferred to the new address; otherwise, there is no way for the processor to get back from the call. The CALL (call procedure) and RET (return from procedure) instructions allow a jump from one procedure (or subroutine) to another and a subsequent jump back (return) to the calling procedure. The CALL instruction transfers program control from the current procedure (the calling procedure) to another procedure (the called procedure). To allow a subsequent return to the calling procedure, the CALL instruction saves the current contents of the EIP register on the stack before jumping to the called procedure. The EIP register (prior to transferring program control) contains the address of the instruction following the CALL instruction. When this address is pushed on the stack, it is referred to as the return instruction pointer or return address. The address of the called procedure (the address of the first instruction in the procedure being jumped to) is specified in a CALL instruction in the same way it is in a JMP instruction (described above).

Most processors provide an instruction to allow a program to explicitly raise a specified interrupt. The INT instruction can raise any of the processor’s interrupts or exceptions by encoding the vector number or the interrupt or exception in the instruction or exception, which in turn causes the handler routine for the interrupt or exception to be called. This is typically used by user space programs to call operating system services. Table 5.10 shows the instructions that affect the program flow.

Table 5.10. Program Flow with Saved Return State

On Intel platforms there is quite a lot of history associated with the INT calls. Legacy (non-EFI) BIOS supports a number of INT calls to provide support to operating systems. An example of a well-known INT call is the E820. This is an interrupt call that the operating system can use to get a report of the memory map. The data are obtained by calling INT 15h while setting the AX register to E820h. For embedded programmers, there is an ever-decreasing dependence on the INT service provided by the BIOS. The Linux kernel reports the memory map reported by the BIOS in the dmesg logs at startup.

The BIOS environment is transitioning from a traditional legacy BIOS, which was first developed a few decades ago, to a more modern codebase. The newer codebase is known as Unified Extensible Firmware Interface (UEFI). At the time of writing, many products are transitioning from this legacy BIOS to EFI. More information on this topic can be found in Chapter 6.

Structure/Procedure Instructions

Modern languages such as C/C++ define a frame structure on the stack to allocate local variables and define how parameters are passed on the stack. The IA-32 provides two instructions to support the creation and management of these stack frames, namely, ENTER and LEAVE. The stack frame is discussed as part of the Application Binary Interface discussed earlier in this chapter.

SIMD Instructions

Several classes of embedded applications are a mix of traditional general-purpose (scalar) workloads with an additional moderate digital signaling workload. Most embedded processors provide extensions to their core instruction set to support these additional workloads. These instructions are known as Single Instruction Multiple Data (SIMD). On Intel platforms the extensions are known as Intel Streaming SIMD Extensions (Intel SSE). Other processors have more basic extensions to provide single multiply-and-accumulate (MAC) operations. This operation is the basic building block for a large number of simple DSP algorithms. Over time the range of SSE instructions has grown significantly, and in fact they can bring substantial performance benefits to a wide range of workloads:

Generally, code that contains small-sized repetitive loops that operate on sequential arrays of integers of 8, 16, or 32 bits, single-precision 32-bit floating-point data, and double-precision 64-bit floating-point data are a good candidate for tuning using SSE instructions. The repetitiveness of these loops incurs costly application processing time. However, these routines have the potential for increased performance when you convert them to use one of the SIMD technologies.

On the Intel Atom processor, the SSSE3 SIMD instructions are supported and the instructions operate on packed byte, word, and double-word integers, as well as single-precision floating point. A key efficiency of the SIMD instructions is the fact that a single instruction operates on a number of data elements in parallel. If your data structures can be structured to make use of SIMD instructions, significant improvements in performance can be obtained. Figure 5.4 shows a typical SIMD operation.

FIGURE 5.4 Typical SIMD Operation.

The following code sequence is an example of how to get the data into the XMM registers and have the SIMD operation performed. It is written in the Intel Compiler format to incorporate assembly into C code.

1 void add(float ∗a, float ∗b, float ∗c)

2   {

3    __asm {

4      mov eax, a

5      mov edx, b

6      mov ecx, c

7      movaps xmm0, XMMWORD PTR [eax]

8      addps xmm0, XMMWORD PTR [edx]

9      movaps XMMWORD PTR [ecx], xmm0

10  }

11

The first three instructions get pointers for a, b, and c into EAX, EDX, and ECX registers, respectively. The MOVAPS instruction in line 7 above moves four quad-words containing four packed single-precision floating-point values from memory pointed to by EAX (128 bytes) into the XMM0 register. The ADDPS instruction performs a SIMD add of the four packed single-precision floating-point values in XMM0 with the four packed single-precision floating-point values pointed to by the EDX register. The final MOVAPS instruction saves the XMM0 (128-byte contents) to the memory pointed to by the ECX register. This is far more efficient than the traditional scalar equivalent.

Precise and Imprecise Exceptions

Exceptions can be categorized as precise or imprecise. Precise exceptions are those that indicate precisely the address of the instruction that caused the exception. Again, the divide-by-zero exception is an excellent example of a precise exception because the faulting instruction can be identified. Imprecise exceptions, on the other hand, cannot directly be associated with an instruction. The processor has continued execution of an indeterminate number of instructions between the time the exception was triggered and when the processor processed it; alternatively, the exception was generated by an event that was not due to an instruction execution. An example of an imprecise exception is the detection of an uncorrectable ECC error discovered in a cache. Imprecise exceptions are not generally recoverable; although the Linux machine check handler does all it can to avoid a kernel panic and the resulting reboot, imprecise exceptions are referred to as aborts on Intel architectures. Precise exceptions fall into two categories on Intel architectures, faults and traps:

You may have noticed that the fault handler points to the faulting instruction for faults; this is because the handler is likely to rerun the faulting instruction once the underlying reason for the fault is resolved. For example, if a page fault is generated, the operating system will load the page from disk, set up the page table to map the page, and then rerun the instruction. Instructions that generate a trap, on the other hand, are not rerun. On other embedded platforms such as ARM, the fault address recorded on the exception is always that of the next instruction to run. However, when all instructions are the same size (32-bits) it’s a trivial matter to establish the faulting instruction; it’s not quite as straightforward with a variable size instruction set. The list of exceptions and interrupts that a processor supports is part of a processor’s architecture definition. All exceptions and interrupts are assigned a vector number. The processor uses the vector assigned to the exception or interrupt as an index into a vector table. On Intel architectures this vector table is called the Interrupt Descriptor Table (IDT). The table provides the entry point to the exception or interrupt handler. IA-32 defines an allowable vector range of 0 to 255. Vectors from 0 to 31 are reserved for architecture-defined exceptions and interrupts. Vectors 32 to 255 are designed as user-defined interrupts. The user-defined vectors are typically allocated to external (to the processor) hardware generated interrupts and software interrupts. Table 5.11 is a partial list of the IA-32 Protected mode exceptions and interrupts. See Chapter 5 of IA-32 Intel Architecture Systems Programming Guide.

Table 5.11. Abbreviated IA-32 Exceptions and Interrupts

In summary, exceptions and interrupts can come from a wide range of sources from the processor core, caches, floating-point units, bus interfaces, and external peripherals. Figure 5.5 shows the wide range of sources.

FIGURE 5.5 Exception and Interrupt Sources.

Translation Caching

As you may predict, the translation from a virtual address to a physical address would be a very costly event if it were required for every single memory transaction. For the Intel architecture each translation could result in two dependent reads. To reduce the impact of translation, processors generally employ techniques to cache the translation and additional details it provides in a translation cache. This is usually very effective given that page table access exhibits excellent temporal locality. These are generally known as translation look-aside buffers (TLBs). The TLB is typically constructed as a fully or highly associative cache, where the virtual address is compared against all cache entries. If the TLB hits, the contents of the TLB are used for the translation, access permissions, and so on. The management of the TLB is shared between the operating system and hardware. The TLB will allocate space automatically for new entries, typically by employing a (Least Recently Used) LRU policy for replacement. There is no hardware coherency enforcement between the TLBs and the page tables in memory. If software modifies a page or table entry (e.g., for resetting Accessed bit or changing the mapping) directly, then the TLBs must be invalidated. Otherwise, there is a risk that an out-of-date translation is used. The INVLPG instruction invalidates a single TLB entry; software writes to the CR3 register can be used to invalidate the entire TLB. On some processors, the TLB is managed in software with hardware-assist functions to perform the page walks.

An optimization can improve the effectiveness of the TLB during process context switches (kernel/user and user/user). The page tables provide a Global bit in the page entries. If this is set, then the page table entry is not flushed on a global TLB flush event.

The current Intel Atom processor has a number of separate TLB structures:

As an embedded software developer, you may find yourself tuning the overall platform execution. High TLB miss rates can contribute to poor overall system performance. It is prudent to review performance counters in the platform and focus on the overall TLB miss rate. There are a few strategies to reduce the overall system TLB miss rate. The first is to consolidate program and data hot spots into a minimum memory footprint, which reduces the number of pages and subsequently TLB entries needed during the hot spot execution of the platform software. This can be achieved by linking the hot spot objects close together. An additional technique to consider is using large pages to cover hot spot areas. The ability to use pages other than the standard 4 kB is dependent on the operating system providing such control. A Linux kernel feature known as HughTLB provides a memory allocation mechanism that allocates a contiguous piece of physical memory that is covered by a single large TLB entry. If you have an application that jumps around (a nontechnical term for exhibits poor locality) a large data structure, using a large page entry could significantly reduce the TLB misses.

Local Memory

On some embedded SOC devices dedicated SRAM is provided on die. The SRAM device is mapped to a unique address in the memory map and must be explicitly managed by the application or operating system. This is usually achieved by using a specific tool chain linker section and mapping specific data structures with your application to this tool chain linker section. The access times are usually similar to that of an L1 cache (perhaps a little faster because there are no tag lookups required). The contents that you place in these RAM devices are really a function of the application; they are often used to increase the determinism of interrupt handing software because access to the memory cannot miss, as is the case for caches. These embedded SOC SRAMs are not usually coherent with the remainder of the I/O system. This local memory is sometimes called tightly coupled memory (TCM).

Cache Hierarchy

The fastest memory closest to the processor is typically structured as caches. A cache is a memory structure that stores a copy of the data that appear in main memory (or the next level in the hierarchy). The copy in the cache may at times be different from the copy in main memory. When a memory transaction is generated by the processor (reads/writes), the cache is searched to see if it contains the data for the requested address. The address used in the cache structure may be a virtual or physical address depending on whether the cache is situated before the MMU translation or after. For example, the level one cache of the Intel XScale processor is addressed using virtual addresses. Caches in the Intel Architecture processors are all addressed with physical addresses. The term tag is used to refer to the fields within the address that are used to search the cache for a match.

Caches are structures that take advantage of a program’s temporal and spatial locality:

The caches themselves are often organized in a hierarchy, again with increasing access times and increased size. Even though the technology of the SRAMs is usually similar for L1 and L2 caches, the L2 cache takes longer to return the contents of a hit as it is farther from the core, and it takes longer to search a larger cache. The L1 cache is usually close to the same frequency as the core, whereas the L2 is often clocked at a slower speed.

Theoretically, cache structures can be organized in many different forms. Two extreme implementations are the following:

In reality, neither approach is feasible; the cache structures are a hybrid of both the direct mapped and fully associative. The structures created on Intel processors are N-way set associative, where N is a different number based on the product and level of the cache. The (physical) address used to look up a set associative cache is divided into three separate fields:

Figure 5.11 shows the logical structure of a 24-K Intel Atom instruction cache.

FIGURE 5.11 Six-Way Set Associative 24-K Data Cache.

As a software designer, the cache structure can be largely transparent; however, an awareness of the structure can help greatly when you start to optimize the code for performance. At a minimum, be aware of the cache line size and structure your data such that commonly used elements fall within the same cache line. The Intel Atom platform has the following caches, all with a cache line size of 64 bytes:

Cache Coherency

A number of agents other than the processor access system memory. For example, a peripheral such as a network interface must read and write system memory through direct memory access to transmit and receive packets. When the processor generates a network packet to be transmitted on the network, the system must ensure that the DMA engines of the network interface get the most recent consistent data when it reads system memory for the packet. Conversely, when a processor reads data written by the network card, it should get the most recent content from the memory. The mechanism that manages the data consistency is known as cache coherency.

To ensure that the contents read by the DMA engine are correct, there are two approaches. The first is to ensure that the system memory contents reflect the latest payload before the NIC issues the DMA read. The NIC driver could use software instruction to flush all cache lines associated with packet. This is a costly operation, but not untypical on some embedded SOC platforms. The second approach calls for dedicated logic to search the cache when other agents are reading and writing to system memory.

Managing coherency using software is the responsibility of each device driver on the platform. The operating system typically provides calls that the driver must call to ensure consistency. For an example of an Ethernet device driver that manages coherence in software, you can review the Linux Coldfire fast Ethernet Driver linux-source-2.6.24/drivers/net/fec.c. The following code segment flushes the dcache lines associated with the packet to ensure memory is consistent before transmission.

flush_dcache_range((unsigned long)skb->data,

           (unsigned long)skb->data + skb->len);

On Intel platforms, the hardware maintains coherence by snooping the memory transactions to ensure consistency. The processor maintains cache consistency with the MESI (Modified, Exclusive, Shared, Invalid) protocol. Cache consistency is maintained for I/O agents and other processors (with caches). MESI is used to allow the cache to decide whether a memory entry should be updated or invalidated. Two functions are performed to allow its internal cache to stay consistent:

These mechanisms ensure that data read or written to the system memory from an IO agent are always consistent with the caches. The availably of hardware-managed coherence greatly simplifies software development of the operating system device drivers, especially when it comes to debugging—it’s tricky to debug cache coherence issues. Newer ARM Cortex™ A9 processors have introduced a Snoop Control Unit for use with Multicore designs. See http://www.arm.com/products/processors/cortex-a/cortex-a9.php for more details.

You should be aware that even though the system may support cache coherence, it is not sufficient to guarantee that writes issued by the processor can be snooped by the snoop logic. The write may be in a write buffer on the way to the cache or memory subsystem. It may still be important that the driver issues a memory barrier before the hardware is notified that it can read any data. The smp_wmb() in Linux performs this function; if the barrier is not needed on a target system it will degenerate to be a null operation. In the case of PCIe™ devices, device drivers usually perform a read from the device. This is a serialization event in the processor and ensures that all memory writes are visible to any snoop logic.

The Linux kernel has a number of APIs that should be used to manage memory that is coherent with the I/O subsystem. For an example of pci_alloc_consistent() refer to Chapter 8.

System Bus Interface

In previous generations of embedded systems the processors or system bus was externally exposed on the device, and the system bus was routed throughout the printed circuit board. However, with advances in integration the system bus is now an internal (to the SOC) interconnect. The processor bus within some Intel Atom devices is a derivative of the previously exposed Front Side Bus (FSB). The FSB was the bus that connected the CPU to the Memory Controller Hub (MCH) in older generations of Intel platforms. The memory controller hub feature set is now integrated into the SOC; as a result, the memory controller interface is exposed for direct attachment to the memory devices. On the Intel Atom-based SOC, a bus similar to the FSB logically remains within the SOC.

Memory Technology

At the time of writing the memory technology attached to the processor is usually Double Data Rate Synchronous Dynamic Random Access Memory (DDR3 is common on desktop platforms, while DDR2 is still prevalent on embedded systems). The interface for DDR is defined by the JEDEC organization (reference: http://www.jedec.org). Both the density and bandwidth of the dynamic memory technology have increased considerably over time; however, the latencies associated with memory access have not declined at nearly the same rate. The memory controller and memory devices are highly pipelined. Attributes of the memory hierarchy described above are all structured to try to ensure that the application does not directly experience the latency of a DRAM transaction. To that end, in addition to the cache structure, some platforms have pre-fetchers. A pre-fetcher is a piece of logic that attempts to predict future memory access that will be made based on the history of the application. It then issues a speculative read transaction to the memory controller to bring a likely entry closer to the processor before it is actually needed.

The memory controller can operate in burst and single-transaction mode. Burst mode transactions are significantly more efficient than the single transaction. This burst mode transaction matches well to the cache line fill/write-back behavior of the caches.

More details on the memory interfaces are provided in Chapter 2.

Microarchitecture

The Intel Atom processor microarchitecture consists of the set of components on the processor that enables it to implement and provide support for the IA-32 and Intel 64 ISA. Embedded software developers need a basic understanding of the microarchitecture if they are engaged in low-level assembly language programming, such as developers working on device drivers or performance-critical portions of an application. Embedded software developers focused on utmost performance must also understand the microarchitecture and its implications for high-performing assembly or machine language. The ability to inspect assembly language code or a disassembly of an application and to understand the difference between high-performing and low-performing code sequences when executing on the Intel Atom processor is critical for product success. This section provides the basic understanding of the microarchitecture required to do so.

Figure 5.12 is a high-level depiction of the Intel Atom processor microarchitecture. At a first level, the microarchitecture of the initial Intel Atom processor is classified as an in-order, superscalar pipeline. The term in-order means that the machine instructions execute in the same order that they appear in the application. The term superscalar means that more than one instruction can execute at the same time. The Intel Atom processor is classified as two-wide superscalar since it has the ability to execute and retire two instructions in the same clock cycle. Modern processors are pipelined, which allows multiple instructions to be in different stages of processing at the same time.

FIGURE 5.12 Intel Atom Processor Microarchitecture.

The integer pipeline for the Intel Atom processor is detailed in Figure 5.13. The pipeline is divided into six phases of instruction processing:

The integer pipeline consists of 16 stages and the floating-point pipeline consists of 19 stages. In normal pipeline operation each stage takes one cycle to execute. The number of stages for each phase is detailed in Table 5.15. Note that each phase is pipelined; for example, it is possible for three instructions to be in the different stages of the instruction fetch phase (IF1, IF2, and IF3) at the same time.

FIGURE 5.13 Integer Pipeline.

Table 5.15. Intel Atom Processor Pipeline

For the integer pipeline, the instruction fetch phase is three stages and the instruction decode is three stages. Instruction issue consists of three stages and data access consists of three stages. Instruction execution consists of one stage and write back consists of three stages. For floating-point instructions, the instruction fetch consists of three stages and instruction decode consists of three stages. Instruction issue consists of three stages and data access consists of three stages. Instruction execution consists of seven stages and write back consists of one stage. There are many exceptions when an instruction can take longer to execute than 16 or 19 cycles; examples include division operations and instructions that decode in the microcode sequencer.

As mentioned previously, the Intel Atom processor microarchitecture is in-order. You can understand what in-order means by comparing it to an out-of-order microarchitecture such as the Intel Core i7 processor. Consider the sequence of instructions listed in Figure 5.14, which has the following program-specified dependencies:

An in-order processor would execute the instructions in the order the instructions are listed. In an in-order superscalar processor such as the Intel Atom processor with two execution pipelines, instruction 2 would attempt to execute at the same time as instruction 1, but due to the dependency, the pipeline would stall until the result of instruction 1 were ready. Instruction 3 would not start executing until instruction 2 had the result from instruction 1. Instruction 4 could start execution as soon as instruction 2 finished; however, instruction 5 would be stalled until instruction 4 had a result ready.

FIGURE 5.14 In-Order Execution.

An out-of-order processor allows independent instructions to execute out of order as long as the instruction’s dependencies have been fulfilled. On an out-of-order processor with sufficient execution resources the instruction schedule is more efficient. Instruction 4 can execute at the same time as instruction 1. Instruction 5 can execute at the same time as instruction 2. Instruction 6 can execute at the same time as instruction 3. The results are still written in program order; however, superscalar execution enables more efficient usage of process resources.

This is a fundamental difference between the Intel Atom processor and other modern out-of-order processors. One method of addressing this disadvantage is to use a compiler that schedules for the Intel Atom processor. If the compiler laid out the instructions in the order specified in Figure 5.15, many of the same benefits of out-of-order execution would result when executing on the Intel Atom processor.

FIGURE 5.15 Instruction Schedule for In-Order Execution.

Front End

The components that comprise what is termed the front end of the processor are charged with finding and placing the instructions into the execution units. The components that comprise the front end of the processor and their functionality are the following:

The front end of the microarchitecture performs the instruction fetch and instruction issue phases of the pipeline. The first action in placing an instruction into the pipeline is to obtain the address of the instruction. This address comes from one of two places. If the current instruction that is just entering the pipeline is not a branch instruction, the next instruction is equal to the address of the current instruction plus the size of the current instruction. If the current instruction is a branch, the next instruction is determined by the branch prediction unit that caches previously seen branches and provides a prediction as to the direction and target of the branch. The instruction TLB translates the virtual address used by the program into the physical address where the instruction is actually stored. The instruction cache is used to keep recently executed instructions closer to the processor if those particular instructions are executed again. The instruction cache is 24 kB.

Memory Execution Cluster

The memory execution cluster provides the functionality for generating addresses and accessing data. Components of the memory execution cluster and their functionality include the following:

In the optimum case for a data access, the data are resident in the L1 cache; however, if the data are not resident there, an L2 cache request is made. One architectural optimization is the inclusion of a data prefetcher that analyzes historical access patterns and attempts to fetch future data in advance of reference.

Common to many architectures, the memory subsystem supports store forwarding, which takes a result of a previous store and forwards the value internally for use in a proceeding load operation. This forwarding eliminates potential pipeline stalls because the load instruction does not need to wait until the stored value is committed to the cache. A special case of forwarding results from operations that affect the flags and instructions dependent upon them. Branch operations have an implicit one-cycle penalty. All other instructions have a two-cycle bubble.

One common occurrence in instruction sequences is the computation of an address for use in subsequent instructions. For example, Figure 5.16 shows an instruction sequence where the second instruction depends on an address calculation from the previous instruction. This dependency would cause a three-cycle stall because the second instruction cannot execute until the result of the first instruction is known. Figure 5.16 shows the impact on the pipeline as there is a bubble between the AG and EX pipeline stages.

FIGURE 5.16 Address Generation Stall.

Bus Cluster

The bus cluster is the connection from the processor to the memory subsystem and contains a 512-kB, eight-way L2 cache. The bus cluster also contains the Bus Interface Unit (BIU) and Advanced Programmable Interrupt Controller (APIC).