© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022
S. Banik, V. ZimmerFirmware Developmenthttps://doi.org/10.1007/978-1-4842-7974-8_2

2. Tools

Subrata Banik1   and Vincent Zimmer2
(1)
Bangalore, Karnataka, India
(2)
Issaquah, WA, USA
 

“If the only tool you have is a hammer, it’s hard to eat spaghetti.”

—David Allen

Looking back at history, tools have been the impetus for the evolution of the human race. Starting from the Stone Age through the Iron Age and to the modern computing age, tools have eased the work that humans have to do. Having the right tools for developing a boot firmware product is absolutely necessary to ease the development effort as well, provide a flexible interface for configuration, and offer seamless upgradability. This chapter focuses on the details of the various types of tools that a system developer should be equipped with for creating their own boot firmware.

The system firmware development journey includes various tools from its creation until deployment, as shown in Figure 2-1. This book is committed to providing detailed knowledge about various tooling requirements for each phase to prepare developers to work with the prerequisite software development kits (SDKs).

Typically, to get started with firmware development, a developer has to be equipped with these tools:

  • Integrated development environment: An integrated development environment (IDE) is software that helps developers ease their development process. It typically is a source code editor that provides some basic functionality such as the ability to create new files, search files, replace a string in a file, and more. Based on developers’ needs, there are various types of IDEs available, from basic ones such as the vim editor to advanced IDEs such as Eclipse, for firmware development.

A system firmware development model includes source code, I D E, infrastructure tools, building firmware images, debugging, stitching, configuration, and flashing tools.

Figure 2-1

System firmware development model

  • Infrastructure tools: Chapter 3 provides ample details about the usage of infrastructure tools in system firmware development. Hence, this chapter will refrain from discussing the topic.

  • Debugging tools: Refer to Chapter 4 for a detailed overview of the debug methodology and debug tools usage on cross-architecture platforms. That chapter highlights the different types of tools used in system firmware development.

  • Build tools (compiler and stitching tools): Build tools are the first step toward creating a firmware image for embedded systems. This process involves taking source code as input and generating binary files as output. The next step combines all the binaries and produces the final firmware image. This process is highly dependent on the chip architecture, firmware architecture, and technology being used to create the system firmware.

  • Configuration tools: Configurability is a fundamental right for system integrators and validation engineers working on an embedded system. Relying on the development engineers to alter the source code to allow configuration and re-generate firmware images is not a scaled solution. This approach is a bottleneck for innovation. For example, it limits the scope for performing boundary testing with a wide range of data inputs. Even not a sustained model when the goal is to enable more ODM/OEM platforms with reduced development cost. Configuration tools allow developers to generate firmware images based on SoC and board-level configuration changes without rebuilding the system firmware image.

  • Flashing tools: The most important step to bring an embedded system to life is programming the flash parts. The purpose of flashing tools is to ensure the firmware image is correctly burned into the embedded system boot devices. The flashing process and mechanisms vary a lot, based on the underlying hardware design, target operating system, firmware architecture, technology being used, etc. A flashing tool might have a different interface and mechanism to consider based on the product lifecycle. The tooling process is different on the development side, factory side, and end-user side. Another consideration is that some firmware updates are over the air (OTA), meaning accomplished via a network.

The purpose of this chapter is to limit the discussion to the three major tooling needs (build, configuration, and flashing tools) while creating the system firmware and focus on the tools architecture across different system firmwares.

Build Tools

The most effective way to understand a system firmware architecture is by understanding its building blocks. Different firmware systems have much in common: they mostly use C, with some assembly, for example. The differentiating factors among these different system firmware are the underlying build tools.

This section provides a detailed overview of the build tools and processes of two popular system firmware products for embedded systems: EDKII and coreboot. Typically, at a high level, all system firmware packages consist of the same structures, as specified in Figure 2-2.
The BaseTools or Util directory consists of build tool binaries, or source code that needs to be compiled, prior to starting to build the package. The package does not have a dependency on the build environment. The package might consist of two types of files.

  • Source code files: Files get compiled to generate object files or binaries that need further processing.

  • Flash layout files: Files are processed to combine various binaries to create the final firmware image.

The build process includes three main phases.
  1. 1.

    Set up the environment.

     
  2. 2.

    Build the package components.

     
  3. 3.

    Package the components to create the final firmware image.

     

A framework of a high-level system includes building tools that lead to packaging and buildable code and flash layout files in the end.

Figure 2-2

High-level system firmware package structure

EDKII Build Tools and Process

EDKII is an open source firmware development project utilizing the UEFI and PI specification. EDKII is intended to improve the build experience compared to its predecessor, the EDK platform. EDKII adopts modern, feature-rich tools like Python for building, providing flexibility for developers while choosing correct toolchains using intermediate text files and relying on the C source code to generate tool binaries. Figure 2-3 describes the control flow of an EDKII build.

A framework of E D K I I building tools flows from scripts, tools AutoGen process with parsing tool to make the process of build binary to flash build binary by tool metadata, binary files, and P E binary which leads to flash image.

Figure 2-3

EDKII build tools and build process at a high level

Prior to discussing the build process and understanding how the underlying build tools are getting used, let’s first understand the prerequisites to initiate the build process.

Build Environment Setup

Several build environments must be prepared prior to development for EKDII. Some of these tools are dependent on host machines and host operating systems.

Operating Systems

Compiler Tool Chains

Microsoft Windows

Visual Studio C compiler and Windows Driver Kit (WDK)

Linux

Native GCC Installation 4.4 onward

Some other build tools may also be required, depending on the source package, as part of system firmware development needs, in addition to the previous list.

Additional Compilers

Details

Nasm

Nasm is required for the EDKII build if the source package has Intel-style assembly code.

iASL

This is needed to compile ACPI Source Language (ASL) and generate .aml files.

Python

EDKII has adopted Python for several build-related tools; hence, developers are expected to install Python to run Python-based tools from source. Examples: Build, GenFds, Trim, etc.

The EDKII prebuild phase ensures that the required basic environment variables are set, by executing script files named edksetup.bat or edksetup.sh, depending on the host system. Here is a list of environmental variables that the EDKII build process depends on:

Environment Variables

Description

WORKSPACE

The first variable to be set is WORKSPACE. This variable points to the root directory of an EDKII directory tree. More than one development tree can exist, and this environment variable is used to identify the current working directory tree.

PACKAGES_PATH

This variable points out all possible required repositories for building the target project. For example, for building MinPlatformPkg, the developer needs to point to these three repositories:

Edk2

Edk2Platforms/Platform/Intel

Edk2Platforms/Silicon/Intel

EDK_TOOLS_PATH

This points to the BaseTools directory belonging to the EDKII package. It contains binaries as mentioned in the EDK_TOOLS_BIN environment variable.

EDK_TOOLS_BIN

This is the path to point out build tool binaries. This depends on the operating system.

CONF_PATH

This is the variable to point out tool metadata files.

The CONF_PATH environment variable is used to point at the configuration files. The configuration file tools_def.txt is used to provide compiler path information, assembler information, linker information, etc., while another file, target.txt, is used to describe the build process.

ACTIVE_PLATFORM

Path to the .dsc file that represents the target embedded system

TARGET

Characteristics of system firmware like DEBUG or RELEASE

TARGET_ARCH

Underlying SoC or CPU architecture of target hardware

TOOL_CHAIN_TAG

To specify the compiler name example: GCC5 or VS2015x86

BUILD_RULE_CONF

Path to the build rule file (build_rule.txt) that specifies the build process, usage of build tools based input and output file types and specifies the type of cross-architecture platform.

At this stage, the EDKII prebuild is done with all prerequisites except the build tools, which is required to start the build process.

Build Binaries

The EDKII package consists of both the package source code and tools source code, separately. EDKII has introduced two sets of tools source code, as shown in the following tree structure:
|-- BaseTools
|   |-- Bin
|   |-- gcc
|   |-- Scripts
|   |-- Source
|   |   |-- C
|   |   |   |-- EfiRom
|   |   |   |-- GenFfs
|   |   |   |-- GenFv
|   |   |   |-- GenFw
|   |   |   |-- GenSec
|   |   |   |-- GenVtf
|   |   |   |-- Makefiles
|   |   |   |-- Split
|   |   |   |-- TianoCompress
|   |   |   |-- VfrCompile
|   |   |   `-- VolInfo
|   |   `-- Python
|   |       |-- AutoGen
|   |       |-- build
|   |       |-- GenFds
|   |       |-- Trim
|-- Conf
|-- CryptoPkg
|-- EdkSetupFsp.sh
|-- IntelFsp2Pkg
|-- MdeModulePkg
|-- MdePkg
|-- PcAtChipsetPkg
`-- UefiCpuPkg

Developers to build the BaseTools directory at least once to create build tool executables and update EDK_TOOLS_BIN before compiling the target platform package (.dsc) to generate the final Flash Descriptor (FD) file.

Depending on the host machine, perform the following command to generate executable Build Tools:

For Windows: nmake all

For Linux: make -C Edk2/BaseTools

All the required utilities will be copied into the path specified by the EDK_TOOLS_BIN environment variable. This is a manual process and expected to execute once, unless the developer cleans the entire workspace.

The build_rule.txt file is intended to provide detailed input and the expected outcome from each build tool taking part in the build process.

Let’s look at a few widely used build tools in detail and their usage prior to discussing the build process.

Utilities

Description

build.py

The main interface to complete the entire build process. This tool is written in Python. The build calls the AutoGen process; then it calls make process, and finally the ImageGen process to create the final FD binary.

AutoGen.py

Another key tool in the build process; this gets called by the build command. This tool is written in Python to work as a parsing tool to parse metadata files (INF, DSC, DEC, FDF) to autogenerate C source code files, makefiles per module, and the master makefile for the project with the help of other sets of libraries such as the following:

GenC Library: Used by AutoGen.py to create AutoGen.h and AutoGen.c after parsing the metadata files to resolve PCDs, GUIDs, etc.

GenMake Library: Used to create a makefile for each module and the top-level makefiles that can be further processed by nmake or gmake or cmake

GenFw.exe

Part of the C source code tool. Responsible for creating UEFI Firmware Image (.efi) files based on the module types listed in the INF files as part of each module.

GenSec.exe

Used to generate a valid EFI_SECTION type based on INF files. For example, as part of the .efi binary, each module also has several sections such as name sections, GUIDed sections, version sections, etc., based on associated fields in the INF file.

GenFfs.exe

Part of ImageGen process to create FFS files based on FDF to take part into firmware volume (FV). One can make use of the GenFds tool as well to create the FFS file.

GenFds.py

To process the files generated as part of the build binary and associated binary files that are listed in the FDF and DSC files to be part of the final Flash Device (FD) image.

In addition to the previous list, there are other useful utilities that are used for some specific purposes such as compression, generating an EFI option ROM image from the .efi file, creating a configuration file and/or patching a binary module, and creating an FD image.

Build Process

Figure 2-3 already provided the high-level build process flow for the EDKII platform, where a build command has triggered the build process. The entire build process can be divided into three major stages.

  • AutoGen process

  • Make process

  • ImageGen process

Every build process listed here involves certain build tools (described in earlier sections), driven by a fixed set of input files, that generate output files, which are used as input to the next build process. This process continues until a final firmware file (a so-called FD file) is generated.

AutoGen Process

This process was introduced in the EDKII build infrastructure. The major drawback that EDK had was each module needed to create its own makefile (similar to the C programming logic) to link the required libraries, GUID, etc., prior to compiling a module. It is difficult for a new developer who is transitioning into an EDK-based framework for firmware development.

EDKII has introduced this concept of AutoGen to parse the metadata files as part of the target package and/or individual module to generate some C source code files, the makefiles for each module, and even the master makefile for the target project.

Parsing Tools: This process has involved more than one tool and associated libraries. All tools that are part of this build process are written in Python. Refer to the build tools discussion for more details.

  • AutoGen.py: This tool is used to parse the metadata files with the help of another two libraries, GenC and GenMake, to create AutoGen.c, AutoGen.h, and other makefiles.

Input Files: At a high level, the parsing tools takes the following files as input:

File

Name

Description

.dsc

Platform Description

Each package has one DSC file to describe the build rules, libraries, and components (INF) being used.

A platform build process starts with this file.

.dec

Package Declaration

Declares the interfaces that are being used in that package.

.inf

Module Definition

Each module has one INF file to define the interface, source code, libraries, usage of GUID, etc.

.fdf

Flash Description File

File in each package that provides the flash layout and associated firmware volumes. Each component that is described in this file will take part in final binary creation.

Output Files: As mentioned earlier, the purpose of the parsing tool is to ease the development effort; hence, the output binaries that are part of this process are as follows:

  • Top-level makefile: This is the makefile for the entire package that resolves all libraries, GUIDs, and any global definitions as part of the package.

  • Makefiles per module: A C-based module can’t really compile without having a dedicated makefile. The parsing tool will generate a makefile for each module based on the INF file, after resolving the required dependencies using other packages.

  • Autogenerated C source code: The source code as mentioned in the INF files is mostly written in C, where it relies on C-based data structures. With the metadata-based implementation in EDKII, it needs a parser to create an autogenerated AutoGen.h or .AutoGen.C file based on the need to provide macro definitions and resolve external symbols for the next build process (which is the make process).

Make Process
The make process is not exceptional from any standard C-based compilation process to generate binary files that are further processed by special build tools to create EFI firmware image type files. Figure 2-4 describes the make process control flow:

  • Build binary: This process involves standard C-compilers, assemblers, static linkers, and dynamic linkers to generate PE/PE32+/COFF images from component or module types listed in the INF files. Additionally, the GenFw tool is used to generate EFI firmware image files.

A high-level makes process framework starts from source code, compiler, assembler with object files, static linker with static library files, dynamic linker with dynamic library files, and Gen Fw with E F I image files.

Figure 2-4

High-level make process structure

  • Input files: The make process takes two types of files as inputs. The source code belongs to the target and associated packages (C source, headers, ASM source and includes, ACPI and ASL files, etc.) and the autogenerated C source code headers and makefiles.

  • Output files: The purpose of the make process is to generate EFI file images that can be further used during the ImageGen process. But based on the input files, it can also generate .acpi, .aml, and .bin binaries. Figure 2-4 has provided a detailed control flow for generating EFI files based on input C source code. This process involves several intermediate steps, as shown here:
    1. 1.

      The standard C-based compiler has compiled the source code to generate .obj files.

       
    2. 2.

      .obj files have been given as input to the static linker to generate static library files (.lib), which further process into dynamic library files (.dll) with the help of a dynamic linker.

       
    3. 3.

      Finally, the GenFw tool as part of the $(make) phase converts the .dll into an EFI_SECTION type. The EFI file format is compatible with the PE32/PE32+/COFF format.

       
ImageGen Process
This phase is responsible for taking the EFI format files generated as part of the make process and parsing the package metadata files to verify whether all those EFI file types are intended to be part of the final flash image. This process can also take binary files as input, which is not part of the target DSC file or not the outcome of the build process while creating the final firmware image. Figure 2-5 describes the ImageGen process control flow.

A framework of High-level ImageGen starts by dividing the F D into 2 F V, and each F V is classified into 2 F F S. F F S has P E 32 by C O F F, which has dot E F I.

Figure 2-5

High-level ImageGen structure

  • Build binary: Once the EFI section files have been created in the previous step, they need to be placed within an FFS file. GenFfs is the build tool that generates FFS files after combining those EFI_SECTION type binaries with the FFS header. GenFds is used to construct the final firmware binary.

  • In addition, there might be a few modules and components that are part of the DSC files that are not meant to be part of a final firmware binary like Option ROM (OpROM). The EfiRom tool builds an option ROM image from an EFI file.

  • Input files: At this stage, all components or modules that are part of the target platform package DSC file are now converted into build binaries. Out of those build binaries, module types with UEFI applications and OpROM are excluded from being part of the final Flash Descriptor (FD) creation process. This final build process depends on the FDF file to create the flash layout.

  • Output files: The FDF file section provides an overview of generating one or more firmware volumes (FVs). The FV section describes how to combine FFS files to create FV files. Multiple FV files are combined to create the final firmware image as a flash device (FD). FD tokens provide the BaseAddress, Size, ErasePolarity, BlockSize, and NumBlocks fields.

BaseAddress

$(FLASH_BASE) | gSiPkgTokenSpaceGuid.PcdBiosAreaBaseAddress

Size

$(FLASH_SIZE) | gSiPkgTokenSpaceGuid.PcdBiosSize

ErasePolarity

1

BlockSize

$(FLASH_BLOCK_SIZE)

NumBlocks

$(FLASH_NUM_BLOCKS)

At the end of this phase, the final firmware binary at a size of $(FLASH_SIZE) is ready to use the EDKII framework to flash on the target embedded system.

This section provided an overview of the build system for an EDKII-based system firmware development. The next section will focus on the build tool and its creation process using another system firmware architecture, known as coreboot.

coreboot Build Tools and Process

coreboot was developed based on the open source firmware development principle that relies more on C-standard build tools and build environments to generate the final firmware binary. This process doesn’t require high-level build tools and is much simpler compared to the EDKII build process described in earlier sections. Figure 2-6 shows the coreboot high-level build process.

A framework flows from Kconfig, the AutoGen process the Parsing tool to make the process of building binary with E L F binary, and Image gen process the Flash build binary to get the flash image.

Figure 2-6

High-level coreboot build tools and build process

For a better understanding and to be able to compare the build process with EDKII, this section presents the coreboot build flow almost in the same manner as was presented in the previous section.

Build Environment Setup

This section provides the prerequisites to download the coreboot source code, which includes both the package source code and the source code for build tools. Additionally, it provides details about the compiler based on the host systems and environment variables that are expected to be set based on the target embedded system architecture.

The coreboot build process is based on GNU make; hence, developers need to have a Linux and UNIX-equivalent operating system on the host side with native GCC installed. GCC is the compiler for building coreboot (alternatively, one can also use LLVM/clang to build coreboot).

Some other build tools may also be required depending on the source package as part of the system firmware development needs in addition to the previous list:

Additional Compilers

Details

iASL

This compiles ACPI Source Language (ASL) and generate .aml files.

Flex

The GNU flex tool is used to parse the device tree field as it works as a lexical parser.

Bison

The GNU bison tool works as a grammar parser for device tree files.

The coreboot prebuild phase is to ensure that all associated coreboot project-related files are also getting synced from their respective repositories prior to building the target project or mainboard. These associated projects are maintained separately as part of Git submodules and checked out automatically after fresh coreboot code sync using git submodule update --init --checkout. The reason for maintaining these submodules separately is because they might be additional firmware projects by themselves like vboot or might be some SoC-specific closed source binary blobs like FSP, CSE, or PSP, etc., used to generate the final firmware binary.

The prebuild phase also sets up the required basic environment variables.

Here is a list of environmental variables that the coreboot build process is dependent on:

Environment Variables

Description

FIRMWARE_ARCH

This variable points to the target hardware architecture for which these files are getting compiled. Values can be used as x86, arm, etc.

KCONFIG_CONFIG

This variable points to the input config file. Typically, this is referred to as $(DOTCONFIG).

KCONFIG_STRICT

The value assigned to this variable is build/util/kconfig/conf --oldconfig src/Kconfig.

Define to enable warning as errors.

KCONFIG_AUTOHEADER

This variable points to the path and filename of the auto.conf file.

The tool chain configuration file (toolchain.inc) is meant to provide paths for compiler, assembler, and linkers based on the target hardware architecture.

At this stage, the coreboot prebuild has been done with all prerequisites except the build tools, which are required to start the build process.

Build Binaries

The coreboot package consists of both source code and tools source code separately, as shown in the following tree structure:
|-- 3rdparty
|-- configs
|-- Documentation
|-- payloads
|-- src
|-- util
|   |-- abuild
|   |-- acpi
|   |-- amdfwtool
|   |-- amdtools
|   |-- apcb
|   |-- cbfstool
|   |-- cbmem
|   |-- crossgcc
|   |-- futility
|   |-- ifdtool
|   |-- kconfig
|   |-- sconfig

To start the final firmware binary generation process based on the target hardware, coreboot needs to build the cross-compiler, such as x86, arm, risc v, etc. For coreboot, the entire build process is designed based on GNU make; hence, running a make help command would list the possible tool chain options.

To build the coreboot for all supported architectures, developers can use the following command:
make crossgcc CPUS=n [n = number of CPU cores to use.]

Additionally, developers need to build other tools as per package source support or build flow needs.

Let’s understand a few widely used build tools/utilities in detail and their usage prior to discussing the build process.

Kconfig
Kconfig is a widely used tool in Linux kernel projects to allow configuration mechanisms by selecting the required modules based on developer input. Similar to a coreboot project, Kconfig is used to allow developers to select the SoC feature or platform feature to create an autogenerated file that can be given to the make process for further processing. The Kconfig utility in coreboot gets built from the source code in util/kconfig. The Kconfig language is designed to describe a series of menu entries, and each Kconfig line starts with keywords like config, menuconfig, choice/endchoice, menu/endmenu, if/endif, etc. All Kconfig symbols in the coreboot project are referred to with the CONFIG_ prefix. Because of this advantage, the Kconfig language can be used easily to enable or disable any feature at build time. While building the coreboot project, developers are mostly familiar with the following modes to use Kconfig:

  • config: Text mode configuration; asks developer about each configuration option

  • menuconfig: Menu-driven configuration tool (added from Linux 2.5.45) still in Text mode

Output: After parsing all the source files, the Kconfig tool generates a HEADER file with a list of values inside build/auto.conf, which will be further used by the source code and makefiles of the project.

Sconfig
The coreboot device tree is one of the most important concepts in the coreboot project. The device tree is designed to represent the platform hardware device structure in the form of a device node that can be the bridge, the underlying bus architecture, and then finally the endpoint device. Sconfig is the tool that is used to compile the device tree files in a coreboot project to generate the static device configuration. The Sconfig utility is built from the source code inside util/sconfig. The Sconfig tool is internally using the Lex and Yacc tools to parse device tree files to create C source code, which can be further used by C-based compilers.

  • Lex: The GNU flex tool is used as a lexical analysis tool that can parse the .cb file to identify specific text strings such as chip, register, device, pci, on, off, domain, cpu_cluster, irq, etc.

  • Yacc: This is a grammar parser. The GNU bison tool is used to provide this functionality. The output of the flex tool is used as input for this tool to understand what action to take when a token is being identified. Examples: for Interrupts INT[A-D], for decimals [0-9.]+, for hexadecimal 0x[0-9 a-f A-F.]+, etc.

Output: The main goal of Sconfig is to convert the mainboard (and SoC if any) device tree files to create static C source and header files. It takes a few steps to generate desired output files.
  1. 1.

    util/sconfig/lex.yy.c_shipped is converted to lex.yy.c as a source file.

     
  2. 2.

    YACC takes lex.yy.c as input and creates the sconfig.tab.c and sconfig.tab.h files that contain the macros for the tokens.

     
  3. 3.

    The Sconfig utility is generated from the combined output from lex and YACC.

     
  4. 4.

    Finally, the Sconfig utility is used to generate static.c, static.h, and static_fw_config.h files.

     
cbfstool
cbfstool is a utility for managing the coreboot file system (CBFS) components during the final firmware image (ROM) generation process. The basic operations that cbfstool supports are add and remove modules into or from ROM images. For the platform using SPINOR as a boot device, one also needs to be aware that SPINOR memory is getting mapped into system runtime memory. Module relocation is not an option for early coreboot modules or .elf binaries in absence of physical memory at reset on x86 platforms. Thus, cbfstool needs to take care of two types of modules.

  • eXecute-In-Place (XIP): cbfs components marked as --xip will execute from the address where they’ve been mapped in SPINOR. Examples: verstage, FSP-M binary. To support XIP components with a higher SPINOR (32MB), cbfs has introduced concepts called extended window and extended window size that are mapped anywhere outside (4GB to 16MB) the system memory range.

  • Position-independent modules: Modules can be relocated anywhere in physical memory after it’s being available. The raw binaries like data or configuration files also belong to this category.

There could be more than one cbfs inside the coreboot final binary based on project design. For example, on Chrome OS projects there are three CBFSs per image: COREBOOT, FW_MAIN_A, and FW_MAIN_B.

cbfstool is built using the coreboot source code inside util/cbfstool. Inside the cbfstool directory there is source code to create other binaries as follows:

  • fmaptool: This is the Flashmap Descriptor language and compiler. It’s a tool that is able to parse the textual representation of an .fmd file and describe the layout of the flash chips that might contain more than one CBFS. This tool creates an intermediate file called fmap_config.h with the start and size of each component that would like to be part of fmap blobs. The final output of this tool is fmap.fmap.

  • rmodtool: This is another tool that is part of cbfstool, which is intended to parse and convert ELF type files to rmodules. For .elf files as part of the coreboot project, those that are supposed to get executed after the physical memory is initialized and not using SPI mapped memory, they are using rmodtool to generate position-independent code and data blocks.

In addition to the generic build tools, there are few important SoC vendor-specific tools that are used to generate the final firmware binary or perform key platform initialization or reset operation as follows:

Utility

Description

ifdtool

Intel Flash Descriptor (IFD), an open source tool that is used on the Intel platform to stitch boot-critical binaries (referred to as SI_ALL in .fmd) like Intel ME, GbE, EC, and Flash Descriptor along with coreboot BIOS region, referred as SI_BIOS in the .fmd file. Developers can use this tool to override the Flash Descriptor fields as well as part of the manufacturing flow.

apcb

AMD Platform Security Processor (PSP) Control Block Tools. This tool is capable of patching an existing APCB binary into the PSP blob. A part of this tool (apcb_edit.py) is allowed to patch the AMD PSP Customization Block (APCB). A binary has been integrated into the PSP to provide the SPD information and perform the required GPIO programming to select and load the right SPD.

amdfwtool

This tool is used to inject various images needed by the PSP to complete the reset flow. This tool takes image name, size, and intended location in the firmware structure. The output of this tool is amdfw.rom, which holds headers, pointers, and added firmware images.

amdcompress

A utility to generate a compressed BIOS image for AMD Family 17h. This compressed image is added into PSP’s amdfw.rom binary. The modern AMD system has the PSP, which is able to bring up DRAM prior to x86 reset; hence, the PSP decompresses (the PSP has support for zlib engine) the BIOS image into the DRAM and starts execution.

Build Process

Figure 2-6 has already provided the high-level build process flow for the coreboot platform where the GNU make command has initiated the build process. From opening up the terminal on the host system until the final firmware (.ROM) file generation process, the entire build process can be divided into three major stages.

  • AutoGen process

  • Make process

  • ImageGen process

Every build process listed here involves picking up the correct build tools (described in earlier sections) and having a fixed set of input files, and the target is to generate the output files that can be further worked as input for the next builder script. This process continues unless a final firmware file is generated.

AutoGen Process
This process involves parsing the Kconfig source language files (.kconfig) and package device tree files (.cb) for generating two types of files.

  • Autogenerated C header that holds the value of lists of Kconfig being used on the project package (CONFIG_*). The auto.conf header file is the outcome of parsing the Kconfig file by the Kconfig build tool.

  • Static C source and header files are being generated as part of this process by the Sconfig tool. These files are getting used further to provide the hardware configuration snapshot to the system firmware during boot.

File

Name

Description

*.CB

Device Tree Files

Each package should have at least one devicetree.cb file, which allows for the board-level configuration along with providing the snapshot of the CPU and PCI bus endpoint devices. At runtime this snapshot is being used to enable/disable the hardware interface or initialize a device.

Make Process

The top-level makefile for the project and all the package makefiles are used in the make process. The input for the make process includes autogenerated C source and headers from the previous stage; coreboot provides assembly and C source code per stage such as bootblock, romstage, postcar, ramstage, and SMM. The coreboot build process is flexible enough to make any boot stage optional. For example, if the system firmware design doesn’t need to have a dedicated ramstage as a stage for loading the payload, then coreboot can generate the final ROM image without compiling files for ramstage. These coreboot stages are independent enough in this build process to generate the ELF file after using the FIRMWARE_ARCH-specific compiler, assemblers, and linkers.

On the x86 platform where SPINOR is being used as a boot device and in absence of physical memory, it needs to ensure that the bootblock is able to patch at the reset-vector where coreboot is able to execute the bootblock instruction upon hitting CPU reset. In addition, there are a few stages (.ELF) that would like to load as a position-independent binary; hence, there is a need to use rmodtool to generate a special .rmod binary for stages that are typically getting executed after DRAM is initialized.

ACPI modules are also getting compiled using the iASL compiler and generated in this process. The fmap tool is used in this process to parse the FMD file to create a flash descriptor file with the names of possible CBFSs.

File

Name

Description

.fmd

Flashmap File

Each board package is equipped with one .fmd file to provide the description about flash layout. This .fmd file might consist of more than one region (SI_ME, SI_EC, etc.) in addition to the BIOS region, specified as SI_BIOS. Based on the project requirement, this FMD file might have more than one CBFS as well.

ImageGen Process

This phase is responsible for creating the final firmware image (ROM) after adding all those build binaries from the previous stage into the CBFS.

This process basically divided into two phases:

  • Create CBFSs: The coreboot build process relies on the cbfstool build to add the .elf and .bin binaries into sections into cbfs as per the following example where different coreboot stages and raw binary is getting injected to create the final ROM image:

printf "    CBFS       fallback/romstage "
CBFS       fallback/romstage
build/util/cbfstool/cbfstool build/coreboot.pre.tmp add-stage -f build/cbfs/fallback/romstage.elf -n fallback/romstage  -c none  -r COREBOOT -a 64 -S ".car.data" --xip
printf "    CBFS       fallback/ramstage "
CBFS       fallback/ramstage
build/util/cbfstool/cbfstool build/coreboot.pre.tmp add-stage -f build/cbfs/fallback/ramstage.elf -n fallback/ramstage  -c LZMA  -r COREBOOT
printf "    CBFS       fallback/dsdt.aml "
CBFS       fallback/dsdt.aml
build/util/cbfstool/cbfstool build/coreboot.pre.tmp add -f build/dsdt.aml -n fallback/dsdt.aml -t raw -c none  -r COREBOOT

  • Add SoC/CPU vendor-specific binary: This process involves using vendor-specific binaries and tools to inject into coreboot.rom to call it the final firmware binary. For example, on the Intel platform, ifdtool is used to inject a descriptor, ME and EC, whereas on the AMD platform, an additional step has to be performed to compress the bootblock (using amdcompress) and create the build binary (amdfw.rom using amdfwtool) into PSP; it can then be added directly into the coreboot image.

At the end of this phase, the final firmware binary (ROM) at a size of CONFIG_ROM_SIZE is ready using the coreboot open source firmware model to flash on the target embedded system.

Configuration Tools

The configuration tools are intended to allow changes in the final firmware image without going through the entire build process. The scope of configuration can vary between static and dynamic based on the system where this tool is running. For example, a tool can be running as part of the host system. Taking a final firmware binary (FD or ROM) as input to modify its default configuration value is known as static configuration. Or, if during the runtime execution on the target hardware a native configuration interface allows modification of the configuration database, this is called a dynamic configuration tool.

In this section, we will discuss a few widely used tools in the scope of different system firmware architectures like coreboot, EDKII, and Slimboot that are allowed to modify the configuration database.

Human Interface Infrastructure

Legacy system firmware was lagging in terms of allowing a unified approach to configure the underlying hardware with different hardware vendors providing their own configuration tools and access mechanisms. This made it harder for system integrators to design a robust interface for end users and various other users of system firmware.

On the UEFI platform, the Human Interface Infrastructure (HII) is used to provide a flexible and standard way to configure the target hardware. HII allows the platform configuration to access the hardware interface and store the data using the form browser. The form browser is like a web page that uses display and input devices for configuration to take place. Figure 2-7 shows the high-level operational model of HII.

HII is designed to create the platform configuration in the form of a data structure that needs localized text and a GUI to interface with the user. It needs six types of components to allow platform configuration using HII.

  • HID devices: Input devices are used for configuration in the form of localization. HII supports localization, a process that helps a product adapt to the local market. HII supports keyboard mapping to allow users to choose their own language as input.

A block diagram flows from H I I devices, Forms browser, display devices and N V storage, Driver, and H I I database, which again flows to forms browser.

Figure 2-7

High-level operational model of HII

  • Display devices: The output devices support the localization. HII supports Unicode characters, which allows it to support all possible languages to display as part of the form.

  • HII database: The HII database is created dynamically as the system boots. The UEFI driver is required to register a list of HII packages into the HI database. The package list provides different types of binary data. The data types could be font, string, image, keyboard layout, form, etc.

  • Driver: The UEFI driver provides the Config Routing Protocol as ExtractConfig, RouteConfig, and Callback to retrieve and save configuration information associated with HII forms.

  • NV Storage: The NV Storage is to store any data which remains persistent even after the system is resuming from the mechanical off state. The NVRAM is getting used as the NV storage. The HII form retrieves the configuration data from NV Storage and allows modification of these parameters from the available configuration list. The EFI variable services protocol is used to access the NV Storage.

  • Forms Browser: This is a GUI to represent the HII and allow users to configure the options. HII has its own standard architecture and language as IFR and VFR to present the browser with the help of a Unicode string.

    • IFR : Internal Forms Representation is the architectural binary encoding used to present the user interface pages. The Vfr compiler takes VFR files as input and generates output IFR files.

    • VFR: Visual Forms Representation is the source code language that is used by developers to design a form page.

The following table provides a conversion from VFR to IFR for developers’ understanding:

VFR

IFR

form formid = 1,

  title = STRING_TOKEN(

                STR_FORM1_TITLE

            ),

typedef struct _EFI_IFR_FORM {

EFI_IFR_OP_HEADER  Header;

UINT16                 FormId;

EFI_STRING_ID      FormTitle;

} EFI_IFR_FORM;

Figure 2-8 represents a UEFI browser setup page created using HII.

A screenshot of the E D K I I configuration setup browser indicates the B I O S information, vendor, project version, board, processor, and memory information.

Figure 2-8

EDKII setup browser using HII

YAML-Based Configuration

YAML-based configuration is part of Slim Bootloader (SBL) configuration process to provide a simple and flexible method to modify the board-specific parameter to support new boards. SBL provides configuration parameters that are getting used for platform initialization and are typically categorized into memory; SoC hardware interfaces like USB, PCIE, and GPIO; and OS boot options.

SBL configuration parameters are packed in a configuration binary blob. This binary blob is stitched with BIOS regions to apply the configuration at runtime (during Stage 1B phase of SBL).

The idea of the configuration binary block is to support multiple board configurations using a single system firmware image; hence, this blob contains configuration parameters for multiple different boards. The configuration binary block starts with a configuration blob header and is followed by the configuration parameters that are organized in configuration blocks. Each configuration block contains a block header followed by the parameter structure. Each configuration block is identified by a unique tag as PLATFORM_CFG_DATA, MEMORY_CFG_DATA, etc.

In SBL firmware architecture, the platform configuration relies on YAML files. Figure 2-9 shows the high-level view of the YAML-based configuration.

A flow diagram flows from Y A M L, tools divided into header and binary files. Header files copy into a source of S B L source code to the final flash image. The binary files c f g data stitch to final flash image.

Figure 2-9

High-level YAML-based configuration

YAML is a data serialized language that can be used for generating the configuration blobs while working with other modern programming languages. The idea here is to create all possible configuration options in SBL source code in the form of YAML syntax inside cfgdata. For example, a template of Debug Consent configure is shown here:
- DebugInterfaceEnable :
  name         : Enable or Disable processor debug features
  type         : Combo
  option       : $EN_DIS
  help         : >
                     Enable or Disable processor debug features; <b>0- Disable</b>; 1- Enable.
  length       : 0x01
  value        : 0x00

Users can open the CfgDataDef.yami file using the ConfigEditor GUI tool to allow the default value to be overridden.

All configuration YAML files will be processed by configuration tools like GenCfgData, CfgDataTool, and CfgDataStitch to generate configuration header files and binary blobs.

Finally, use the CfgDataStitch tool to patch the new configuration data file into the final firmware binary.

Firmware Configuration Interface

Traditionally, coreboot has limited dynamic configuration capabilities compared to other system firmware used on the embedded system. Because coreboot was designed with the principle of instant boot and a small footprint, it doesn’t provide much scope for a preboot configuration environment like a UEFI browser. This eventually results in source-based modifications to enable/disable certain hardware interfaces as per system firmware users’ needs.

The firmware configuration interface in coreboot is designed to overcome such limitations and allow users to configure the possible hardware interface at runtime. This interface will also help to maintain a single firmware image that can work seamlessly on different motherboards where the base schematics is the same but the I/Os might be different.

The coreboot Kconfig option CONFIG_FW_CONFIG can be used to enable this feature where the platform has decided to provide a bunch of configuration options as part of the devicetree.cb source code for runtime configuration.

The firmware configuration structure today is limited to a 64-bit value where the bitmask is used to determine the feature that needs to be configured at runtime. There are two possible implementations for enabling the firmware configuration interface in coreboot.

  • CBFS

  • A 64-bit raw value can be stored into CBFS with the name CONFIG_CBFS_PREFIX/fw_config. To enable this feature, coreboot needs to be built with the CONFIG_FW_CONFIG_CBFS option.

  • At runtime while the fw_config_probe() function is getting called, it will check the Kconfig option and load the CBFS filename to get this configuration value. cbfstool can be also used to override this binary blob at build time.

  • Embedded Controller

  • On the Chrome OS platform, the embedded controller interface will read and write the firmware configuration value using the CrOS Board Info (CBI) command. Mainboard users can select CONFIG_FW_CONFIG_CHROME_EC_CBI options to read the fw_config value from EC CBI.

  • The ectool command can also be used to configure this value using the ChromeOS environment.

  • Firmware Configuration Table

  • The firmware configuration table, which is part of the mainboard devicetree.cb file, starts with a special token as fw_config and terminates with end. The SCONFIG tool is used to parse this fw_config token to generate a static_fw_config.h file after understanding the grammar, where each field is defined by providing the field name, the start bit, and the end bit. Inside each field block, the option is used to provide the possible option name and associated value. This example configures the EMMC boot using fw_config:

field BOOT_DEVICE_EMMC 22
               option BOOT_EMMC_DISABLED 0
               option BOOT_EMMC_ENABLED 1
end

Binary Configuration Tool (BCT)/Config Editor

The hybrid firmware development model is where the open source firmware development model gets stitched with closed binary blobs like the Firmware Support Package (FSP) to create the final firmware binary for embedded systems. Previous sections discussed in detail the possible options to configure the open source boot firmware, and this section will provide mechanisms to allow changes in the configuration settings for FSP binaries. The Binary Configuration Tool (BCT) (almost deprecated and replaced with a newer utility named Error! Hyperlink reference not valid.Config Editor) was developed by Intel to allow configuration changes in static UPD configurations. Static UPDs are a kind of configuration parameter that isn’t really meant to change based on certain runtime decisions; hence, users can make use of BCT/Config Editor to open the FSP binary and modify the default UPD value as part of the SPI Flash image. This tool is not designed to manage the dynamic UPD configuration. See Figure 2-10.

A screenshot of the Binary configuration tool depicts the menu bar and has three panels, standard enabled with F S P T settings, F S P T settings, and P c d SerialloUartNumber and default value.

Figure 2-10

Configuring static FSP-UPD using BCT

Flashing Tools

In the system firmware development approach, the last but very significant tool is the flashing tool. Starting from the source code development all the way up to generating the final firmware image, the process will fulfill its purpose only if the embedded system is able to boot after flashing the final image on the targeted embedded system’s boot device. The flashing tools are highly specific to the target embedded system; they need to know the hardware interface that can be used to flash into the boot device from the local or remote host. At the high level, these flashing tools can be divided into two categories: hardware-based tools and software-based tools.

In this section, we will discuss a few popular flashing tools used on the target hardware for flashing the system firmware. The assumption being made here is that all these embedded systems are using SPINOR as the boot device.

Hardware-Based Tools

Hardware-based tools are used to flash the embedded systems in some specific scenarios like a device with an empty SPINOR as the very first boot during the manufacturing process, a corrupted SPINOR, or the device being unable to boot to the OS or pre-boot console to allow flashing using a software-based mechanism. This type of tool requires the host system to be connected to the target device where the host and the target device will connect using the hardware interface like USB or serial and where the host system is running the flashing software to write into the target device SPINOR.

SPINOR Programmer

Mostly in all the latest embedded platforms, the SPINOR is the default and soldered onto the board; hence, either the board design has an SF programming header where the SPINOR programmer can be attached to flash the firmware image or you need to make use of some custom clip solution to attach directly onto the SPINOR chip itself for flashing. Dediprog SF100 and SF600 are the widely used in system programing. Programmers can flash the firmware on the target system using a USB interface.

Servo

Servo is a debug board used for Chrome OS projects for multipurpose operations. One such operations is to allow the developer to flash the CPU and EC SPINOR using the USB interface. Over the generations, the Servo hardware specification has evolved from v2 to v4. Flashrom is the utility for updating the SPI Flash using Servo while running from the Host machine as well. The utility expects to mention the external SPI programmer support using the -p option followed by the name of the FTDI FT2232 device. For example:
sudo flashrom -V -p ft2232_spi:type=google-servo-v2 -w $IMAGE

This ensures the recovery of the bricked system by flashing the EC and system firmware using Servo.

Software-Based Tools

The software-based tools don’t have any prerequisites for the underlying hardware interface or any dependency over the host system. These kinds of tools run on the target device to allow read, write, or erase on the SPINOR using the pre-OS or OS environment. The following are a few popular software-based flashing utilities being used on cross-firmware projects.

Flashrom

An open source flashing utility is used across various operating systems and motherboards for detecting, reading, writing, erasing, and verifying the flash chips. The Flashrom tool can be used to flash coreboot/EFI images on the supported mainboard. Flashrom has support for various AICs like network, graphics, and storage.

It also supports a wide range of SPI programmers using USB interfaces like FTDI FT2232/FT4232H. The flashing mechanism is different from flashrom while using hardware-based tools like Servo to run as a user space application on the DUT itself.

Users can run the flashrom -p -w $IMAGE command to write the final firmware image into the SPINOR using the DUT OS command line. The flashrom source code is getting managed inside https://github.com/flashrom/flashrom, and users are expected to build the flashrom source code to generate the flashrom tool binary. While working inside the Chrome OS project repository, developers can run the following command to generate the flashrom tool:
cros_workon --host start flashrom; sudo emerge flashrom

UEFI Tools and Utility

Different vendors have created their own platform flashing tools using UEFI pre-boot services or native OS-based drivers to access the underlying SPI Flash device with the highest privilege.

AMI Firmware Update (AFU) is one popular tool (AfuEfix64.efi) used on the Aptio platform either from the pre-boot environment or from the OS layer to update the SPINOR. Many OEMs are using the uefiflash.efi binary to update the final firmware image from the EFI Shell environment as part of the shell automated script.

Summary

This chapter has provided detailed analysis of various system firmware development tools such as build tools, configuration tools, and flashing tools that a developer has to be equipped with prior to starting their own system firmware development. This chapter might also be useful for firmware developers to understand the underlying tools architecture and build process for popular boot firmware like EDKII and coreboot. This knowledge can be directly applied while creating your own system firmware using a new SoC/CPU architecture that is not yet supported by existing system firmware (EDKII or coreboot).

The book System Firmware: An Essential Guide to Open Source and Embedded Solutions has a case study about adding support for a new RISC-based CPU using EDKII.

We also discussed possible configuration options across different leading system firmware solutions so that developers can make the right decision when choosing the correct boot firmware for their target embedded system. Understanding flashing tools and the underlying hardware interface is useful to ensure guaranteed recovery even from the bricking state of the target device.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.117.227.194