The structure of the Yocto Project makes more sense if you look at the background first. Its roots are in OpenEmbedded, http://openembedded.org/ which, in turn, grew out of a number of projects to port Linux to various hand-held computers, including the Sharp Zaurus and Compaq iPaq. OpenEmbedded came to life in 2003 as the build system for those hand-held computers but quickly expanded to encompass other embedded boards. It was developed and continues to be developed by an enthusiastic community of programmers.

The OpenEmbedded project set out to create a set of binary packages using the compact .ipk format, which could then be combined in various ways to create a target system and be installed on the target at runtime. It did this by creating recipes for each piece of software and using BitBake as the task scheduler. It was, and is, very flexible. By supplying the right metadata, you can create an entire Linux distribution to your own specification. One that is fairly well known is The Ångström Distribution, http://www.angstrom-distribution.org, but there are many others.

At some time in 2005 Richard Purdie, then a developer at OpenedHand, created a fork of OpenEmbedded which had a more conservative choice of packages and created releases that were stable over a period of time. He named it Poky, after the Japanese snack (if you are worried about these things, Poky is pronounced to rhyme with hockey). Although Poky was a fork, OpenEmbedded and Poky continued to run alongside each other, sharing updates and keeping the architectures more or less in step. Intel brought out OpenedHand in 2008 and they transferred Poky Linux to the Linux Foundation in 2010 when they formed the Yocto Project.

Since 2010, the common components of OpenEmbedded and Poky have been combined into a separate project known as OpenEmbedded core, or just oe-core.

Therefore, the Yocto Project collects together several components, the most important of which are the following:

Strictly speaking, the Yocto Project is an umbrella for these sub-projects. It uses OpenEmbedded as its build system, and Poky as its default configuration and reference environment. However, people often use the term "the Yocto Project" to refer to the build system alone. I feel that it is too late for me to turn this tide, so for brevity I will do the same. I apologise in advance to the developers of OpenEmbedded.

The Yocto Project provides a stable base which can be used as it is or which can be extended using meta layers, which I will discuss later in this chapter. Many SoC vendors provide board support packages for their devices in this way. Meta layers can also be used to create extended, or just different, build systems. Some are open source, such as the Angstrom Project, others are commercial, such as MontaVista Carrier Grade Edition, Mentor Embedded Linux, and Wind River Linux. The Yocto Project has a branding and compatibility testing scheme to ensure that there is interoperability between components. You will see statements like Yocto Project Compatible 1.7 on various web pages.

Consequently, you should think of the Yocto Project as the foundation of a whole sector of embedded Linux, as well as being a complete build system in its own right. You may be wondering about the name, yocto. A yocto is the SI prefix for 10-24, in the same way that micro is 10-6. Why name the project yocto? It was partly to indicate that it could build very small Linux systems (although, to be fair, so can other build systems), but also, perhaps, to steal a march on the Ångström distribution which is based on OpenEmbedded. An Ångström is 10-10. That's huge, compared to a yocto!

Usually, there is a release of the Yocto Project every six months, in April and October. They are principally known by the code name, but it is useful to know the version numbers of the Yocto Project and Poky as well. Here is a table of the four most recent releases at the time of writing:

The stable releases are supported with security and critical bug fixes for the current release cycle and the next cycle, that is approximately twelve months after release. No toolchain or kernel version changes are allowed for these updates. As with Buildroot, if you want continued support, you can update to the next stable release or you can backport changes to your version. You also also have the option of commercial support for periods of several years with the Yocto Project from operating system vendors such as Mentor Graphics, Wind River, and many others.

To get a copy of the Yocto Project, you can either clone the repository, choosing the code name as the branch which is fido in this case:

$ git clone -b fido git://git.yoctoproject.org/poky.git

You can also download the archive from http://downloads.yoctoproject.org/releases/yocto/yocto-1.8/poky-fido-13.0.0.tar.bz2.

In the first case, you will find everything in the poky directory, in the second case, poky-fido-13.0.0/.

In addition, you should read the section titled System Requirements from the Yocto Project Reference Manual (http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#detailed-supported-distros) and, in particular, you should make sure that the packages listed there are installed on your host computer.

As with Buildroot, let's begin with a build for the ARM QEMU emulator. Begin by sourcing a script to set up the environment:

$ cd poky
$ source oe-init-build-env

That creates a working directory for you named build and makes it the current directory. All of the configuration, intermediate, and deployable files will be put in this directory. You must source this script each time you want to work on this project.

You can choose a different working directory by adding it as a parameter to oe-init-build-env, for example:

$ source oe-init-build-env build-qemuarm

That will put you into the build-qemuarm directory. You can then have several projects on the go at the same time: you choose which one you want to work with through the parameter to oe-init-build-env.

Initially, the build directory contains only one subdirectory named conf, which contains the configuration files for this project:

For now, we just need to set the MACHINE variable in local.conf to qemuarm by removing the comment character at the start of this line:

MACHINE ?= "qemuarm"

To actually perform the build, you need to run bitbake, telling it which root filesystem image you want to create. Some common images are as follows:

By giving BitBake the final target, it will work backwards and build all the dependencies first, beginning with the toolchain. For now, we just want to create a minimal image to see whether or not it works:

$ bitbake core-image-minimal

The build is likely to take some time, maybe more than an hour. When it is complete, you will find several new directories in the build directory including build/downloads, which contains all the source downloaded for the build, and build/tmp which contains most of the build artifacts. You should see the following in tmp:

When you build a QEMU target, an internal version of QEMU is generated, which removes the need to install the QEMU package for your distribution and thus avoids version dependencies. There is a wrapper script named runqemu for this internal QEMU.

To run the QEMU emulation, make sure that you have sourced oe-init-build-env and then just type:

$ runqemu qemuarm

In this case, QEMU has been configured with a graphic console so that the boot messages and login prompt appear in the black framebuffer screen:

You can log on as root, without a password. You can close down QEMU by closing the framebuffer window. You can launch QEMU without the graphic window by adding nographic to the command line:

$ runqemu qemuarm nographic

In this case, you close QEMU using the key sequence Ctrl + A + X.

The runqemu script has many other options, type runqemu help for more information.

The metadata for the Yocto Project is structured into layers, by convention, each with a name beginning with meta. The core layers of the Yocto Project are as follows:

The list of layers in which BitBake searches for recipes is stored in <your build directory>/conf/bblayers.conf and, by default, includes all three layers mentioned in the preceding list.

By structuring the recipes and other configuration data in this way, it is very easy to extend the Yocto Project by adding new layers. Additional layers are available from SoC manufacturers, the Yocto Project itself, and a wide range of people wishing to add value to the Yocto Project and OpenEmbedded. There is a useful list of layers at http://layers.openembedded.org. Here are some examples:

Adding a layer is as simple as copying the meta directory into a suitable location, usually alongside the default meta layers, and adding it to bblayers.conf. Just make sure it is compatible with the version of the Yocto Project you are using.

To illustrate the way layers work, let's create a layer for our Nova board which we can use for the remainder of the chapter as we add features. Each meta layer has to have at least one configuration file, conf/layer.conf, and should also have a README file and a license. There is a handy helper script that does the basics for us:

$ cd poky
$ scripts/yocto-layer create nova

The script asks for a priority, and if you want to create sample recipes. In the example here, I just accepted the defaults:

Please enter the layer priority you'd like to use for the layer: [default: 6]
Would you like to have an example recipe created? (y/n) [default: n]
Would you like to have an example bbappend file created? (y/n) [default: n]
New layer created in meta-nova.
Don't forget to add it to your BBLAYERS (for details see meta-novaREADME).

That will create a layer named meta-nova with a conf/layer.conf, an outline README and a MIT license in COPYING.MIT. The layer.conf file looks like this:

# We have a conf and classes directory, add to BBPATH
BBPATH .= ":${LAYERDIR}"

# We have recipes-* directories, add to BBFILES
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb 
${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "nova"
BBFILE_PATTERN_nova = "^${LAYERDIR}/"
BBFILE_PRIORITY_nova = "6"

It adds itself to BBPATH and the recipes it contains to BBFILES. From looking at the code, you can see that the recipes are found in the directories with names beginning recipes- and have file names ending in .bb (for normal BitBake recipes), or .bbappend (for recipes that extend existing normal recipes by adding and overriding instructions). This layer has the name nova which is added to the list of layers in BBFILE_COLLECTIONS and it has a priority of 6. The layer priority is used if the same recipe appears in several layers: the one in the layer with the highest priority wins.

Since you are about to build a new configuration, it is best to begin by creating a new build directory named build-nova:

$ cd ~/poky
$ . oe-init-build-env build-nova

Now you need to add this layer to your build configuration, conf/bblayers.conf:

LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " 
  /home/chris/poky/meta 
  /home/chris/poky/meta-yocto 
  /home/chris/poky/meta-yocto-bsp 
  /home/chris/poky/meta-nova 
  "
BBLAYERS_NON_REMOVABLE ?= " 
  /home/chris/poky/meta 
  /home/chris/poky/meta-yocto "

You can confirm that it is set up correctly by using another helper script:

$ bitbake-layers show-layers
layer                 path                     priority
==========================================================
meta              /home/chris/poky/meta            5
meta-yocto        /home/chris/poky/meta-yocto      5
meta-yocto-bsp    /home/chris/poky/meta-yocto-bsp  5
meta-nova         /home/chris/poky/meta-nova       6

There you can see the new layer. It has priority 6 which means that we could override recipes in the other layers, which have a lower priority.

At this point it would be a good idea to run a build, using this empty layer. The final target will be the Nova board but, for now, build for a BeagelBone Black by removing the comment before MACHINE ?= "beaglebone" in conf/local.conf. Then, build a small image using bitbake core-image-minimal as before.

As well as recipes, layers may contain BitBake classes, configuration files for machines, distributions, and more. I will look at recipes next and show you how to create a customized image and how to create a package.

$ bitbake -c listtasks core-image-minimal

$ bitbake -c fetch busybox

$ bitbake -c fetchall core-image-minimal

meta-nova/recipes-local/helloworld
├── files
│   └── helloworld.c
└── helloworld_1.0.bb

DESCRIPTION = "A friendly program that prints Hello World!"
PRIORITY = "optional"
SECTION = "examples"

LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6"

SRC_URI = "file://helloworld.c"
S = "${WORKDIR}"

do_compile() {
  ${CC} ${CFLAGS} -o helloworld helloworld.c
}

do_install() {
  install -d ${D}${bindir}
  install -m 0755 helloworld ${D}${bindir}
}

LICENSE_FLAGS = "commercial"

LICENSE_FLAGS_WHITELIST = "commercial"

$ bitbake  helloworld

IMAGE_INSTALL_append = " helloworld"

$ bitbake core-image-minimal

You may often want to add a package to an image during development or tweak it in other ways. As shown previously, you can simply append to the list of packages to be installed by adding a statement like this:

IMAGE_INSTALL_append = " strace helloworld"

It should be no surprise that you can also do the opposite: you can remove a package using this syntax:

IMAGE_INSTALL_remove = "someapp"

You can make more sweeping changes via EXTRA_IMAGE_FEATURES. There are too many to list here, I recommend you look at the Image Features section of the Yocto Project Reference Manual and the code in meta/classes/core-image.bbclass. Here is a short list which should give you an idea of the features you can enable:

The problem with making changes to local.conf is that they are, well, local. If you want to create an image that is to be shared with other developers, or to be loaded onto a production system, then you should put the changes into an image recipe.

An image recipe contains instructions about how to create the image files for a target, including the bootloader, the kernel, and the root filesystem images. You can get a list of the images that are available by using this command:

$ ls meta*/recipes*/images/*.bb

The recipe for core-image-minimal is in meta/recipes-core/images/core-image-minimal.bb.

A simple approach is to take an existing image recipe and modify it using statements similar to those you used in local.conf.

For example, imagine that you want an image that is the same as core-image-minimal but includes your helloworld program and the strace utility. You can do that with a two-line recipe file which includes (using the require keyword) the base image and adds the packages you want. It is conventional to put the image in a directory named images, so add the recipe nova-image.bb with this content in meta-nova/recipes-local/images:

require recipes-core/images/core-image-minimal.bb
IMAGE_INSTALL += "helloworld strace"

Now you can remove the IMAGE_INSTALL_append line from your local.conf and build it using:

$ bitbake nova-image

If you want to go further and take total control of the contents of the root filesystem, you can start from scratch with an empty IMAGE_INSTALL variable and populate it like this:

SUMMARY = "A small image with helloworld and strace packages" IMAGE_INSTALL = "packagegroup-core-boot helloworld strace"
IMAGE_LINGUAS = " "
LICENSE = "MIT"
IMAGE_ROOTFS_SIZE ?= "8192"
inherit core-image

IMAGE_LINGUAS contains a list of glibc locales to be installed in the target image. They can take up a lot of space so, in this case, we set the list to be empty, which is fine so long as we do not need locale-dependent library functions. IMAGE_ROOTFS_SIZE is the size of the resulting disk image, in KiB. Most of the work is done by the core-image class which we inherit at the end.

It is very useful to be able to create a standalone toolchain that other developers can install, avoiding the need for everyone in the team to have a full installation of the Yocto Project. Ideally, you want the toolchain to include development libraries and header files for all the libraries installed on the target. You can do that for any image using the populate_sdk task, as shown here:

$ bitbake nova-image -c populate_sdk

The result is a self-installing shell script in tmp/deploy/sdk named:

poky-<c_library>-<host_machine>-<target_image><target_machine>-toolchain-<version>.sh

Here is an example:

poky-glibc-x86_64-nova-image-cortexa8hf-vfp-neon-toolchain-1.8.1.sh

Note that, by default, the toolchain does not include static libraries. You can enable them individually by adding lines like this to your local.conf or the image recipe:

TOOLCHAIN_TARGET_TASK_append = " glibc-staticdev"

You can also enable them globally as shown:

SDKIMAGE_FEATURES_append = " staticdev-pkgs"

If you only want a basic toolchain with just C and C++ cross compilers, the C library and header files, you can instead run:

$ bitbake meta-toolchain

To install the SDK, just run the shell script. The default install directory is /opt/poky, but the install script allows you to change that:

$ tmp/deploy/sdk/poky-glibc-x86_64-nova-image-cortexa8hf-vfp-neon-toolchain-1.8.1.sh

Enter target directory for SDK (default: /opt/poky/1.8.1):

You are about to install the SDK to "/opt/poky/1.8.1". Proceed[Y/n]?

[sudo] password for chris:

Extracting SDK...done

Setting it up...done

SDK has been successfully set up and is ready to be used.

To make use of the toolchain, first source the environment set up script:

. /opt/poky/1.8.1/environment-setup-cortexa8hf-vfp-neon-poky-linux-gnueabi

Toolchains generated in this way are not configured with a valid sysroot:

$ arm-poky-linux-gnueabi-gcc -print-sysroot

/not/exist

Consequently, if you try to cross compile as I have shown in previous chapters, it will fail like this:

$ arm-poky-linux-gnueabi-gcc helloworld.c -o helloworld

helloworld.c:1:19: fatal error: stdio.h: No such file or directory

#include <stdio.h>

                   ^

compilation terminated.

This is because the compiler has been configured to be generic to a wide range of ARM processors, and the fine tuning is done when you launch it using the right set of gcc flags. So long as you use $CC to compile, everything should work fine:

$ $CC helloworld.c -o helloworld

The Yocto Project insists that each package has a license. A copy of the license is in tmp/deploy/licenses/[packagenam.e] for each package, as it is built. In addition, a summary of the packages and licenses used in an image are in the <image name>-<machine name>-<date stamp> directory. This is shown here:

$ ls tmp/deploy/licenses/nova-image-beaglebone-20151104150124
license.manifest  package.manifest

The first file lists the licenses used by each package, the second lists the package names only.