Bootloaders

Like most embedded Linux devices, the Beagle boards do not have a UEFI/BIOS or battery-backed memory by default (a battery-backed real-time clock is added to a board in Chapter 8). Instead, it uses a combination of bootloaders. Bootloaders are typically small programs that perform the critical function of linking the specific hardware of your board to the Linux OS. Bootloaders perform the following:

• Initialize the controllers (memory, graphics, I/O)
• Prepare and allocate the system memory for the OS
• Locate the OS and provide the facility for loading it
• Load the OS and pass control to it

The Beagle boards use an open source Linux bootloader, called Das U-Boot (“The” Universal Bootloader). It is custom built for each Beagle board using detailed knowledge of the hardware description, which is provided in board-specific software patches.

The /boot directory contains the Linux kernel and the other files required to boot the board. For example, on the PocketBeagle (with two different kernel versions), you will see the directory listing, as shown here:

debian@beaglebone:/boot$ ls -l
-rw-r--r-- 1 root root   159786 Apr 22 07:17 config-4.14.35-ti-rt-r44
-rw-r--r-- 1 root root   163837 Apr 22 11:44 config-4.9.88-ti-rt-r111
drwxr-xr-x 6 root root     4096 Apr 25 01:14 dtbs
-rw-r--r-- 1 root root  4468991 Apr 24 01:01 initrd.img-4.14.35-ti-rt-r44
-rw-r--r-- 1 root root  6523044 Apr 25 01:34 initrd.img-4.9.88-ti-rt-r111
-rw-r--r-- 1 root root      492 Mar  5 13:11 SOC.sh
-rw-r--r-- 1 root root  3634267 Apr 22 07:17 System.map-4.14.35-ti-rt-r44
-rw-r--r-- 1 root root  3939313 Apr 22 11:44 System.map-4.9.88-ti-rt-r111
drwxr-xr-x 2 root root     4096 Mar  5 07:14 uboot
-rw-r--r-- 1 root root     1882 Apr 25 01:26 uEnv.txt
-rwxr-xr-x 1 root root 10379776 Apr 22 07:17 vmlinuz-4.14.35-ti-rt-r44
-rwxr-xr-x 1 root root 10070592 Apr 22 11:44 vmlinuz-4.9.88-ti-rt-r111 

These files have the following functions:

• The Linux kernel is in the compressed vmlinuz* files.
• The initial RAM disk (file system in memory) is in the initrd* files, which creates a temporary root file system (called an early user space) that is used to identify hardware and load the kernel modules that are required to boot the “real” file system.
• The uEnv.txt file sets the boot parameters for your board. You can edit this file to set custom boot properties (e.g., set up a flasher image as described in Chapter 2).
• The System.map* files provide a debug mapping from kernel-specific build codes to human-readable symbols so that any crash dump files can be analyzed. This file is not required for boot.
• The config* files list the options that were chosen when the kernel was built. It is human readable and not required for boot. You can type more /boot/config-4.xxx to see the build configuration.
• SOC.sh is a script that contains bootloader information that is required when the bootloader is to be upgraded. It is human readable and not required for boot.
• The /boot/dtbs/ directory contains subdirectories with the device tree binaries that are required to boot a board. Each subdirectory provides binaries for a specific kernel build.

Figure 3-1 illustrates the boot process on the Beagle boards, where each bootloader stage is loaded and invoked by the preceding stage bootloader. The primary program loader in the AM335x executes when the reset vector is invoked. It loads the first stage bootloader (aka the second stage program loader) file called MLO from the eMMC/SD card. This loader initializes clocks and memory and loads the second stage bootloader u-boot.img from the eMMC/SD card. The MLO (~92 KB) and u-boot.img (~420 KB) files exists in a raw area of the SD card that precedes the Linux ext4 partition.¹ For example, when you call lsblk, you see that the ext4 partition begins at sector 8,192 (4,194,304 bytes), as shown here:

debian@beaglebone:/boot$ sudo fdisk -l
Disk /dev/mmcblk0: 29.8 GiB, 32010928128 bytes, 62521344 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xaf2e67c6
Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk0p1 *     8192 62521343 62513152 29.8G 83 Linux 

which gives sufficient space for storing these files.

This file system design has the advantage of making accidental deletion of these critical files unlikely. Importantly, the u-boot.img and MLO files were built using a patched version of the standard U-Boot distribution that was built with full knowledge of the hardware description of the Beagle boards. The second-stage bootloader (U-Boot) performs additional initialization and then loads and passes control to the Linux kernel, which it finds in the /boot/ directory of the ext4 partition.

The output that follows is a typical boot sequence that was captured using the USB to UART TTL 3V3 serial cable that is introduced in Chapter 1. The cable was attached the BeagleBone header, as illustrated in Figure 1-11(a), and the data was captured at a baud rate of 115,200. The following is an extract of the console output as a BBB is booting. It displays important system information, such as memory mappings.

U-Boot SPL 2018.01-00002-ge9ff418fb8 (Feb 20 2018 - 20:14:57)
Trying to boot from MMC2
 
U-Boot 2018.01-00002-ge9ff418fb8 (Feb 20 2018 - 20:14:57 -0600),
Build: jenkins-github_Bootloader-Builder-38
CPU  : AM335X-GP rev 2.1
I2C:   ready
DRAM:  512 MiB
Reset Source: Power-on reset has occurred.
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
Using default environment
 
Board: BeagleBone Black
<ethaddr> not set. Validating first E-fuse MAC
BeagleBone Black:
BeagleBone: cape eeprom: i2c_probe: 0x54:
BeagleBone: cape eeprom: i2c_probe: 0x55:
BeagleBone: cape eeprom: i2c_probe: 0x56:
BeagleBone: cape eeprom: i2c_probe: 0x57:
Net:   eth0: MII MODE
cpsw, usb_ether
Press SPACE to abort autoboot in 2 seconds
board_name=[A335BNLT] …
board_rev=[00C0] …
Card did not respond to voltage select!
mmc_init: -95, time 13
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
Card did not respond to voltage select!
mmc_init: -95, time 13
switch to partitions #0, OK
mmc1(part 0) is current device
Scanning mmc 1:1…
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
switch to partitions #0, OK
mmc1(part 0) is current device
gpio: pin 54 (gpio 54) value is 1
Checking for: /uEnv.txt …
Checking for: /boot.scr …
Checking for: /boot/boot.scr …
Checking for: /boot/uEnv.txt …
gpio: pin 55 (gpio 55) value is 1
1879 bytes read in 22 ms (83 KiB/s)
Loaded environment from /boot/uEnv.txt
Checking if uname_r is set in /boot/uEnv.txt…
gpio: pin 56 (gpio 56) value is 1
Running uname_boot …
loading /boot/vmlinuz-4.9.82-ti-r102 …
9970640 bytes read in 655 ms (14.5 MiB/s)
uboot_overlays: [uboot_base_dtb=am335x-boneblack-uboot.dtb] …
uboot_overlays: Switching too: dtb=am335x-boneblack-uboot.dtb …
loading /boot/dtbs/4.9.82-ti-r102/am335x-boneblack-uboot.dtb …
61622 bytes read in 56 ms (1 MiB/s)
uboot_overlays: [fdt_buffer=0x60000] …
uboot_overlays: loading /lib/firmware/BB-BONE-eMMC1-01-00A0.dtbo …
1440 bytes read in 605 ms (2 KiB/s)
uboot_overlays: loading /lib/firmware/BB-HDMI-TDA998x-00A0.dtbo …
5127 bytes read in 80 ms (62.5 KiB/s)
uboot_overlays: loading /lib/firmware/BB-ADC-00A0.dtbo …
711 bytes read in 397 ms (1000 Bytes/s)
uboot_overlays: loading /lib/firmware/univ-bbb-EVA-00A0.dtbo …
62008 bytes read in 470 ms (127.9 KiB/s)
loading /boot/initrd.img-4.9.82-ti-r102 …
6510886 bytes read in 435 ms (14.3 MiB/s)
debug: [console=ttyO0,115200n8 bone_capemgr.uboot_capemgr_enabled=1
root=/dev/mmcblk1p1 ro rootfstype=ext4 rootwait coherent_pool=1M
net.ifnames=0 quiet] …
debug: [bootz 0x82000000 0x88080000:635926 88000000] …
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
   Loading Ramdisk to 8f9ca000, end 8ffff926 … OK
   reserving fdt memory region: addr=88000000 size=7e000
   Loading Device Tree to 8f949000, end 8f9c9fff … OK
 
Starting kernel …
[    0.001915] clocksource_probe: no matching clocksources found
[    1.459462] wkup_m3_ipc 44e11324.wkup_m3_ipc: could not get rproc handle
[    1.751983] omap_voltage_late_init: Voltage driver support not added
[    1.759505] PM: Cannot get wkup_m3_ipc handle
rootfs: clean, 82338/239520 files, 481317/957440 blocks
 
Debian GNU/Linux 9 beaglebone ttyS0
BeagleBoard.org Debian Image 2018-03-05
Support/FAQ: http://elinux.org/Beagleboard:BeagleBoneBlack_Debian
default username:password is [debian:temppwd]
beaglebone login: 

You can see that the initial hardware state is set, but most entries will seem quite mysterious for the moment. These are some important points to note (as highlighted in the preceding output segment):

• The U-Boot SPL first-stage bootloader is loaded first.
• The U-Boot second-stage bootloader is loaded next, and a full hardware description is then available.
• The GPIOs are checked to determine whether to boot from the eMMC or the SD card (e.g., check whether the Boot button is pressed).
• The configuration file /boot/uEnv.txt is loaded and parsed.
• The Linux kernel version is loaded (e.g., 4.9.82-ti-r102).
• U-Boot loads the U-Boot overlays to configure hardware (e.g., HDMI, analog-to-digital converters).
• The kernel begins executing.

The primary configuration file for the Beagle boards is /boot/uEnv.txt. You can manually edit this file (e.g., sudo nano /boot/uEnv.txt) to enable/disable hardware, to provide Linux command-line arguments, and to use the eMMC flasher script by commenting and uncommenting lines. For example, here is a cut-down configuration file:

debian@ebb:/boot$ more uEnv.txt
#Docs: http://elinux.org/Beagleboard:U-boot_partitioning_layout_2.0
###Master Enable
enable_uboot_overlays=1
 
###Custom Cape
#dtb_overlay=/lib/firmware/<file8>.dtbo
###
###Disable auto loading of virtual capes (emmc/video/wireless/adc)
#disable_uboot_overlay_emmc=1
#disable_uboot_overlay_video=1
#disable_uboot_overlay_audio=1
#disable_uboot_overlay_wireless=1
#disable_uboot_overlay_adc=1
 
###Cape Universal Enable
enable_uboot_cape_universal=1
 
cmdline=coherent_pool=1M net.ifnames=0 quiet
 
##enable Generic eMMC Flasher:
##make sure, these tools are installed: dosfstools rsync
#cmdline=init=/opt/scripts/tools/eMMC/init-eMMC-flasher-v3.sh 

Remove the comment in front of the last line in the file to call a script that overwrites the BeagleBone eMMC with the Linux image that is on the SD card.

Das U-Boot uses a board configuration file called a device tree (also called a device tree binary) containing the board-specific information that the kernel requires to boot the board. This file contains all the information needed to describe the memory size, clock speeds, on-board devices, and so on. This device tree binary (DTB), or the binary, is created from a DTS (the source) file using the device tree compiler (dtc). This topic is described in detail in Chapter 6, when examining how to interface to the Beagle board GPIOs.

debian@ebb:/boot/dtbs/4.9.82-ti-r102$ ls -l *.dtb
…
-rw-r--r-- 1 root root  64035 Feb 22 01:26 am335x-boneblack.dtb
-rw-r--r-- 1 root root  66633 Feb 22 01:26 am335x-boneblack-wireless.dtb
-rw-r--r-- 1 root root 137415 Feb 22 01:26 am335x-boneblue.dtb
-rw-r--r-- 1 root root  61710 Feb 22 01:26 am335x-bonegreen.dtb
…
-rw-r--r-- 1 root root 132769 Feb 22 01:26 am335x-pocketbeagle.dtb
… 

The DTS has the same syntax as the following extract, which details the four user LED pins and one of the two I²C buses on the BBB:

am33xx_pinmux: pinmux@44e10800 {
   pinctrl-names = "default";
   pinctrl-0 = <&userled_pins>;
 
   userled_pins: pinmux_userled_pins {
      pinctrl-single,pins = <
         0x54 0x07       /* gpmc_a5.gpio1_21, OUTPUT        | MODE7 */
         0x58 0x17       /* gpmc_a6.gpio1_22, OUTPUT_PULLUP | MODE7 */
         0x5c 0x07       /* gpmc_a7.gpio1_23, OUTPUT        | MODE7 */
         0x60 0x17       /* gpmc_a8.gpio1_24, OUTPUT_PULLUP | MODE7 */
      >;
    };
    i2c0_pins: pinmux_i2c0_pins {
       pinctrl-single,pins = <
          0x188 0x70      /* i2c0_sda, SLEWCTRL_SLOW | INPUT_PULLUP */
          0x18c 0x70      /* i2c0_scl, SLEWCTRL_SLOW | INPUT_PULLUP */
       >;
    }; …
}; 

The full description for the device tree source for Linux 4.x.x is available with the source code distribution of this book in the Chapter 3 directory. You will see how to download this code at the end of the chapter. This tree structure is discussed in detail in Chapter 6, when custom circuits are interfaced to the Beagle boards.

Kernel Space and User Space

The Linux kernel runs in an area of system memory called the kernel space, and regular user applications run in an area of system memory called user space. A hard boundary between these two spaces prevents user applications from accessing memory and resources required by the Linux kernel. This helps prevent the Linux kernel from crashing because of badly written user code; because it prevents applications that belong to one user from interfering with applications and resources that belong to another user, it also provides a degree of security.

The Linux kernel “owns” and has full access to all the physical memory and resources on the Beagle boards. Therefore, you have to be careful that only the most stable and trusted code is permitted to run in kernel space. You can see the architectures and interfaces illustrated in Figure 3-2, where user applications use the GNU C Library (glibc) to make calls to the kernel's system call interface. The kernel services are then made available to the user space in a controlled way through the use of system calls.

A kernel module is an object file that contains binary code, which can be loaded and unloaded from the kernel on demand. In many cases, the kernel can even load and unload modules while it is executing, without needing to reboot the board. For example, if you plug a USB Wi-Fi adapter into your board, it is possible for the kernel to use a loadable kernel module (LKM) to utilize the adapter. Without this modular capability, the Linux kernel would be extremely large, as it would have to support every driver that would ever be needed on any Beagle board. You would also have to rebuild the kernel every time you wanted to add new hardware. One downside of LKMs is that driver files have to be maintained for each device. (Interaction with LKMs is described throughout the book, and you will see how you can write your own LKMs in Chapter 16.)

As described in Figure 3-1, the bootloader stages pass control to the kernel after it has been decompressed into memory. The kernel then mounts the root file system. The kernel's last step in the boot process is to call systemd init (/sbin/init on the Beagle boards with the Debian Stretch image), which is the first user-space process that is started and the next topic that is discussed.

The systemd System and Service Manager

A system and service manager starts and stops services (e.g., web servers, Secure Shell [SSH] server) depending on the current state of the board (e.g., starting up, shutting down). The systemd system and service manager is a recent and somewhat controversial addition to Linux that aims to replace and remain backward compatible with System V (SysV) init. One major drawback of SysV init is that it starts tasks in series, waiting for one task to complete before beginning the next, which can lead to lengthy boot times. The systemd system is enabled by default in Debian 9 (Stretch). It starts up system services in parallel, helping to keep boot times short, particularly on multicore processors such as the BeagleBoard X15. In fact, you can display the boot time using the following (you may be able to log in some time before the user-space processes have completed):

debian@beaglebone:~$ systemctl --version
systemd 232 +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT
+UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID
+ELFUTILS +KMOD +IDN
debian@beaglebone:~$ systemd-analyze time
Startup finished in 15.638s (kernel) + 45.405s (userspace) = 1min 1.044s 

As well as being a system and service manager, systemd consists of a software bundle for login management, journal logging, device management, time synchronization, and more. Critics of systemd claim that its development project has suffered from “mission creep” and that it has taken on development work that is outside of its core mission. To some extent, this change in mission has resulted in systemd becoming core to the future of Linux itself, possibly even removing choice from users; however, it is clear that systemd is being widely adopted by many Linux distributions and here to stay.

You can use the systemctl command to inspect and control the state of systemd. If called with no arguments, it provides a full list of the services that are running on your board (there can be 100+ services running on a typical board). Use the Spacebar to page, and use Q to quit.

debian@beaglebone:~$ systemctl
UNIT                        LOAD   ACTIVE SUB       DESCRIPTION
-.mount                     loaded active mounted   Root Mount
apache2.service             loaded active running   The Apache HTTP Server
bonescript-autorun.service  loaded active running   Bonescript autorun
connman.service             loaded active running   Connection service
cron.service                loaded active running   Regular background …
[email protected]          loaded active running   Getty on tty1
[email protected] loaded active running   Serial Getty on ttyGS0
[email protected]  loaded active running   Serial Getty on ttyS0
ssh.service                 loaded active running   OpenBSD Secure Shell
… 

systemd uses service files, which have a .service extension to configure how the different services should behave on startup, shutdown, reload, and so on; see the /lib/systemd/system directory.

The Apache service runs by default upon installation. The systemd system can be used to manage such services on your board. For example, you can identify the exact service name and get its status using the following steps:

debian@beaglebone:~$ systemctl list-units -t service | grep apache
apache2.service    loaded active running The Apache HTTP Server
debian@beaglebone:~$ systemctl status apache2.service
• apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; …
   Active: active (running) since Wed 2018-05-02 16:04:32 IST; 1h 41min ago
  Process: 1029 ExecStart=/usr/sbin/apachectl start
 Main PID: 1117 (apache2)
    Tasks: 55 (limit: 4915)
   CGroup: /system.slice/apache2.service
           └_1117 /usr/sbin/apache2 -k start
           └_1120 /usr/sbin/apache2 -k start
           └_1122 /usr/sbin/apache2 -k start
May 02 16:04:28 beaglebone systemd[1]: Starting The Apache HTTP Server…
May 02 16:04:32 beaglebone systemd[1]: Started The Apache HTTP Server. 

At this point you can connect to the board on port 8080 using your web browser to verify that the Apache server is running as a service on the board, as illustrated in Figure 3-3.

You can stop the apache2 service using the systemctl command, whereupon the browser will issue a “connection refused” message.

debian@beaglebone:~$ sudo systemctl stop apache2
[sudo] password for debian:
debian@beaglebone:~$ systemctl status apache2
• apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; …
   Active: inactive (dead) since Wed 2018-05-02 17:51:12 IST; 17s ago
  Process: 1969 ExecStop=/usr/sbin/apachectl stop
  Process: 1029 ExecStart=/usr/sbin/apachectl start
 Main PID: 1117 (code=exited, status=0/SUCCESS) 

The service can then be restarted as follows:

debian@beaglebone:~$ sudo systemctl start apache2 

Table 3-1 provides a summary of systemd commands, using the apache2 service as a syntax example. Many of these commands require elevation to superuser permissions by use of the sudo tool, as described in the next section.

The runlevel describes the current state of your board and can be used to control which processes or services are started by the init system. Under SysV, there are different runlevels, identified as 0 (halt), 1 (single-user mode), 2 through 5 (multiuser modes), 6 (reboot), and S (startup). When the init process begins, the runlevel starts at N (none). It then enters runlevel S to initialize the system in single-user mode and finally enters one of the multiuser runlevels (2 through 5). To determine the current runlevel, type the following:

debian@beaglebone:~$ who -r
         run-level 5  2018-05-02 16:05 

In this case, the board is running at runlevel 5. You can change the runlevel by typing init followed by the level number. For example, you can reboot your board by typing the following:

debian@beaglebone:~$ sudo hostnamectl --static set-hostname ebb
debian@beaglebone:~$ hostname
ebb 

Also, edit the /etc/hosts file to have an entry for ebb, or you will receive “unable to resolve host” messages when you use the sudo command.

debian@beaglebone:~$ more /etc/hosts
127.0.0.1       localhost
127.0.1.1       beaglebone.localdomain  beaglebone  ebb
…
debian@beaglebone:~$ sudo init 6

As demonstrated, systemd retains some backward compatibility with the SysV runlevels and their numbers, as the previous SysV commands work correctly under systemd. However, the use of runlevels in systemd is considered to be a dated practice. Instead, systemd uses named target units, some of which are listed in Table 3-2, which includes an indicative alignment with SysV runlevels. You can identify the current default target on the board.

debian@ebb:~$ systemctl get-default
graphical.target 

This indicates that the current configuration is for the board to have a headful windowing display. You can also see the list of units that the target loads using the following:

debian@ebb:~$ systemctl list-units --type=target
UNIT                  LOAD   ACTIVE SUB     DESCRIPTION
basic.target          loaded active active  Basic System
cryptsetup.target     loaded active active  Encrypted Volumes
getty.target          loaded active active  Login Prompts
graphical.target      loaded active active  Graphical Interface
local-fs-pre.target   loaded active active  Local File Systems (Pre)
local-fs.target       loaded active active  Local File Systems
multi-user.target     loaded active active  Multi-User System
… 

If you are using your board as a network-attached device that does not have a display attached (i.e., headless), it is wasteful of CPU/memory resources to have the windowing services running.² You can switch to a headless target using the following call, whereupon the LXQt windowing interface will no longer be present, and the graphical.target entry will no longer appear in the list of units.

debian@ebb:~$ sudo systemctl isolate multi-user.target
debian@ebb:~$ who -r
         run-level 3  2018-05-02 16:18      last=5 

And, you can re-enable the headful graphical display using the following:

debian@ebb:~$ sudo systemctl isolate graphical.target
debian@ebb:~$ who -r
         run-level 5  2018-05-02 16:20      last=3 

Finally, to set up the board so that it uses a different default runlevel on boot (e.g., for a headless display), you can use the following:

debian@ebb:~$ sudo systemctl set-default multi-user.target
Created symlink /etc/systemd/system/default.target →
/lib/systemd/system/multi-user.target.
debian@ebb:~$ systemctl get-default
multi-user.target 

After reboot, the windowing services (if they are present) do not start, and the notional equivalent SysV runlevel is displayed as runlevel 3.

The Linux File System

Linux uses data structures, called inodes, to represent file system objects such as files and directories. When a Linux extended file system (e.g., ext3/ext4) is created on a physical disk, an inode table is created. This table links to an inode data structure for each file and directory on that physical disk. The inode data structure for each file and directory stores information such as permission attributes, pointers to raw physical disk block locations, time stamps, and link counts. You can see this with an example by performing a listing ls -ail of the root directory, where -i causes ls to display the inode indexes. You will see the following for the /tmp directory entry:

debian@ebb:~$ cd /
debian@ebb:/$ ls -ail | grep tmp
41337 drwxrwxrwt  10 root root  4096 May  2 18:17 tmp 

Therefore, 41337 is the /tmp directory's inode index. If you enter the /tmp directory by using cd, create a temporary file (a.txt), and perform ls -ail, you will see that the current (.) directory has the same inode index.

debian@ebb:/$ cd tmp
debian@ebb:/tmp$ touch a.txt
debian@ebb:/tmp$ ls -ail
total 40
 41337 drwxrwxrwt 10 root   root   4096 May  2 18:36 .
     2 drwxr-xr-x 21 root   root   4096 Apr 23 01:47 ..
  1893 -rw-r--r--  1 debian debian    0 May  2 18:36 a.txt 

You can also see that the root directory ( . .) has the inode index of 2 and that a text file ( a.txt ) also has an inode index, 1893. Therefore, you cannot cd directly to an inode index because the inode index might not refer to a directory.

Figure 3-4 illustrates the Linux directory listing and file permissions that relate to working with files under Linux. The first letter indicates the file type—for example, whether the listing is a (d) directory, (l) link, or (-) regular file. There are also some more obscure file types: (c) character special, (b) block special, (p) fifo, and (s) socket. Directories and regular files do not need further explanation, but links need special attention, as described next.

Links to Files and Directories

There are two types of links in Linux: soft links and hard links. A soft link (or symbolic link) is a file that refers to the location of another file or directory. Hard links, conversely, link directly to the inode index, but they cannot be linked to a directory. You create a link using ln /path/to/file.txt linkname. You create a symbolic link by adding -s to the call. To illustrate the usage, the following example creates a soft link and a hard link to a file /tmp/test.txt:

debian@ebb:~$ cd /tmp
debian@ebb:/tmp$ touch test.txt
debian@ebb:/tmp$ ln -s /tmp/test.txt softlink
debian@ebb:/tmp$ ln /tmp/test.txt hardlink
debian@ebb:/tmp$ ls -ail
total 40
 41337 drwxrwxrwt 10 root   root   4096 May  2 18:40 .
     2 drwxr-xr-x 21 root   root   4096 Apr 23 01:47 ..
  2172 -rw-r--r--  2 debian debian    0 May  2 18:40 hardlink
  2268 lrwxrwxrwx  1 debian debian   13 May  2 18:40 softlink -> /tmp/test.txt
  2172 -rw-r--r--  2 debian debian    0 May  2 18:40 test.txt 

The hard link has the same inode index as the test.txt file, and you can also see there is a number 2 in front of the file test.txt and hardlink entry (after the file permissions). This is the number of hard links that are associated with the file. This is a count value that was incremented by 1 when the hard link, called hardlink, was created. If you were to delete the hard link (e.g., using rm hardlink), this counter would decrement back to 1. To illustrate the difference between soft links and hard links, some text is added to the test.txt file.

debian@ebb:/tmp$ echo "testing links on the BB" >> test.txt
debian@ebb:/tmp$ more hardlink
testing links on the BB
debian@ebb:/tmp$ more softlink
testing links on the BB
debian@ebb:/tmp$ mkdir sub
debian@ebb:/tmp$ mv test.txt sub/
debian@ebb:/tmp$ more hardlink
testing links on the BB
debian@ebb:/tmp$ more softlink
more: stat of softlink failed: No such file or directory
  

You can see that when the test.txt file is moved to the subdirectory, the soft link breaks, but the hard link still works perfectly. Therefore, symbolic links are not updated when the linked file is moved, but hard links always refer to the source, even if moved or removed. To illustrate the last point, the file test.txt can be removed using the following:

debian@ebb:/tmp$ rm sub/test.txt
debian@ebb:/tmp$ more hardlink
testing links on the BB
debian@ebb:/tmp$ ls -ail hardlink
2172 -rw-r--r-- 1 debian debian 24 May  2 18:59 hardlink 

Yet, the file still exists! And it will not be deleted until you delete the hard link called hardlink, thus decrementing the link count to zero. You can see that the count is currently 1 by examining the hardlink entry. Therefore, if a file has a hard link count of zero and it is not being used by a process, it will be deleted. In effect, the filename itself, test.txt, was just a hard link. Note that you cannot hard link across different file systems, because each file system will have its own inode index table that starts at 1. Therefore, inode 269, which is the inode index of the /tmp directory, is likely describing something quite different on another file system. Type the command man ln to see a particularly useful guide on linking.

Users and Groups

Linux is a multiuser OS, which uses the following three distinct classes to manage access permissions:

• User: You can create different user accounts on your board. This is useful if you want to limit access to processes and areas of the file system. The root user account is the superuser for Debian and has access to every file; so, for example, it may not be safe to run a public web server from this account or the debian user account if the server supports local scripting.
• Group: User accounts may be flagged as belonging to one or more groups, whereby each group has different levels of access to different resources (e.g., gpios, I²C buses).
• Others: This includes all users of the system besides the file's owner, or a member of the group listed in the permissions.

You can create users at the Linux terminal. The full list of groups is available by typing more /etc/group. The following example demonstrates how you can create a new user account on the board and modify the properties of that account to suit your needs. You can list the groups that a user belongs to by typing groups at the shell prompt.

debian@ebb:~$ groups
debian adm kmem dialout cdrom floppy audio dip video plugdev
users systemd-journal i2c bluetooth netdev cloud9ide xenomai
weston-launch tisdk spi admin eqep pwm gpio 

To practice with the topics that were introduced earlier in this chapter, the following examples are performed using the molloyd user account. The first example demonstrates how to change the ownership of a file using the change ownership chown command and to change the group ownership of the file using the change group chgrp command.

For the sudo tool to be invoked correctly in the example, the user molloyd must be present in the sudo group, which is the case for the user debian and the user molloyd (from the previous example).

Alternatively, you can edit the sudoers file, which is achieved by the debian user account executing the visudo command. The file can be modified to include a molloyd entry, such as the following:

debian@ebb:~$ sudo visudo
debian@ebb:~$ sudo tail -n 1 /etc/sudoers
molloyd ALL=(ALL) ALL 

The molloyd user account can now execute the sudo command and must enter their user password to do so.

File System Permissions

The file system permissions state what levels of access each of the permissions classes have to a file or directory. The change mode command chmod enables a user to change the access permissions for file system objects. You can specify the permissions in a relative way. For example, chmod a+w test.txt gives all users write access to a file test.txt but leaves all other permissions the same. You can also apply the permissions in an absolute way. For example, chmod a=r test.txt sets all users to only have read access to the file test.txt. The next example demonstrates how to modify the file system permissions of a file using the chmod command.

Table 3-3 provides examples of the command structure for chown and chgrp. It also lists some example commands for working with users, groups, and permissions.

Here is an example of the last entry in Table 3-3, the stat command:

molloyd@ebb:/tmp$ stat /tmp/test.txt
  File: /tmp/test.txt
  Size: 0               Blocks: 0     IO Block: 4096   regular empty file
Device: b301h/45825d    Inode: 2305   Links: 1
Access: (0644/-rw-r--r--)  Uid: (0/ root)   Gid: ( 0/ root)
Access: 2018-05-02 20:03:01.164235706 +0100
Modify: 2018-05-02 20:03:01.164235706 +0100
Change: 2018-05-02 20:03:35.020235710 +0100
 Birth: - 

Note that each file in Linux retains an access, modify, and change time. You can update the access and modify times artificially using touch -a text.txt and touch -m test.txt, respectively (the change time is affected in both cases). The change time is also affected by system operations such as chmod; the modify time is affected by a write to the file; and, the access time is in theory affected by a file read. However, such operational behavior means that reading a file causes a write! This feature of Linux causes significant wear on the BeagleBone eMMC or SD card and results in I/O performance deficiencies. Therefore, the file access time feature is typically disabled on the boot eMMC/SD card using the mount option noatime within the /etc/fstab configuration file (covered in the next section). Note that there is also a similar nodiratime option that can be used to disable access time updates for directories only; however, the noatime option disables access time updates for both files and directories. You can see the noatime setting being set on the boot partition in /etc/fstab as follows:

molloyd@ebb:~$ more /etc/fstab | grep mmcblk
/dev/mmcblk0p1  /  ext4  noatime,errors=remount-ro  0  1 

Just to finish the discussion of Figure 3-4: The example in the figure has 22 hard links to the file. For a directory this represents the number of subdirectories, the parent directory (..) and itself (.). The entry is owned by root, and it is in the root group. The next entry of 4096 is the size required to store the metadata about files contained in that directory (the minimum size is one sector, typically 4,096 bytes).

One final point: if you perform a directory listing ls -ld in the root directory, you will see a t bit in the permissions of the /tmp directory. This is called the sticky bit, meaning that write permission is not sufficient to delete files. Therefore, in the /tmp directory any user can create files, but no user can delete another user's files.

molloyd@ebb:~$ cd /
molloyd@ebb:/$ ls -dhl tmp
drwxrwxrwt 11 root root 4.0K May  2 20:32 tmp 

The ls -dhl command lists (d) directory names (not their contents), with (h) human-readable file sizes, in (l) long format.

The Linux Root Directory

Exploring the Linux file system can be daunting for new Linux users. If you go to the top-level directory using cd / on the board and type ls, you will get the top-level directory structure, of the following form:

molloyd@ebb:/$ ls
bbb-uEnv.txt  boot  etc  ID.txt  lost+found  mnt   opt  root
sbin          sys   usr  bin     dev         home  lib  media
nfs-uEnv.txt  proc  run  srv     tmp         var 

What does it all mean? Well, each of these directories has a role, and if you understand the roles, you can start to get an idea of where to search for configuration files or the binary files that you need. Table 3-4 briefly describes the content of each top-level Linux subdirectory.

Commands for File Systems

In addition to commands for working with files and directories on file systems, there are commands for working with the file system itself. The first commands you should examine are df (remember as disk free) and mount. The df command provides an overview of the file systems on the board. Adding -T lists the file system types.

molloyd@ebb:~$ df -T
Filesystem     Type     1K-blocks    Used Available Use% Mounted on
udev           devtmpfs    219200       0    219200   0% /dev
tmpfs          tmpfs        49496    5604     43892  12% /run
/dev/mmcblk0p1 ext4      30707172 2066088  27342152   8% /
tmpfs          tmpfs       247476       0    247476   0% /sys/fs/cgroup
… 

The df command is useful for determining whether you are running short on disk space; you can see that the root file system (/) is 8% used in this case, with 27.3 GB (of a 32 GB SD card) available for additional software installations. Also listed are several temporary file system (tmpfs) entries that actually refer to virtual file systems, which are mapped to the board's DDR RAM. (The /sys/fs/* entries are discussed in detail in Chapter 8.)

The list block devices command lsblk provides you with a concise tree-structure list of the block devices, such as SD cards, USB memory keys, and USB card readers (if any), that are attached to the board. As shown in the following output, you can see that mmcblk0 (a boot SD card on a BeagleBone) has a single partition, p1, which is attached to the root of the file system: /. The eMMC on the BeagleBone is present as mmcblk1 in this case, even though the board is booted using the SD card:

molloyd@ebb:~$ lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0      179:0    0 29.8G  0 disk
└_mmcblk0p1  179:1    0 29.8G  0 part /
mmcblk1      179:8    0  3.7G  0 disk
└_mmcblk1p1  179:9    0  3.7G  0 part
mmcblk1boot0 179:24   0    1M  1 disk
mmcblk1boot1 179:32   0    1M  1 disk 

USB ports (or the SD card on a BeagleBone) can be used for additional storage, which is useful if you are capturing video data and there is insufficient capacity on the eMMC/system SD card. You can test the performance of SD cards to ensure that they meet the needs of your applications using the example that follows.

Using the mount command with no arguments provides you with further information about the file system on the board.

molloyd@ebb:~$ mount
/dev/mmcblk0p1 on / type ext4 (rw,noatime,errors=remount-ro,data=ordered)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)  … 

As previously discussed, the file system is organized as a single tree that is rooted at the root: /. Typing cd / brings you to the root point. The mount command can be used to attach a file system on a physical disk to this tree. File systems on separate physical devices can all be attached to named points at arbitrary locations on the single tree. Table 3-5 describes some file system commands that you can use to manage your file system and thereafter follows two examples that demonstrate how to utilize the mount command for important Linux system administration tasks.

molloyd@ebb:/$ sudo find . -name iostream*
./usr/include/c++/6/iostream
./usr/share/ti/cgt-pru/include/iostream
./usr/share/ti/cgt-pru/lib/src/iostream
./usr/share/ti/cgt-pru/lib/src/iostream.cpp … 

molloyd@ebb:~$ echo "Test file" >> new.txt
molloyd@ebb:~$ sudo find /home -mtime -1
/home/molloyd
/home/molloyd/.bash_history
/home/molloyd/new.txt
molloyd@ebb:~$ sudo find /home -mtime +1
/home/debian
/home/debian/.xsessionrc
/home/debian/.bash_logout    … 

molloyd@ebb:~$ whereis find
find: /usr/bin/find /usr/share/man/man1/find.1.gz /usr/share/info/find.info.gz 

molloyd@ebb:~$ more -5 /var/log/messages
Apr 30 06:25:05 beaglebone liblogging-stdlog:  …
Apr 30 03:37:41 beaglebone kernel: [    0.000000] Booting Linux …
Apr 30 03:37:41 beaglebone kernel: [    0.000000] Linux version …
--More--(0%) 

molloyd@ebb:~$ less /var/log/messages

molloyd@ebb:~$ dmesg 

The Reliability of SD Card/eMMC File Systems

One of the most likely points of failure of a single-board computer (SBC) is its SD card, which is more generally known as a multimedia card (MMC) or an embedded MMC (eMMC) when it is built onto the SBC (as is the case for the BeagleBone). NAND-based flash memory, such as that in MMCs, has a large capacity and a low cost, but it is prone to wear, which can result in file system errors.

The high capacity of MMCs is largely because of the development of multilevel cell (MLC) memory. Unlike single-level cell (SLC) memory, more than 1 bit can be stored in a single memory cell. The high voltage levels required in the process of deleting a memory cell disturbs adjacent cells, so NAND flash memory is erased in blocks of 1 KB to 4 KB. Over time, the process of writing to the NAND flash memory causes electrons to become trapped, reducing the conductivity difference between the set and erased states. (For a discussion on SLC versus MLC for high-reliability applications, see tiny.cc/beagle301.) MLCs use different charge levels and higher voltages to store more states in a single cell. (Commercial MLC products typically offer 4 to 16 states per cell.) Because SLCs store only a single state, they have a reliability advantage (typically 60,000–100,000 erase/write cycles) versus MLC (typically 10,000 cycles). MMCs are perfectly suitable for daily use in applications such as digital photography or security camera recording; 10,000 cycles should last more than 27 years at one entire card write per day.

However, embedded Linux devices constantly write to their MMCs for tasks such as logging system events in /var/log/. If your board writes to a log file 20 times per day, the lifespan of the SD card could be as low as 8 months. These are conservative figures, and thanks to wear leveling algorithms, the lifespan may be much longer. Wear leveling is employed by MMCs during data writes to ensure that rewrites are evenly distributed over the entire MMC media, thus avoiding system failure of Linux devices because of concentrated modifications, such as changes to log files.

For your Beagle board, ensure that you purchase a high-quality branded SD card. In addition, the more unused space you have on the SD card, the better, because it further enhances the wear leveling performance. The BeagleBone uses an eMMC storage—essentially an MMC on a chip that has the same order of reliability as SD cards. However, one important advantage of eMMCs is that the board manufacturer has control over the quality and specification of the storage device used. Finally, most consumer SSDs are also MLC based, with the more expensive SLC-based SSDs typically reserved for enterprise-class servers.

For Beagle applications that require extended reliability, a RAM file system (tmpfs) could be used for the /tmp directory, for the /var/cache directory, and for log files (particularly /var/log/apt). You can achieve this by editing the /etc/fstab file to mount the desired directories in memory. For example, if you have processes that require file data to be shared between them for the purpose of data interchange, you could use the /tmp directory as a RAM file system (tmpfs) by editing the /etc/fstab file as follows:

molloyd@ebb:/etc$ sudo nano fstab
molloyd@ebb:/etc$ more fstab
/dev/mmcblk0p1  /   ext4  noatime,errors=remount-ro  0  1
debugfs         /sys/kernel/debug  debugfs  defaults  0  0
tempfs          /tmp   tempfs size=100M  0  0
molloyd@ebb:/etc$ mkdir /tmp/swap
You can then apply these settings using the mount command:
molloyd@ebb:/etc$ sudo mount -a
And then check that the settings have been applied:
molloyd@ebb:/etc$ mount
…
tmpfs on /tmp/swap type tmpfs (rw,relatime,size=102400k) 

The root directory (/) is mounted by default with the noatime attribute set, which dramatically reduces the number of writes and increases I/O performance (as described earlier in the chapter). You should apply this attribute when possible to all solid-state storage devices (e.g., USB memory keys), but it is not necessary for RAM-based storage.

Remember that any data written to a tmpfs will be lost on reboot. Therefore, if you use a tmpfs for /var/log, any system errors that caused your board to crash will not be visible on reboot. You can test this fact by creating a file in the /tmp/swap/ directory as configured earlier and rebooting.

The actual RAM allocation grows and shrinks depending on the file usage on the tmpfs disk; therefore, you can be reasonably generous with the memory allocation. Here's an example with the 100 MB /tmp/swap/ tmpfs mounted:

molloyd@ebb:/tmp/swap$ cat /proc/meminfo | grep MemFree:
MemFree:          213404 kB
molloyd@ebb:/tmp/swap$ fallocate -l 50000000 test.txt
molloyd@ebb:/tmp/swap$ ls -l test.txt
-rw-r--r-- 1 molloyd molloyd 50000000 May  4 12:30 test.txt
molloyd@ebb:/tmp/swap$ cat /proc/meminfo | grep MemFree:
MemFree:          164548 kB
molloyd@ebb:/tmp/swap$ rm test.txt
molloyd@ebb:/tmp/swap$ cat /proc/meminfo | grep MemFree:
MemFree:          213296 kB 

It is possible to use a read-only file system to improve the life span of the SD card and the stability of the file system (e.g., SquashFS compressed file system), but this requires significant effort and is not suitable for the type of prototype development that takes place in this book. However, keep it in mind for a final project or product deployment where system stability is crucial.

Output and Input Redirection (>, >>, and <)

It is possible to redirect the output to a file using redirection symbols > and >>. The >> symbol was used previously in this chapter to add text to temporary files. The > symbol can be used to send the output to a new file. Here's an example:

molloyd@ebb:/tmp$ date > a.txt
molloyd@ebb:/tmp$ more a.txt
Fri May  4 12:33:32 IST 2018
molloyd@ebb:/tmp$ date > a.txt
molloyd@ebb:/tmp$ more a.txt
Fri May  4 12:33:45 IST 2018 

The >> symbol indicates that you want to append to the file. The following example illustrates the use of >> with the new file a.txt:

molloyd@ebb:/tmp$ date >> a.txt
molloyd@ebb:/tmp$ more a.txt
Fri May  4 12:33:45 IST 2018
Fri May  4 12:34:23 IST 2018 

Standard input using the < symbol works in much the same way. The inclusion of -e enables parsing of escape characters, such as the return ( ) characters, which places each animal type on a new line.

molloyd@ebb:/tmp$ echo -e "dog
cat
yak
cow" > animals.txt
molloyd@ebb:/tmp$ sort < animals.txt
cat
cow
dog
yak 

You can combine input and output redirection operations. Using the same animals.txt file, you can perform operations such as the following:

molloyd@ebb:/tmp$ sort < animals.txt > sorted.txt
molloyd@ebb:/tmp$ more sorted.txt
cat
cow
dog
yak 

Pipes (| and tee)

Simply put, pipes (|) enable you to connect Linux commands. Just as you redirected the output to a file, you can redirect the output of one command into the input of another command. For example, to list the root directory (from anywhere on the system) and send (or pipe) the output into the sort command, where it is listed in reverse (-r) order, use the following:

molloyd@ebb:~$ ls / | sort -r
var
usr
…
bin 

You can identify which user installations in the /opt directory occupy the most disk space: du gives you the disk used. Passing the argument -d1 means only list the sizes of 1 level below the current directory level, and -h means list the values in human-readable form. You can pipe this output into the sort filter command to do a numeric sort in reverse order (largest at the top). Therefore, the command is as follows:

molloyd@ebb:~$ du -d1 -h /opt | sort -nr
500K    /opt/backup
198M    /opt
98M     /opt/source
75M     /opt/cloud9
25M     /opt/scripts 

Another useful tool, tee, enables you to both redirect an output to a file and pass it on to the next command in the pipe (e.g., store and view). Using the previous example, if you want to send the unsorted output of du to a file but display a sorted output, you could enter the following:

molloyd@ebb:~$ du -d1 -h /opt | tee unsorted.txt | sort -nr
500K    /opt/backup
198M    /opt
98M     /opt/source
75M     /opt/cloud9
25M     /opt/scripts
molloyd@ebb:~$ more unsorted.txt
500K    /opt/backup
25M     /opt/scripts
98M     /opt/source
75M     /opt/cloud9
198M    /opt 

You can also use tee to write the output to several files simultaneously:

molloyd@ebb:~$ du -d1 -h /opt | tee 1.txt 2.txt 3.txt
500K    /opt/backup
…
molloyd@ebb:~$ ls
1.txt  2.txt  3.txt  unsorted.txt 

Filter Commands (from sort to xargs)

Each of the filtering commands provides a useful function:

• sort: This command has several options: -r sorts in reverse, -f ignores case, -d uses dictionary sorting and ignores punctuation, -n does a numeric sort, -b ignores blank space, -i ignores control characters, -u displays duplicate lines only once, and -m merges multiple inputs into a single output.
• wc (word count): Calculates the number of words, lines, or characters in a stream. Here's an example:

  molloyd@ebb:/tmp$ more animals.txt
  dog
  cat
  yak
  cow
  molloyd@ebb:/tmp$ wc < animals.txt
  4  4 16 

This command returns that there are 4 lines, 4 words, and 16 characters (including hidden characters, e.g., carriage returns). You can select the values independently by using line count (-l), word count (-w), and character count (-m), and -c prints out the byte count (which would also be 16 in this case).

• head: This command displays the first lines of the input, which is useful if you have a long file or stream of information and want to examine only the first few lines. By default, it displays the first 10 lines. You can specify the number of lines using the -n option. For example, to get the first two lines of output of the dmesg command (display message or driver message), which displays the message buffer of the kernel, use the following:

     molloyd@ebb:~$ dmesg | head -n2
     [    0.000000] Booting Linux on physical CPU 0x0
     [    0.000000] Linux version 4.9.88-ti-rt-r111 … 

• tail: This command works like head except that it displays the last lines of a file or stream. Using it in combination with dmesg provides useful output, as shown here:

     molloyd@ebb:~$ dmesg | tail -n2
     [  178.194805] pvrsrvkm: loading out-of-tree module taints kernel.
     [  178.556171] [drm] Initialized pvr 1.14.3699939 20110701 on minor 0 

• grep: This command parses lines using text and regular expressions. You can use this command to filter output with options, including ignore case (-i), stop after five matches (-m 5), silent (-q), will exit with return status of 0 if any matches are found, specify a pattern (-e), print a count of matches (-c), print only the matching text (-o), and list the filename of the file containing the match (-l). For example, the following examines the dmesg output for the first three occurrences of the string usb, using -i to ignore case:

     molloyd@ebb:~$ dmesg | grep -i -m3 usb
     [    0.809529] usbcore: registered new interface driver usbfs
     [    0.809622] usbcore: registered new interface driver hub
     [    0.809795] usbcore: registered new device driver usb 

You can combine pipes. For example, you get the same output by using head and displaying only the first two lines of the grep output.

     molloyd@ebb:~$ dmesg | grep -i -m3 usb | head -n2
     [    0.809529] usbcore: registered new interface driver usbfs
     [    0.809622] usbcore: registered new interface driver hub 

• xargs: This command enables you to construct an argument list that you use to call another command or tool. In the following example, a text file args.txt that contains three strings is used to create three new files. The output of cat is piped to xargs, where it passes the three strings as arguments to the touch command, creating three new files: a.txt, b.txt, and c.txt.

  molloyd@ebb:~$ echo "a.txt b.txt c.txt" > args.txt
  molloyd@ebb:~$ cat args.txt | xargs touch
  molloyd@ebb:~$ ls
  args.txt  a.txt  b.txt  c.txt 

Other useful filter commands include awk (to program any type of filter), fmt (to format text), uniq (to find unique lines), and sed (to manipulate a stream). These commands are beyond the scope of this text; for example, awk is a full programming language! Table 3-6 describes useful piped commands to give you some ideas of how to use them.

echo and cat

The echo command simply echoes a string, output of a command, or a value to the standard output. Here are a few examples:

molloyd@ebb:~$ echo 'hello'
hello
molloyd@ebb:~$ echo "Today's date is $(date)"
Today's date is Fri May  4 13:45:36 IST 2018
molloyd@ebb:~$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games 

In the first case, a simple string is echoed. In the second case, the " " are present as a command is issued within the echo call, and in the final case the PATH environment variable is echoed.

The echo command also enables you to see the exit status of a command using $?. Here's an example:

molloyd@ebb:~$ ls
args.txt  a.txt  b.txt  c.txt
molloyd@ebb:~$ echo $?
0
molloyd@ebb:~$ ls /nosuchdirectory
ls: cannot access '/nosuchdirectory': No such file or directory
molloyd@ebb:~$ echo $?
2 

Clearly, the exit status for ls is 0 for a successful call and 2 for an invalid argument. This can be useful when you are writing scripts and your own programs that return a value from the main() function.

The cat command (concatenation) facilitates you in joining two files together at the command line. The following example uses echo to create two files: a.txt and b.txt; cat concatenates the files to create a new file c.txt. You need to use -e if you want to enable the interpretation of escape characters in the string that is passed to echo.

molloyd@ebb:~$ echo "Hello" > a.txt
molloyd@ebb:~$ echo -e "from
the
PocketBeagle" > b.txt
molloyd@ebb:~$ cat a.txt b.txt > c.txt
molloyd@ebb:~$ more c.txt
Hello
from
the
PocketBeagle 

diff

The diff command facilitates you in finding the differences between two files. It provides basic output.

molloyd@ebb:~$ echo -e "dog
cat
bird" > list1.txt
molloyd@ebb:~$ echo -e "dog
cow
bird" > list2.txt
molloyd@ebb:~$ diff list1.txt list2.txt
2c2
< cat
---
> cow 

The value 2c2 in the output indicates that line 2 in the first file changed to line 2 in the second file, and the change is that cat changed to cow. The character a means appended, and d means deleted. For a side-by-side comparison, you can use the following:

molloyd@ebb:~$ diff -y -W30 list1.txt list2.txt
dog             dog
cat           | cow
bird            bird 

where -y enables the side-by-side view and -W30 sets the width of the display to 30-character columns.

If you want a more intuitive (but challenging) difference display between two files, you can use the vimdiff command (installed using sudo apt install vim), which displays a side-by-side comparison of the files using the vim (Vi IMproved) text editor (type vimdiff list1.txt list2.txt and use the VI key sequence: Escape : q ! twice to quit or Escape : w q to save the changes and quit). Vim requires practice to master the key sequences.

tar

The tar command is an archiving utility that enables you to combine files and directories into a single file (like an uncompressed zip file). This file can then be compressed to save space. To archive and compress a directory of files, such as /tmp, use the following:

molloyd@ebb:~$ tar cvfz backup.tar.gz /tmp
molloyd@ebb:~$ ls -l backup.tar.gz
-rw-r--r-- 1 molloyd molloyd 416 May  4 13:54 backup.tar.gz 

where c means new archive, v means verbosely list files, z means compress with gzip, and f means archive name follows. You might also see .tar.gz represented as .tgz. See Table 3-7 for more examples.

md5sum

The md5sum command enables you to check the hash code to verify that the files have not been corrupted maliciously or accidentally in transit. In the following example, the wavemon tool is downloaded as a .deb package but not installed. The md5sum command can be used to generate the md5 checksum.

molloyd@ebb:~$ sudo apt download wavemon
Get:1 http://deb.debian.org/debian stretch/main armhf wavemon armhf
0.8.1-1 [49.3 kB] Fetched 49.3 kB in 0s (247 kB/s)
molloyd@ebb:~$ ls -l *.deb
-rw-r--r-- 1 root root 49286 Jan  2  2017 wavemon_0.8.1-1_armhf.deb
molloyd@ebb:~$ md5sum wavemon_0.8.1-1_armhf.deb
557728e1da32bb7f68b09099755c9ca3  wavemon_0.8.1-1_armhf.deb 

You can now check this checksum against the official checksum to ensure you have a valid file. Unfortunately, it can be difficult to find the checksums for individual packages online. If wavemon is installed, the checksums are in /var/lib/dpkg/info/wavemon.md5sums. You can install a utility under Debian called debsums to check the integrity of the file and its constituent parts.

molloyd@ebb:~$ sudo apt install debsums wavemon
molloyd@ebb:~$ debsums wavemon_0.8.1-1_armhf.deb
/usr/bin/wavemon                                                       OK
…
/usr/share/man/man5/wavemonrc.5.gz                                     OK
/usr/share/menu/wavemon                                                OK 

If you are building your own packages that you want to distribute, it would be useful to also distribute a checksum file against which users can verify their downloaded repository. An alternative to md5sum is sha256sum, which can be used in the same way.

molloyd@ebb:~$ sha256sum wavemon_0.8.1-1_armhf.deb
6536009848b1063e831d7d9aae70f6505e46e8ecc88e161f6bd7034ba11ae1dc 

How to Control Linux Processes

The ps command lists the processes currently running on the board. Typing ps shows that the following PocketBeagle is running two user processes, the bash shell with process ID (PID) 1953 and the ps command itself, which is running with PID 1968. The ps PID is different every time you run it because it runs to completion each time.

molloyd@ebb:~$ ps
  PID TTY          TIME CMD
 1953 pts/0    00:00:00 bash
 1968 pts/0    00:00:00 ps 

To see all running processes, use ps ax. In the following example, it is filtered to search for the string “apache” to discover information about the apache2 processes that are running on the board:

molloyd@ebb:~$ ps ax | grep apache
 1107 ?        Ss     0:00 /usr/sbin/apache2 -k start
 1110 ?        Sl     0:00 /usr/sbin/apache2 -k start
 1111 ?        Sl     0:00 /usr/sbin/apache2 -k start
 1970 pts/0    S+     0:00 grep apache 

It is clear that three different processes are running for the service, enabling it to handle multiple simultaneous connections. In this example, all threads are currently waiting for an event to complete (S), PID 1107 is the session leader (Ss), 1110 and 1111 are its multithreaded clones (Sl), and the 1970 grep process is in the foreground group (S+). As described earlier, a call to systemctl status apache2 provides information about the services running on the PocketBeagle—if you execute the call, you will see that the process PIDs match those displayed by a call to ps.

Foreground and Background Processes

Linux is a multitasking OS that enables you to run processes in the background while using a program that is running in the foreground. This concept is similar to the behavior of a windowing system (e.g., Windows, macOS). For example, the desktop clock continues to update the time while you use a web browser.

The same is true of applications that run in a terminal window. To demonstrate that, here is a small segment of C code to display “Hello World!” every five seconds in a Linux terminal. Exactly how this works is covered in Chapter 5, but for the moment, you can enter the code verbatim into a file called HelloBeagle.c using the nano file editor within the molloyd user home directory, as follows:

molloyd@ebb:~$ nano HelloBeagle.c
molloyd@ebb:~$ more HelloBeagle.c
#include<unistd.h>
#include<stdio.h>
int main(){
   int x=0;
   do{
      printf("Hello Beagle!
");
      sleep(5);
   }while(x++<50);
   return 0;
} 

The program has 50 iterations, displaying a message and sleeping for five seconds on each iteration. After saving the file as HelloBeagle.c, it can be compiled to an executable by typing the following (-o specifies the executable filename):

molloyd@ebb:~$ gcc HelloBeagle.c -o hello
molloyd@ebb:~$ ls -l hello
-rwxr-xr-x 1 molloyd molloyd 8384 May  4 16:26 hello 

If this works correctly, you will now have the source file and the executable program called hello (note that the executable x flag is set). It can then be executed.

molloyd@ebb:~$ ./hello
Hello Beagle!
Hello Beagle!
^C 

It will continue to output this message every five seconds; it can be killed using Ctrl+C. However, if you would like to run this in the background, you have two options.

The first way is that, instead of using Ctrl+C to kill the process, use Ctrl+Z, and then at the prompt type the bg (background) command.

molloyd@ebb:~$ ./hello
Hello Beagle!
^Z
[1]+  Stopped                 ./hello
molloyd@ebb:~$ bg
[1]+ ./hello &
molloyd@ebb:~$ Hello Beagle!
Hello Beagle!
Hello Beagle! 

When you press Ctrl+Z, the ^Z displays in the output. When bg is entered, the process is placed in the background and continues to execute. In fact, you can continue to use the terminal, but it will be frustrating, because “Hello Beagle!” displays every five seconds. You can bring this process back into the foreground using the fg command.

molloyd@ebb:~$ fg
./hello
Hello Beagle!
^C
molloyd@ebb:~$ 

The application is killed when Ctrl+C is typed (displays as ^C).

The second way to place this application in the background is to execute the application with an & symbol after the application name.

molloyd@ebb:~$ ./hello &
[1] 1992
molloyd@ebb:~$ Hello Beagle!
Hello Beagle!
Hello Beagle! 

The process has been placed in the background with PID 1992 in this case. To stop the process, use ps with this PID or find the PID using the ps command.

molloyd@ebb:~$ ps |grep hello
 1992 pts/0    00:00:00 hello
molloyd@ebb:~$ Hello Beagle! 

To kill the process, use the kill command.

molloyd@ebb:~$ kill 1992
[1]+  Terminated              ./hello 

You can confirm that a process is dead by using ps again. If a process doesn't die, you can use a -9 argument to ensure death! (e.g., kill -9 1992). A separate command, pkill, will kill a process based on its name, so in this case you can kill the process as follows:

molloyd@ebb:~$ pkill hello 

One more command worth mentioning is watch, which executes a command at a regular interval and shows the outcome full screen on the terminal. For example, to watch the kernel message log, use the following:

molloyd@ebb:~$ watch dmesg 

You can specify the time interval between each execution using -n followed by the number of seconds. A good way to understand watch is to execute it as follows:

molloyd@ebb:~$ watch -n 1 ps a
Every 1.0s: ps a       ebb: Fri May  4 16:34:52 2018
  PID TTY      STAT   TIME COMMAND
 1033 tty1     Ss+    0:00 /sbin/agetty --noclear tty1 linux
 …
 2018 pts/0    S+     0:00 watch -n 1 ps a
 2046 pts/0    S+     0:00 watch -n 1 ps a
 2047 pts/0    S+     0:00 sh -c ps a
 2048 pts/0    R+     0:00 ps a 

You will see the PID of ps, sh, and watch changing every one (1) second, making it clear that watch is actually executing the command (ps) by passing it to a new shell using sh -c. The reason why watch appears twice in the list is that it spawns itself temporarily at the exact moment that it executes ps a.

Cloning a Repository (git clone)

Cloning a repository means making a copy of all the files in the repository on your local file system, as well as the history of changes to that project. You do this operation only once. To clone the repository, issue the command git clone followed by the fully formed repository name:

molloyd@ebb:~$ cd ~/
molloyd@ebb:~$ git clone https://github.com/derekmolloy/test.git
Cloning into 'test'…
remote: Counting objects: 20, done.
remote: Total 20 (delta 0), reused 0 (delta 0), pack-reused 20
Unpacking objects: 100% (20/20), done. 

You now have a full copy of the test repository in the /test/ directory. Your repository is just as complete as the version on the GitHub server; if you were to deploy it over a network, file system, other Git server, or even a different GitHub account, it could assume the role as the main version of this repository. Although there is no need for a central server, it is usually the case, because it enables multiple users to “check in” source code to a known master repository. The repository is created in the /test/ directory, and it currently contains the following:

molloyd@ebb:~/test$ ls -al
total 20
drwxr-xr-x 3 molloyd molloyd 4096 May  4 23:18 .
drwxr-xr-x 4 molloyd molloyd 4096 May  4 23:16 ..
drwxr-xr-x 8 molloyd molloyd 4096 May  4 23:16 .git
-rw-r--r-- 1 molloyd molloyd    4 May  4 23:16 .gitignore
-rw-r--r-- 1 molloyd molloyd   59 May  4 23:16 README.md 

You can see the README.md file that was created when the project was initialized on GitHub; you can use more to view the contents of this file. The directory also contains a hidden .git subdirectory, which contains the following files and directories:

molloyd@ebb:~/test$ cd .git
molloyd@ebb:~/test/.git$ ls
branches  config   description  HEAD  hooks  index  info
logs      objects  packed-refs  refs 

The hidden .git folder contains all the information about the repository, such as commit messages, logs, and the data objects. For example, the remote repository location is maintained in the config file.

molloyd@ebb:~/test/.git$ more config | grep url
        url = https://github.com/derekmolloy/test.git 

The “Further Reading” section at the end of this chapter directs you to an excellent book on Git, which is freely available online, that describes the nature of the .git directory structure in detail. Thankfully, in the following discussion, you do not have to make changes in the .git directory structure because you have Git commands to do that for you.

Getting the Status (git status)

Now that the repository exists, the next step is to add a new text file to the working directory, where it will be in an untracked state. When you call the command git status, you can see a message stating that “untracked files” are present.

molloyd@ebb:~/test$ echo "Just some text" > newfile.txt
molloyd@ebb:~/test$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>…" to include in what will be committed)
        newfile.txt
nothing added to commit but untracked files present (use "git add" to track) 

The next step is to add any untracked files to the staging area. However, if you did not want to add a set of files, you could also create a .gitignore file to ignore those files. For example, this could be useful if you are building C/C++ projects and you decide that you do not want to add intermediate .o files. Here is an example of creating a .gitignore file in order to ignore C/C++ .o files:

molloyd@ebb:~/test$ echo "*.o" > .gitignore
molloyd@ebb:~/test$ more .gitignore
*.o
molloyd@ebb:~/test$ touch testobject.o
molloyd@ebb:~/test$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>…" to include in what will be committed)
        newfile.txt
nothing added to commit but untracked files present (use "git add" to track) 

In this case, two files are untracked, but there is no mention of the testobject.o file, as it is being correctly ignored. Note that the .gitignore file is itself part of the repository and so will persist when the repository is cloned, along with its revision history and so on.

Adding to the Staging Area (git add)

The files in the working directory can now be added to the staging area by typing git add .

This command adds all the files in the working directory, with the exception of the ignored files. In this example, two files are added from the working directory to the staging area, and the status of the repository can then be displayed using the following:

molloyd@ebb:~/test$ git add .
molloyd@ebb:~/test$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>…" to unstage)
       new file:   .gitignore
       new file:   newfile.txt 

To delete (remove) a file from the staging area, use git rm somefile.ext.

Committing to the Local Repository (git commit)

After you add files to the staging area, you can commit the changes from the staging area to the local Git repository. First, you may want to add your name and e-mail address variables to identify who is committing the changes.

molloyd@ebb:~/test$ git config --global user.name "Derek Molloy"
molloyd@ebb:~/test$ git config --global user.email " [email protected]" 

These values are set against your Linux user account, so they will persist when you next log in. You can see them by typing more ~/.gitconfig.

To permanently commit the file additions to the local Git repository, use the git commit command.

molloyd@ebb:~/test$ git commit -m "Testing repository for BeagleBone"
[master b2137ee] Testing repository for BeagleBone
 2 files changed, 2 insertions(+)
 create mode 100644 newfile.txt
 create mode 100644 .gitignore 

The changes are flagged with the username, and a message is also required. If you want to detail the message inline, use -m to set the commit message.

Pushing to the Remote Repository (git push)

To perform this step, you must have your own GitHub account. The git push command pushes any code updates to the remote repository. You must be registered to make changes to the remote repository for the changes to be applied. In Git 2.0, a new more conservative approach, called simple, has been taken to push to remote repositories. It is chosen by default, but a warning message can be squelched, and the push can be performed as follows (replace the user details and repository name with your own account details):

molloyd@ebb:~/test$ git config --global push.default simple
molloyd@ebb:~/test$ git push
Username for 'https://github.com': derekmolloy
Password for 'https://[email protected]': mySuperSecretPassword
Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (4/4), 350 bytes | 0 bytes/s, done.
Total 4 (delta 0), reused 0 (delta 0)
To https://github.com/derekmolloy/test.git
   61b6ee4..b2137ee  master -> master 

After the code has been pushed to the remote repository, you can pull changes back to a local repository on any machine by issuing a git pull command from within the local repository directory.

molloyd@ebb:~/test$ git pull
Already up-to-date. 

In this case, everything is already up-to-date.

Creating a Branch (git branch)

Suppose, for example, you want to create a new branch called mybranch; you can do so using the command git branch mybranch, and then you can switch to that branch using git checkout mybranch, as shown here:

molloyd@ebb:~/test$ git branch mybranch
molloyd@ebb:~/test$ git checkout mybranch
Switched to branch 'mybranch' 

Now, to demonstrate how this works, suppose that a temporary file called testmybranch.txt is added to the repository. This could be a new code file for your project. You can see that the status of the branch makes it clear that the working directory contains an untracked file.

molloyd@ebb:~/test$ touch testmybranch.txt
molloyd@ebb:~/test$ ls
newfile.txt  README.md  testmybranch.txt  testobject.o
molloyd@ebb:~/test$ git status
On branch mybranch
Untracked files:
  (use "git add <file>…" to include in what will be committed)
        testmybranch.txt
nothing added to commit but untracked files present (use "git add" to track) 

You can then add this new file to the staging area of the branch using the same commands.

molloyd@ebb:~/test$ git add .
molloyd@ebb:~/test$ git status
On branch mybranch
Changes to be committed:
  (use "git reset HEAD <file>…" to unstage)
        new file:   testmybranch.txt 

You can commit this change to the mybranch branch of the local repository. This change will affect the mybranch branch but have no impact on the master branch.

molloyd@ebb:~/test$ git commit -m "Test commit to mybranch"
[mybranch 63cae5f] Test commit to mybranch
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 testmybranch.txt
molloyd@ebb:~/test$ git status
On branch mybranch
nothing to commit, working tree clean
molloyd@ebb:~/test$ ls
newfile.txt  README.md  testmybranch.txt  testobject.o 

You can see from the preceding output that the file testmybranch.txt is committed to the local repository and you can see the file in the directory.

If you now switch from the branch mybranch to the master branch using the call git checkout master, you will see that something interesting happens when you request the directory listing.

molloyd@ebb:~/test$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
molloyd@ebb:~/test$ ls
newfile.txt  README.md  testobject.o 

Yes, the file testmybranch.txt has disappeared from the directory! It still exists, but it is in a blob form inside the .git/objects/ directory. If you return to the branch and list the directory, you will see the following:

molloyd@ebb:~/test$ git checkout mybranch
Switched to branch 'mybranch'
molloyd@ebb:~/test$ ls
newfile.txt  README.md  testmybranch.txt  testobject.o 

The file now reappears. Therefore, you can see just how well integrated the branching system is. At this point, you can go back to the master branch and make changes to the original code without the changes in the mybranch branch having any impact on the master code. Even if you change the code in the same file, it has no effect on the original code in the master branch.

Merging a Branch (git merge)

What if you want to apply the changes that you made in the mybranch branch to the master project? You can do this by using git merge.

molloyd@ebb:~/test$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
molloyd@ebb:~/test$ git merge mybranch
Updating b2137ee..63cae5f
Fast-forward
 testmybranch.txt | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 testmybranch.txt
molloyd@ebb:~/test$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)
nothing to commit, working tree clean
molloyd@ebb:~/test$ ls
newfile.txt  README.md  testmybranch.txt  testobject.o 

Now the testmybranch.txt file is in the master branch, and any changes that were made to other documents in the master have been applied. The local repository is now one commit ahead of the remote repository, and you can use git push to update the remote repository.

Deleting a Branch (git branch -d)

If you want to delete a branch, use the git branch -d mybranch command.

molloyd@ebb:~/test$ git branch -d mybranch
Deleted branch mybranch (was 63cae5f).
molloyd@ebb:~/test$ ls
newfile.txt  README.md  testmybranch.txt  testobject.o 

In this case, the file testmybranch.txt is still present in the master project—and it should be because the branch was merged with the master project. If the branch had been deleted before the merge was performed, the file would have been lost.