Programs are linked with libraries. You could link them all statically, in which case, there would be no libraries on the target device. But, that takes up an unnecessarily large amount of storage if you have more than two or three programs. So, you need to copy shared libraries from the toolchain to the staging directory. How do you know which libraries?
One option is to copy all of them since they must be of some use, otherwise they wouldn't exist! That is certainly logical and, if you are creating a platform to be used by others for a range of applications, that would be the correct approach. Be aware, though, that a full glibc is quite large. In the case of a CrossTool-NG build of glibc 2.19, the space taken by /lib and /usr/lib is 33 MiB. Of course, you could cut down on that considerably by using uClibc or Musel libc libraries.
Another option is to cherry pick only those libraries that you require, for which you need a means of discovering library dependencies. Using some of our knowledge from Chapter 2, Learning About Toolchains libraries, you can use readelf for that task:
$ cd ~/rootfs $ arm-cortex_a8-linux-gnueabihf-readelf -a bin/busybox | grep "program interpreter" [Requesting program interpreter: /lib/ld-linux-armhf.so.3] $ arm-cortex_a8-linux-gnueabihf-readelf -a bin/busybox | grep "Shared library" 0x00000001 (NEEDED) Shared library: [libm.so.6] 0x00000001 (NEEDED) Shared library: [libc.so.6]
Now you need to find these files in the toolchain and copy them to the staging directory. Remember that you can find sysroot like this:
$ arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot /home/chris/x-tools/arm-cortex_a8-linux-gnueabihf/arm-cortex_a8-linux-gnueabihf/sysroot
To reduce the amount of typing, I am going to keep a copy of that in a shell variable:
$ export SYSROOT=`arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot`
If you look at /lib/ld-linux-armhf.so.3, in sysroot, you will see that, it is, in fact, a symbolic link:
$ ls -l $SYSROOT/lib/ld-linux-armhf.so.3 [...]/sysroot/lib/ld-linux-armhf.so.3 -> ld-2.19.so
Repeat the exercise for libc.so.6 and libm.so.6 and you will end up with a list of three files and three symbolic links. Copy them using cp -a, which will preserve the symbolic link:
$ cd ~/rootfs $ cp -a $SYSROOT/lib/ld-linux-armhf.so.3 lib $ cp -a $SYSROOT/lib/ld-2.19.so lib $ cp -a $SYSROOT/lib/libc.so.6 lib $ cp -a $SYSROOT/lib/libc-2.19.so lib $ cp -a $SYSROOT/lib/libm.so.6 lib $ cp -a $SYSROOT/lib/libm-2.19.so lib
Repeat this procedure for each program.
Tip
It is only worth doing this to get the very smallest embedded footprint possible. There is a danger that you will miss libraries that are loaded through dlopen(3) calls - plugins mostly. We will look at an example with the NSS libraries when we come to configure network interfaces later on in this chapter.
Libraries and programs are often compiled with a symbol table information built in, more so if you have compiled with the debug switch, -g. You seldom need these on the target. A quick and easy way to save space is to strip them. This example shows libc before and after stripping:
$ file rootfs/lib/libc-2.19.so rootfs/lib/libc-2.19.so: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.15.4, not stripped $ ls -og rootfs/lib/libc-2.19.so -rwxrwxr-x 1 1547371 Feb 5 10:18 rootfs/lib/libc-2.19.so $ arm-cortex_a8-linux-gnueabi-strip rootfs/lib/libc-2.19.so $ file rootfs/lib/libc-2.19.so rootfs/lib/libc-2.19.so: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.15.4, stripped $ ls -l rootfs/lib/libc-2.19.so -rwxrwxr-x 1 chris chris 1226024 Feb 5 10:19 rootfs/lib/libc-2.19.so $ ls -og rootfs/lib/libc-2.19.so -rwxrwxr-x 1 1226024 Feb 5 10:19 rootfs/lib/libc-2.19.so
In this case, we saved 321,347 bytes, which was about 20%.
When stripping kernel modules, use the following command:
strip --strip-unneeded <module name>
Otherwise, you will strip out the symbols needed to relocate the module code and it will fail to load.