The Shared Library

In this section, you'll learn how to create and use shared libraries. You've already seen hints about the shared library, in Figure 13.1.

Limitations of Static Libraries

Figure 13.2 shows how the linker automatically extracts object modules from an archive and loads them as required. Although linking only what you need with your program provides a certain amount of economy, there is still duplication when looking at the system-wide picture. Imagine a huge hypothetical static library that contains 90% of the functions used by the Netscape Web browser. Netscape is then linked with this library, producing perhaps a 5MB executable file. Approximately 90% of this executable file will be a copy of what was contained in the static library.

Assume that you want to build a Web-enabled program that creates a Netscape X Window from within your application. Your 200KB object module links with this Netscape static library, and the resulting executable program is written out with a size of 4.5MB. Now you have a 5MB Netscape executable and a 4.5MB program, but 90% of both programs is the same code.

Consider further that five users running Netscape and three users running your custom application consume a large amount of memory within the system. Add more users, and the UNIX kernel will start doing some serious swapping.

Shared libraries provide a mechanism that allows a single copy of code to be shared by several instances of programs in the system.

Creating a Shared Library

In times past, shared library creation and maintenance required some real hand waving by UNIX system administration wizards. To create a shared library for your own use under FreeBSD or Linux, you can simply use the -shared option of the gcc(1) command. Using the earlier example, the shared library for the class Passwd is created as follows:

$ cc -o libshared.so import.o getuid.o -shared
						

The gcc(1) command is executed with the -shared option, causing the output file to be written as a shared library rather than an executable file. In this case, file libshared.so is the library created. The suffix .so is used to indicate shared library files under FreeBSD and Linux.

Linking with a Shared Library

Using the shared library is straightforward, but there can be some complications. First, examine how the link step is performed:

$ gcc main.o -o getuid -L. -lshared -lstdc++
						

Note the use of the -L and -l options. The -L option specifies an additional directory to search for a shared library. The -lshared option tells it the name of the library to search (the prefix lib and the suffix .so are added for shared libraries, resulting in libshared.so being searched). Because the linker knows that methods Passwd:getuid() and Passwd::getnam() are in the shared library, the linker simply "makes a note" about this in the final executable file that is written. These notes allow the shared library to be loaded when the program is executed.

Choosing Static or Dynamic Libraries

When both shared and static libraries are available, gcc(1) normally will choose the shared library. However, if you specify the option -static on the gcc(1) command line, the link phase will use static libraries instead where possible.

Listing Shared Library References

Under FreeBSD and Linux, you can check the new executable file getuid, to see if it is referencing the new shared library that was created earlier:

$ ldd ./getuid
./getuid:
        libshared.so => not found (0x0)
        libstdc++.so.2 => /usr/lib/libstdc++.so.2 (0x28063000)
        libc.so.3 => /usr/lib/libc.so.3 (0x2809a000)
        libm.so.2 => /usr/lib/libm.so.2 (0x2811b000)
$

From the output shown, it can be seen that ./getuid is indeed referencing a shared library named libshared.so. The not found message indicates that ldd(1) cannot locate the library. Running the program under these conditions would confirm this:

$ ./getuid
/usr/libexec/ld-elf.so.1: Shared object "libshared.so" not found
$

Why didn't the dynamic loader find the shared library? To find out why, you need to understand more about the dynamic loader.

The Dynamic Loader

Shared libraries require more attention than do static libraries. This is because shared libraries must be found and loaded on demand.

When ldd(1) was used earlier, the dynamic loader was used to test each referenced library found in the executable. This dynamic loader is used to perform the loading and dynamic linking of other shared libraries.

Searching for Shared Libraries

In order for the shared library to be loaded at runtime, the dynamic loader must know where to locate it at runtime. Just as the shell must have a search path for commands, the dynamic loader needs a search mechanism for its libraries.

FreeBSD and Linux both share a cache file that indicates where libraries can be found. The following lists where the cache files are located:

These cache files are updated by the ldconfig(8) command under FreeBSD and Linux.

Other UNIX platforms use environment variables to select custom library directories. FreeBSD and Linux also support these environment variables. Among the different UNIX platforms, there are three search path variables in use. Table 13.1 lists these variables and the platforms that use them.

All of these environment variables work in the same fashion as the PATH variable. A colon- separated list of directories to be searched is provided.

Using the LD_LIBRARY_PATH Variable

Since FreeBSD inspects the LD_LIBRARY_PATH variable, the examples given will use it. Recall the example that was shown earlier:

$ ldd ./getuid
./getuid:
        libshared.so => not found (0x0)
        libstdc++.so.2 => /usr/lib/libstdc++.so.2 (0x28063000)
        libc.so.3 => /usr/lib/libc.so.3 (0x2809a000)
        libm.so.2 => /usr/lib/libm.so.2 (0x2811b000)
$

To fix the search difficulty with your newly created shared library, the LD_LIBRARY_PATH variable can be modified to include your current directory (using the shell variable $PWD):

$ LD_LIBRARY_PATH=$PWD
$ export LD_LIBRARY_PATH
$ ldd ./getuid
./getuid:
        libshared.so => /home/me/myproject/libshared.so (0x28063000)
        libstdc++.so.2 => /usr/lib/libstdc++.so.2 (0x28065000)
        libc.so.3 => /usr/lib/libc.so.3 (0x2809c000)
        libm.so.2 => /usr/lib/libm.so.2 (0x2811d000)
$

Notice that, with the LD_LIBRARY_PATH modified to include your current directory, the dynamic loader is able to locate your shared library file libshared.so. If you have other directories already included in the present LD_LIBRARY_PATH variable, this is a better approach:

$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD

This simply appends your current directory to the values you already have in effect.

Testing the LD_LIBRARY_PATH Variable

With the LD_LIBRARY_PATH variable properly set, you can now run the test program, as follows:

$ ./getuid
Root's home dir is /root.
Account uucp uses the shell /usr/libexec/uucp/uucico.
No such file or directory: looking up account xyzzy.
Account games uses the shell /sbin/nologin.
$

If you download the source code for this project, you will see that the output reflects a successful run for this test program. The No such file or directory: error message was supposed to occur as part of this test.

Position-Independent Code

There is one small matter that has been overlooked, which is important to shared libraries. For a shared library to be effective at sharing its code with several programs, it should be compiled in position-independent code form.

When a program is compiled in position-independent code form, it can be executed from any memory location without regard to its starting address. This makes it possible for the same physical memory segments to be shared virtually at different relative positions in each process that references it.

Figure 13.3 shows Program_A and Program_B, two programs that call upon the same shared library. The shaded areas in the memory images show where in the address space the shared code appears. Notice that the shared library code in Program_A is lower than it is in Program_B. Only one physical copy of this code exists in the system's physical memory, which is managed by the UNIX kernel. The shaded areas represent virtual memory mappings of the same shared code in both processes.

For shared library code to execute in the way Figure 13.3 shows, the code must be compiled as position-independent code. If this is not done, the dynamic loader must create multiple copies of the same library in memory, with different starting addresses.

To compile a module as position-independent code, the gcc(1) compile option -fPIC can be used under FreeBSD and Linux:

$ cc -c -D_POSIX_C_SOURCE=199309L -D_POSIX_SOURCE -Wall -fPIC -fhandle-exceptions import.cc
$ cc -c -D_POSIX_C_SOURCE=199309L -D_POSIX_SOURCE -Wall -fPIC -fhandle-exceptions getuid.cc

These commands compile the given modules into position-independent code that can be made into a shared library.

Controlling What Is Shared

When you make a UNIX shared library, you must control what is externally visible to the user of your library. By default, whatever remains external in the normal sense of executables will also be visible externally to the user of your shared library. If you have functions internal to your library, it is a good idea to define them as static functions wherever possible. This keeps them private.

Likewise, it is a good practice to have no unnecessary global variables or common storage. They will be visible from your shared library, also. Sloppiness in this area can cause programs to invoke functions or global variables in your shared library that you did not intend to release to the general public.