A.1.1 Static Libraries

A static library contains object code that is linked with an end-user application and then becomes part of that executable. Figure A.1 illustrates this concept. A static library is sometimes called an archive because it is essentially just a package of compiled object files. These libraries normally have a file extension of .a on UNIX and Mac OS X machines or .lib on Windows, for example, libjpeg.a or jpeg.lib.

Some implications of distributing your API’s implementation as a static library are:

A.1.2 Dynamic Libraries

Dynamic libraries are files linked against at compile time to resolve undefined references and then distributed with the end-user application so that the application can load the library code at run time (see Figure A.2). This normally requires use of a dynamic linker on the end user’s machine to determine and load all dynamic library dependencies at run time, perform the necessary symbol relocations, and then pass control to the application. For example, the Linux dynamic linker is called ld.so and on the Mac it is called dyld. Often, the dynamic linker supports a number of environment variables to modify or debug its behavior. For example, refer to ‘man dyld’ on the Mac.

Dynamic libraries are sometimes called shared libraries because they can be shared by multiple programs. On UNIX machines they can be called Dynamic Shared Objects, and on Windows they are referred to as Dynamic Link Libraries. They have a .so file extension on UNIX platforms, .dll on Windows, and .dylib on Mac OS X, for example, libjpeg.so or jpeg.dll.

Some implications of using dynamic libraries to distribute your API include the following.

It’s also important to understand the behavior of dynamic libraries that depend on other dynamic libraries. If your library depends on other dynamic libraries, then you must also ship those libraries with your SDK, unless you can reasonably expect the end-user’s platform to have those libraries preinstalled. This is a transitive property: you must also ship all of the dependencies of your dependencies. Essentially, the entire chain of dynamic libraries must be available to the application at run time for it to be able to run. Of course, if your dynamic library links against a static library then you don’t need to provide that static library to your clients because the code from the static library will be added directly to your dynamic library.

A.1.3 Dynamic Libraries as Plugins

Dynamic libraries are normally linked against an application and then distributed with that application so that the operating system can load the library when the application is launched. However, it is also possible for an application to load a dynamic library on demand without the application having been compiled and linked against that library.

This can be used to create plugin interfaces, where the application can load additional code at run time that extends the basic capabilities of the program. For example, most Web browsers support plugins to handle specific content types, such as displaying Adobe Flash animations or viewing Apple QuickTime movies. This use of dynamic libraries is illustrated in Figure A.3.

In terms of API development, this gives you the capability to create extensible APIs that allow your clients to drop in new functionality that your API will then load and execute. The Netscape Plugin API is an example of this: it is the API that you develop against in order to create a plugin (i.e., dynamic library) that browsers such as Firefox, Safari, Opera, and Chrome can then load and run. I discussed the use of plugins to create extensible APIs in Chapter 12.

A.2.1 Importing and Exporting Functions

As discussed in Chapter 6, if you want a function to be callable from a DLL on Windows, you must explicitly mark its declaration with the following keyword:

For example,

Conversely, if you want to use an exported DLL function in an application then you must prefix the function prototype with the following keyword:

Consequently, it’s common to employ preprocessor macros to use the export declaration when building an API but the import decoration when using the same API in an application. It’s also important to note that because these __declspec decorations may cause compile errors on non-Windows compilers, you should only use them when compiling under Windows. The following preprocessor code provides a simple demonstration of this. (See Section 6.9 in Chapter 6 for a more complete cross-platform example.)

You can then declare all of the symbols you want to export from your DLL as follows:

As an alternative to modifying your source code with these __declspec declarations, you can create a module definition .def file to specify the symbols to export. A minimal DEF file contains a LIBRARY statement to specify the name of the DLL the file is associated with, and an EXPORTS statement followed by a list of symbol names to export.

The DEF file syntax also supports more powerful manipulations of your symbols, such as renaming symbols or using an ordinal number as the export name to help minimize the size of the DLL. The ordinal number represents the position of a symbol’s address pointer in the DLL’s export table. Using ordinal numbers for your DLL symbols can produce slightly faster and smaller libraries. However, from an API stability perspective this can be risky because seemingly innocuous changes to the DEF file can then change the exported symbols for your API. Therefore, I recommend using full symbol names rather than ordinal values when specifying your DLL exports.

A.2.2 The DLL Entry Point

DLLs can provide an optional entry point function to initialize data structures when a thread or process loads the DLL or to clean up memory when the DLL is unloaded. This is managed by a function called DllMain() that you define and export within the DLL. If the entry point function returns FALSE, this is assumed to be a fatal error and the application will fail to start. The following code provides a DLL entry point template.

A.2.3 Creating Libraries on Windows

The following steps describe how to create a static library on Windows. These steps are for Microsoft Visual Studio 2008 (9.0), although the steps are similar for other versions of Visual Studio.

You can then add new or existing source files to your project under the Source Files folder in the left-hand pane. Then, when you perform a build for your project, the result will be a static library .lib file.

The steps to create a DLL are very similar. The only difference is during Step 4, where you will select the DLL option instead of Static library. Then, when you build your project, Visual Studio will generate a .dll file and an associated .lib import file.

A.2.4 Useful Windows Utilities

A number of programs can help you manage DLLs on Windows and investigate DLL problems. Many of these are command-line tools that you can run from the MS-DOS prompt. A few of these DLL utilities are:

A.2.5 Loading Plugins on Windows

On the Windows platform, the LoadLibrary() or LoadLibraryEx() functions can be used to load a dynamic library into a process, with the GetProcAddress() function being used to obtain the address of an exported symbol in the DLL. Note that you do not need an import library .lib file in order to load a dynamic library in this way. To demonstrate this, consider the following simple plugin interface used to create a plugin.dll library.

Then the following code snippet illustrates how to load this DLL on demand and call the DoSomething() method from that library.

A.3.1 Creating Static Libraries on Linux

On Linux, a static library is simply an archive of object (.o) files. You can use the Linux ar command to compile a number of object files into a static library. For example, the following commands demonstrate how to compile three .cpp files to .o files using the GNU C++ compiler and then creating a static library from those object files.

The -c option to g++ tells the compiler to produce a .o file from the input .cpp file. The options to ar are -c creates an archive, -r inserts the supplied .o files into that archive, and -s creates an index for the archive (equivalent to the older convention of running ranlib on the resulting archive).

Your users can then link against your library using the -l option to ld or g++. This specifies the name of the library to link against. The -L linker option can also be used to specify the directory where your library can be found. For example,

In this example, the end-user application userapp is created by compiling usercode.cpp and linking against the libmyapi.a static library in the same directory.

The order of archives on this command line is significant. For each archive that the linker finds on the command line, it looks to see if that archive defines any symbols that were referenced from any object files specified earlier on the command line. If it does define any needed symbols, the object files with those symbols are copied into the executable. It is therefore best practice to specify libraries at the end of the command line (Mitchell et al., 2001).

While I am discussing the creation of static libraries, it is worth noting proper usage of the -static compiler option. This flag is used for the creation of executables, not libraries. It is therefore applicable to users of your API, but not to the building of your API itself. This flag instructs the compiler to prefer linking the static versions of all dependent libraries into the executable so that it depends on no dynamic libraries at run time.

A.3.2 Creating Dynamic Libraries on Linux

Creating a dynamic library on Linux is a very similar process to creating a static library. Using the GNU C++ compiler, you can simply use the -shared linker option to generate a .so file instead of an executable.

On platforms where it is not the default behavior, you should also specify either the -fpic or the -fPIC command line option to instruct the compiler to emit position-independent code (PIC). This is needed because the code in a shared library may be loaded into a different memory location for different executables. It’s therefore important to generate PIC code for shared libraries so that user code does not depend on the absolute memory address of symbols. The following example illustrates how to compile three source files into a dynamic library.

Users can then link your dynamic library into their code using the same compile line shown earlier for the static library case, that is,

If you have both a static library and a dynamic library with the same base name in the same directory, that is, libmyapi.a and libmyapi.so, the compiler will use the dynamic library (unless you use the -static library option to require only static libraries to be used). To favor use of a static library over a dynamic library with the same base name, you could place the static library in a different directory and ensure that this directory appears earlier in the library search path (using the -L linker option).

Note that in a dynamic library, all code is essentially flattened into a single object file. This is in contrast to static libraries that are represented as a collection of object files that can be copied individually into an executable as needed (i.e., object files in a static archive that are not needed are not copied into the executable image). As a result, loading a dynamic library will involve loading all the code defined in that .so file (Mitchell et al., 2001).

By default, all symbols in a DSO are exported publicly (unless you specify the -fvisibility= hidden compiler option). However, the GNU C++ compiler supports the concept of export maps to define explicitly the set of symbols in a dynamic library that will be visible to client programs. This is a simple ASCII format where symbols can be listed individually or using glob-style expressions. For example, the following map file, export.map, specifies that only the DoSomething() function should be exported and that all other symbols should be hidden.

This map file can then be passed to the compiler when building a dynamic library using the --version-script linker option, as in the following example:

A.3.3 Shared Library Entry Points

It’s possible to define functions that will be called automatically when your shared library is loaded or unloaded. This can be used to perform library initialization and cleanup operations without requiring your users to call explicit functions to perform this.

One way to do this is using static constructors and destructors. This will work for any compiler and any platform, although you should remember that the order of initialization of static constructors is not defined across translation unit boundaries, that is, you should never depend on static variables in other .cpp files being initialized. Bearing this caveat in mind, you could create a shared library entry point in one of your .cpp files as follows:

There is an alternate, more elegant, approach. However, it is specific to the GNU compiler. This involves using the constructor and destructor __attribute__ decorations for functions. For example, the following code shows you how to define library initialization and cleanup routines and hide these within one of your .cpp files.

If you use this approach, you should be aware that your shared library must not be compiled with the GNU GCC arguments -nostartfiles or -nostdlib.

A.3.4 Useful Linux Utilities

Several standard Linux utilities can help you work with static and shared libraries. Of particular note is the GNU libtool shell script. This command provides a consistent and portable interface for creating libraries on different UNIX platforms. The libtool script can be used in various ways, but in its simplest form you can just give libtool a list of object files, specify either the -static or the -dynamic option, and it will then create a static or dynamic library, respectively. For example,

The libtool script can be very useful if you want your source code to compile easily on a range of UNIX platforms without worrying about the idiosyncrasies of creating libraries on each platform.

Another useful command for working with libraries is nm, which can be used to display symbol names in an object file or library. This is useful to find out if a library defines or uses a given symbol. For example, the following command line will output all of the global (external) symbols in a library:

This will produce output like the following:

The character in the second column specifies the symbol type, where “T” refers to a text section symbol that is defined in this library and “U” refers to a symbol that is referenced by the library but is not defined by it. An uppercase letter represents an external symbol, whereas a lowercase character represents an internal symbol. The string in the third column provides the mangled symbol name. This can be unmangled using the c++filt command, for example,

Another useful command is ldd. This can be used to display the list of dynamic libraries that an executable depends on. Because this will display the full path that will be used for each library, you can see which version of a dynamic library will be loaded and whether any dynamic libraries cannot be found by the operating system. For example, the following output is produced on Linux for a simple executable.

An executable that has been linked with the -static option will not depend on any dynamic libraries. Running ldd on such an executable results in the following output on Linux:

Finally, if you have a static library it’s possible to convert it to a dynamic library library if it was compiled with position-independent code (e.g., -fPIC). Recall that a static archive (.a) is just a packaging of object files (.o). You can therefore extract the individual object files using the ar command and then relink them as a dynamic library. For example,

A.3.5 Loading Plugins on Linux

On Linux platforms, you can use the dlopen() function call to load a .so file into the current process. Then you can use the dlsym() function to access symbols within that library. This lets you create plugin interfaces, as described earlier. For example, consider the following very simple plugin interface:

You can build a dynamic library for this API, such as libplugin.so. Then the following code demonstrates how to load this library and call the DoSomething() function within that .so file:

A.3.6 Finding Dynamic Libraries at Run Time

When you run an executable that depends on a dynamic library, the system will search for this library in a number of standard locations, normally /lib and /usr/lib. If the .so file cannot be found in any of these locations the executable will fail to launch. Recall that the ldd command can be used to tell you if the system cannot find any dependent dynamic library. This is obviously a concern for creating executable programs that depend on your API. Three main options are available to your clients to ensure that any executable they build using your API can find your library at run time.

A.4.1 Creating Static Libraries on Mac OS X

Static libraries can be created on Mac OS X in the same way as for Linux, that is, using ar or libtool. However, there are some different behaviors when linking a static library into an application.

In particular, Apple discourages use of the -static compiler option to generate executables with all library dependencies linked statically. This is because Apple wants to ensure that applications always pull in the latest system libraries that they distribute. In fact, the gcc man page states that -static “will not work on Mac OS X unless all libraries (including libgcc.a) have also been compiled with -static. Since neither a static version of libSystem.dylib nor crt0.o is provided, this option is not useful to most people.”

Essentially, the -static option on the Mac is reserved for building the kernel, or for the very brave.

Related to this situation, by default the Mac linker will scan through all paths in the library search path looking for a dynamic library. If it fails, it will then scan the paths again looking for a static library. This means that you cannot use the trick of favoring a static library by placing it in a directory that appears earlier in the library search path. However, there is a linker option called -search-paths-first that will cause the linker to look in each search path for a dynamic library and then, if not found, to look for a static library in the same directory. This option makes the Mac linker behavior more like the Linux linker in this respect. Note, however, that there is no way to favor linking against a static library over a dynamic library on the Mac when both are located in the same directory.

A.4.2 Creating Dynamic Libraries on Mac OS X

Dynamic libraries can be created on Mac OS X in a very similar way to the Linux instructions given earlier. There is one important difference in that you should use the -dynamiclib option to g++ to create dynamic libraries on the Mac instead of -shared.

Also, note the use of the -headerpad_max_install_names option. This flag is highly recommended when building dynamic libraries on the Mac for reasons I will explain in a moment.

It should also be noted that the ldd command is not available on Mac OS X. Instead, you can use the otool command to list the collection of dynamic libraries that an executable depends on, that is,

A.4.3 Frameworks on Mac OS X

Mac OS X also introduces the concept of frameworks as a way to distribute all the files necessary to compile and link against an API in a single package. A framework is simply a directory with a .framework extension that can contain various resources such as dynamic libraries, header files, and reference documentation. It is essentially a bundle, in the NSBundle sense. This bundling of all necessary development files in a single package can make it easier to install and uninstall a library. Also, a framework can contain multiple versions of a library in the same bundle to make it easier to maintain backward compatibility for older applications.

The following directory listing gives an example layout for a framework bundle, where the -> symbol represents a symbol link.

Most of Apple’s APIs are distributed as frameworks, including Cocoa, Foundation, and Core Services. They can be found under the /Developer/SDKs directory, assuming that you have installed the Mac OS X development environment. You may therefore wish to distribute your API as a framework on the Mac to make it appear more like a native Mac library.

You can build your API as a framework using Apple’s XCode development environment. This can be done by selecting the File > New Project menu and then selecting Framework in the left-hand panel (see Figure A.5). By default, you can choose to have your project setup to use either Carbon or Cocoa frameworks. If you don’t need either of these (i.e., you are writing a pure C++ library), you can simply remove these frameworks after XCode has created the project for you.

Clients can link against your framework by supplying the -framework option to g++ or ld. They can also specify the -F option to specify the directory to find your framework bundle.

A.4.4 Finding Dynamic Libraries at Run Time

The same Linux principles for finding dynamic libraries at run time apply to applications run under Mac OS X, with a couple of small differences. The first is that the environment variable used to augment the library search path is called DYLD_LIBRARY_PATH (although LD_LIBRARY_PATH is now also supported by more recent versions of Mac OS X).

Also, Mac OS X does not support the Linux -rpath linker option. Instead, it provides the notion of install names. An install name is a path that is burned into a Mach-O binary to specify the path to search for dependent dynamic libraries. This path can be specified relative to the executable program by starting the install name with the special string @executable_path.

You can specify an install name when you build your dynamic library, but your clients may also change this path using the install_name_tool utility. However, they cannot specify a path that is longer than the original path in the .dylib. This is why it is always advisable to build your dynamic libraries with the -headerpad_max_install_names option on the Mac: to give your clients the flexibility to change the library’s install name to whatever they wish.

The following commands demonstrate how a client could change the install name for your library and change the install name for their executable: