Object file linking in practice

To demonstrate some typical problems that you may run into when trying to link object files to Delphi, I picked two simple examples from the Abbrevia open source library. Abbrevia was originally a commercial compression library developed by TurboPower. They were selling multiple libraries—AsyncPro, Orpheus, and SysTools, which were later donated to the open source community. Their current home is at https://github.com/TurboPack.

For simplicity, I have included the complete Abbrevia repository in the code archive. All demos are already configured so you can just press F9 in Delphi and enjoy.

The first demo is called LzmaDecTest and it links in a single object file—LzmaDec.obj. That file is stored in the AbbreviasourceWin32 folder (or Win64 if you are compiling for 64-bit Windows). This demo is a standard Windows console application with one additional line linking the object file:

{$L LzmaDec.obj}

If you try to compile the test project, you'll notice that compilation fails with the following error message:

[dcc32 Error] LzmaDecTest.dpr(22): E2065 Unsatisfied forward or external declaration: 'memcpy'

This happens a lot when linking object files created with C compilers. An object file can contain references to functions that are not implemented inside the object file. The same situation happens with Delphi—a .dcu file can (and most probably will) contain references to methods implemented in different units.

An experienced programmer will try to minimize these dependencies when writing a library, simply because that helps with portability, but there are always some basic functions that are necessary in almost all C programs. Think of the Delphi's System unit—all units in your program implicitly depend on it and most of the time you are not aware of that (and you don't have to be). Most of the object files that you'll have to link to will be compiled with Microsoft's C compiler, and typically they will depend on a msvcrt library.

There are two ways out of this problem. Firstly, you can write the missing functions yourself. They are implemented by the MSVCRT.DLL dynamic library and you only have to link to them.

For example, in our case we would need the missing memcpy function. We can simply import it from the appropriate DLL by adding the following line to the program. We would also have to add the Windows unit to the uses list as the size_t type is defined in it:

function memcpy(dest, src: Pointer; count: size_t): Pointer; cdecl; 
external 'msvcrt.dll';

Alternatively, we can just add the System.Win.Crtl unit to the uses list. It links to all functions from the MSVCRT.DLL dynamic library.

Be aware though, MSVCRT.DLL is not part of the operating system. If you want to distribute an application that links to this DLL, you should also include the Microsoft Visual C++ Redistributable package, which you can download from the Microsoft web server.

For the second demo, I wanted to link to the decompress.obj file, which is also part of Abbrevia and is stored in the same folder as LzmaDec.obj. This demo is named DecompressTest.

Again, I started with an empty console application to which I added {$LINK decompress.obj}. ({$L} and ${LINK} are synonyms.) The compiler reported four errors:

[dcc32 Error] DecompressTest.dpr(45): E2065 Unsatisfied forward or external declaration: 'BZ2_rNums'
[dcc32 Error] DecompressTest.dpr(45): E2065 Unsatisfied forward or external declaration: 'BZ2_hbCreateDecodeTables'
[dcc32 Error] DecompressTest.dpr(45): E2065 Unsatisfied forward or external declaration: 'BZ2_indexIntoF'
[dcc32 Error] DecompressTest.dpr(45): E2065 Unsatisfied forward or external declaration: 'bz_internal_error'

To fix such problems, you'll always have to use the documentation that comes with the object files. There's no way to know what those symbols represent simply by looking at the object file. In this case, I found out that the first is an initialization table (so the symbol is actually a name of a global variable). The second and third functions are implemented in other object files, and the last one is an error handling function that we have to implement in the code. Adding the following code fragment to the file left me with two errors:

var
BZ2_rNums: array[0..511] of Longint;

procedure bz_internal_error(errcode: Integer); cdecl;
begin
raise Exception.CreateFmt('Compression Error %d', [errcode]);
end;

Take note of the calling convention used here. C object files will almost invariably use the cdecl convention.

To find the object files with missing functions, you can use a full-text search tool that handles binary files, and searche for missing names. This will give you false positives as it will also return object files that are using those functions. You can then use the tdump utility to examine potential candidates and find the true source of those units.

Or, you can look into the documentation. But who does that?

I found BZ2_hbCreateDecodeTables in huffman.obj and BZ2_indexIntoF in bzlib.obj. Only two lines are needed to add them to the project:

{$LINK huffman.obj}
{$LINK bzlib.obj}

That fixes the two errors about missing symbols but introduces three new errors:

[dcc32 Error] DecompressTest.dpr(40): E2065 Unsatisfied forward or external declaration: 'BZ2_crc32Table'
[dcc32 Error] DecompressTest.dpr(40): E2065 Unsatisfied forward or external declaration: 'BZ2_compressBlock'
[dcc32 Error] DecompressTest.dpr(40): E2065 Unsatisfied forward or external declaration: 'BZ2_decompress'

BZ2_crc32Table is an initialization table (says the documentation) so we need another variable:

var
BZ2_crc32Table: array[0..255] of Longint;

Further detective work found BZ2_compressBlock in the file compress.obj, so let's add this to the project. We only have to add one line:

{$LINK compress.obj}

The second error is trickier. Research showed that this function is actually implemented in the decompress.obj unit, which is already linked to the project! Why is the error reported then? Simply because that's how the single-pass Delphi compiler works.

As it turns out, the compiler will happily resolve symbols used by object file A if they are defined in object file B, which is loaded after file A. It will, however, not resolve symbols used by object file B if they are defined in object file A.

In our case, the compress.obj file needs the BZ2_decompress symbol, which is defined in decompress.obj, but as the latter is linked before the former, we get the error. You could try rearranging the units but then some other symbol will not be found.

Luckily, this problem is very simple to fix. We just have to tell the compiler that BZ2_decompress exists. The compiler is happy with that and produces workable code. When all units are compiled, linker kicks in to collect them in the EXE file. Linker is smarter than the compiler and finds the correct implementation, regardless of the include order.

If we add the following line to the code, the error about unsatisfied BZ2_decompress declaration goes away. I also like putting in a comment that marks the source of the function:

procedure BZ2_decompress; external; //decompress.obj

Linking compress.obj causes three new errors to appear:

[dcc32 Error] DecompressTest.dpr(45): E2065 Unsatisfied forward or external declaration: 'BZ2_hbMakeCodeLengths'
[dcc32 Error] DecompressTest.dpr(45): E2065 Unsatisfied forward or external declaration: 'BZ2_hbAssignCodes'
[dcc32 Error] DecompressTest.dpr(45): E2065 Unsatisfied forward or external declaration: 'BZ2_blockSort'

The first two are defined in huffman.obj, which is already linked, so we have to add the following two lines to the code:

procedure BZ2_hbMakeCodeLengths; external; //huffman.obj
procedure BZ2_hbAssignCodes; external; //huffman.obj

The last one comes from blocksort.obj, which we have yet to link in. After that, the code finally compiles:

{$LINK blocksort.obj}

This is, of course, just the first step toward a working program. Now we have to check how functions exported from these object files are actually defined. Then, we have to add appropriate external declarations to the Delphi source.

If we look at the previous example, LzmaDecTest, we can implement functions to initialize and free the decoder by adding the following two lines:

procedure LzmaDec_Init(var state); cdecl; external;
procedure LzmaDec_Free(var state; alloc: pointer); cdecl; external;

This declaration is actually incomplete, as the first parameter to both functions should be a complicated record, which I didn't want to try to decipher just for this simple demo. Rather than do that (and a whole lotta additional work), I would use the already-prepared Abbrevia library.

By now, you've probably noticed that linking in object files is hard work. If at all possible, you should try to find a Delphi (or at least a Free Pascal) wrapper. Even if it covers only a part of the functionality you need, it will spare you hours and hours of work.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.147.61.3