Throughout the course of the book, we have used phrases like “IDA does this,” and “IDA does that.” While IDA certainly does an awful lot for us, the intelligence is more correctly attributed to the various modules upon which IDA relies. For example, it is the processor module that makes all of the decisions during the analysis phase, so one could argue that IDA is only as smart as the processor modules on which it relies. Of course, Hex-Rays puts tremendous effort into ensuring that its processor modules are as capable as possible, and for the casual user, IDA neatly hides its modular architecture beneath its user interface.
At some point you may find yourself needing more power than the IDC scripting language has to offer, whether for performance reasons or because you wish to do things that IDC simply was not designed to do. When that moment arrives, it is time to advance to using IDA’s software development kit (SDK) to build your own compiled modules for use with IDA.
Note
The IDC scripting engine is built on top of IDA’s SDK. All IDC functions are ultimately translated to calls to one or more SDK functions that perform the actual work. While it is true that if you can do something in IDC, you can do the same thing using the SDK, the reverse does not hold. The SDK offers far more power than is available using IDC alone, and many SDK actions have no IDC counterpart.
The SDK exposes IDA’s internal programming interfaces in the form of C++ libraries and the header files required to interface to those libraries. The SDK is required in order to create loader modules to handle new file formats, processor modules to disassemble new CPU instruction sets, and plug-in modules that might be viewed as more powerful, compiled alternatives to scripts.
In this chapter we introduce some of the core capabilities of the SDK. You will find these capabilities useful whether you are creating plug-ins, loader modules, or processor modules. As each of these types of modules is covered individually in the following three chapters, the examples in this chapter are offered without attempting to supply a specific context in which they might be used.
IDA’s SDK is distributed in much the same manner as the other IDA extras that we have discussed so far. The Zip file containing the SDK can be found on your original IDA CD, or authorized users can download the SDK from the Hex-Rays website. Each version of the SDK is named for the version of IDA with which it is compatible (for example, idasdk61.zip goes with IDA version 6.1). The SDK features the same minimalist documentation typically found in other IDA-related tools, which in the case of the SDK means a top-level readme.txt file and additional README files for plug-ins, processor modules, and loaders.
The SDK defines the published programming interface that modules may use to interact with IDA. Prior to SDK version 4.9, it was not uncommon for these interfaces to change enough that a module that successfully compiled under SDK 4.8 might no longer compile under a newer SDK, such as version 4.9, without the need for changes. With the introduction of version 4.9 of the SDK, Hex-Rays chose to standardize the existing API, which means that not only would modules require no changes to compile successfully with newer versions of the SDK, but modules would also be binary compatible with newer versions of IDA. This means that module users need no longer wait for module authors to update their source code or make available updated binary versions of their modules each time a new version of IDA is released. It does not mean that existing API interfaces are completely frozen; Hex-Rays continues to introduce new features with each new version of the SDK (that is, each new SDK is a superset of its predecessor). Modules that make use of these newer features are typically not compatible with older versions of IDA or the SDK. That said, there have been occasions where, for various reasons, functions have been renamed or marked as obsolete. The SDK offers macros to allow or disallow the use of deprecated functions, making it easy to note when a function has been deprecated.
Prior to version 5.4, the Zip file containing the SDK does not contain a top-level directory. Because the SDK shares several subdirectory names with IDA, it is highly recommended that you create a dedicated SDK directory, such as idasdk53, and extract the SDK contents into that directory. This will make it much easier to distinguish SDK components from IDA components. Beginning with version 5.4, the IDA SDK is packaged within a top-level SDK directory, such as idasdk61, so this step is no longer needed. There is no requirement to install the SDK in a specific location relative to <IDADIR>. Regardless of where you choose to install your SDK, we will refer to the SDK directory generically as <SDKDIR> for the remainder of the book.
A basic understanding of the directory structure used within the SDK will be helpful, both in knowing where you might find documentation and in knowing where you can expect to find the modules that you build. A quick rundown of what you can expect to find in the SDK follows.
- bin directory
This directory is where the example build scripts save their compiled modules following a successful build. Installing a module involves copying the module from the appropriate subdirectory within bin to the appropriate subdirectory in <IDADIR>. Module installation will be covered in more detail in Chapter 17, Chapter 18, and Chapter 19. This directory also contains a postprocessing tool required for the creation of processor modules.
- etc directory
This directory contains source code for two utilities that are required to build some SDK modules. Compiled versions of these utilities are also included with the SDK.
- include directory
This directory contains the header files that define the interface to the IDA API. In short, every API data structure that you are allowed to use and every API function that you are allowed to call are declared in one of the header files in this directory. The SDK’s top-level readme.txt file contains an overview of some of the more commonly used header files in this directory. The files in this directory constitute the bulk of the documentation (as in “read the source”) for the SDK.
- ldr directory
This directory contains the source code and build scripts for several example loader modules. The README file for loaders is nothing more than a rundown of the contents of this directory.
- lib directory
This directory contains a number of subdirectories, which in turn contain the link libraries required to build various IDA modules. The subdirectories are named after the compiler with which they should be used. For example,
x86_win_vc_32(6.1 and later) orvc.w32(6.0 and earlier) contains the library to use with Visual Studio and 32-bit IDA on Windows, whilex64_mac_gcc_64(6.1 and later) orgcc64.mac64(6.0 and earlier) contains the library for use with 64-bit IDA on OSX platforms.- module directory
This directory contains the source code and build scripts for several example processor modules. The README file for processor modules is nothing more than a rundown of the contents of this directory.
- plug-ins directory
This directory contains the source code and build scripts for several example plug-in modules. The README file for plug-ins provides a high-level overview of the plug-in architecture.
- top-level directory
The top level of the SDK contains several make files used for building modules as well as the main readme.txt file for the SDK. Several additional install_xxx.txt files contain information regarding installation and configuration for various compilers (for example, install_visual.txt discusses Visual Studio configuration).
Keep in mind that documentation on using the SDK is sparse. For most developers, knowledge of the SDK has been derived through trial and error and extensive exploration of the contents of the SDK. You may have some luck posting questions to the Research & Resources forum on the Hex-Rays support forums, where other IDA users familiar with the SDK may answer them. An excellent third-party resource providing an introduction to the SDK and plug-in writing is Steve Micallef’s guide titled IDA Plug-in Writing in C/C++.[115]
One of the more frustrating aspects of using the SDK is not related to programming at all. Instead, you may find that it is relatively easy to code up a solution to a problem only to find that it is virtually impossible to successfully build your module. This is true because it can be difficult to support a wide variety of compilers with a single code base, and coding a solution is complicated by the fact that library file formats recognized by Windows compilers are often incompatible with one another.
All of the examples included with the SDK were created to be built using Borland tools. From install_make.txt we have the following quote from Ilfak:
WIN32 versions can be created only by Borland C++ CBuilder v4.0. Probably the old BCC v5.2 will work too, but I haven’t checked it.
That being said, other install_xxx files offer pointers on how to successfully build modules with other compilers. A few of the example modules contain files for building with Visual Studio (<SDKDIR>/plugins/vcsample, for example), while install_visual.txt offers a series of steps for properly configuring SDK projects using Visual C++ Express 2005.
In order to build modules using Unix-style tools, either on a Unix-style system such as Linux or using an environment such as MinGW, the SDK provides a script named idamake.pl that converts the Borland-style make files into Unix-style make files prior to initiating the build process. This process is discussed in install_linux.txt.
Note
The command-line build scripts provided with the SDK expect an environment variable named IDA to point to <SDKDIR> . You can set this globally for all scripts by editing <SDKDIR>/allmake.mak and <SDKDIR>/allmake.unx to set this variable or by adding an IDA environment variable to your global environment.
Steve Micallef’s guide also provides excellent instructions for configuring build environments for building plug-ins with various compilers. Our personal preference when building SDK modules for Windows versions of IDA is to use the MinGW tools gcc and make. The examples presented in Chapter 17, Chapter 18, and Chapter 19 include makefiles and Visual Studio project files that do not rely on any of the build scripts included with the SDK and that are easy to modify to suit the needs of your projects. Module-specific build configuration will also be discussed in each of these chapters.
[114] A blocking operation is an action that causes a program to come to a halt while it awaits completion of the action.