All of the IDA scripting capabilities discussed in Chapter 15 continue to be accessible when you are using the IDA debugger. Scripts may be launched from the File menu, associated with hotkeys, and invoked from the IDA scripting command line. In addition, user-created IDC functions may be referenced from breakpoint conditions and tracing termination expressions.

Basic scripting functions offer the capability to set, modify, and enumerate breakpoints and the ability to read and write register and memory values. Memory access is provided by the DbgByte, PatchDbgByte, DbgWord, PatchDbgWord, DbgDword, and PatchDbgDword functions (analogous to the Byte, Word, Dword, and PatchXXX functions described in Chapter 15). Register and breakpoint manipulation is made possible by the following functions (please see the IDA help file for a complete list).

The following script demonstrates how to install a custom IDC breakpoint-handling function at the current cursor location:

#include <idc.idc>
/*
 * The following should return 1 to break, and 0 to continue execution.
 */
static my_breakpoint_condition() {
   return AskYN(1, "my_breakpoint_condition activated, break now?") == 1;
}

/*
 * This function is required to register my_breakpoint_condition
 * as a breakpoint conditional expression
 */
static main() {
   auto addr;
   addr = ScreenEA();
   AddBpt(addr);
   SetBptCnd(addr, "my_breakpoint_condition()");
}

The complexity of my_breakpoint_condition is entirely up to you. In this example, each time the breakpoint is hit, a dialog will be displayed asking the user if she would like to continue execution of the process or pause at the current location. The value returned by my_breakpoint_condition is used by the debugger to determine whether the breakpoint should be honored or ignored.

Programmatic control of the debugger is possible from both the SDK and through the use of scripts. Within the SDK, IDA utilizes an event-driven model and provides callback notifications to plug-ins when specific debugger events occur. Unfortunately, IDA’s scripting capabilities don’t facilitate the use of an event-driven paradigm within scripts. As a result, Hex-Rays introduced a number of scripting functions that allow for synchronous control of the debugger from within scripts. The basic approach required to drive the debugger using a script is to initiate a debugger action and then wait for the corresponding debugger event code. Keep in mind that a call to a synchronous debugger function (which is all you can do in a script) blocks all other IDA operations until the call completes. The following list details several of the debugging extensions available for scripts:

GetDebuggerEvent must be called after each function that causes the process to execute in order to retrieve the debugger’s event code. Failure to do so may prevent follow-up attempts to step or run the process. For example, the following code fragment will step the debugger only one time because GetDebuggerEvent does not get called to clear the last event type in between invocations of StepOver.

StepOver();
StepOver();    //this and all subsequent calls will fail
StepOver();
StepOver();

The proper way to perform an execution action is to follow up each call with a call to GetDebuggerEvent, as shown in the following example:

StepOver();
GetDebuggerEvent(WFNE_SUSP, −1);
StepOver();
GetDebuggerEvent(WFNE_SUSP, −1);
StepOver();
GetDebuggerEvent(WFNE_SUSP, −1);
StepOver();
GetDebuggerEvent(WFNE_SUSP, −1);

The calls to GetDebuggerEvent allow execution to continue even if you choose to ignore the return value from GetDebuggerEvent. The event type WFNE_SUSP indicates that we wish to wait for an event that results in suspension of the debugged process, such as an exception or a breakpoint. You may have noticed that there is no function that simply resumes execution of a suspended process.^[225] However, it is possible to achieve the same effect by using the WFNE_CONT flag in a call to GetDebuggerEvent, as shown here:

GetDebuggerEvent(WFNE_SUSP | WFNE_CONT, −1);

This particular call waits for the next available suspend event after first resuming execution by continuing the process from the current instruction.

Additional functions are provided for automatically launching the debugger and attaching to running processes. See IDA’s help file for more information on these functions.

An example of a simple debugger script for collecting statistics on the addresses of each executed instruction (provided the debugger is enabled) is shown here:

static main() {
     auto ca, code, addr, count, idx;
    ca = GetArrayId("stats");
     if (ca != −1) {
        DeleteArray(ca);
     }
     ca = CreateArray("stats");
    EnableTracing(TRACE_STEP, 1);
    for (code = GetDebuggerEvent(WFNE_ANY | WFNE_CONT, −1); code > 0;
            code = GetDebuggerEvent(WFNE_ANY | WFNE_CONT, −1)) {
       addr = GetEventEa();
       count = GetArrayElement(AR_LONG, ca, addr) + 1;
       SetArrayLong(ca, addr, count);
     }
     EnableTracing(TRACE_STEP, 0);
    for (idx = GetFirstIndex(AR_LONG, ca);
            idx != BADADDR;
            idx = GetNextIndex(AR_LONG, ca, idx)) {
        count = GetArrayElement(AR_LONG, ca, idx);
        Message("%x: %d
", idx, count);
     }
    DeleteArray(ca);
  }

The script begins by testing for the presence of a global array named stats. If one is found, the array is removed and re-created so that we can start with an empty array. Next , single-step tracing is enabled before entering a loop to drive the single-stepping process. Each time a debug event is generated, the address of the associated event is retrieved , the current count for the associated address is retrieved from the global array and incremented , and the array is updated with the new count . Note that the instruction pointer is used as the index into the sparse global array, which saves time looking up the address in some other form of data structure. Once the process completes, a second loop is used to retrieve and print all values from array locations that have valid values. In this case, the only array indexes that will have valid values represent addresses from which instructions were fetched. The script finishes off by deleting the global array that was used to gather the statistics. Example output from this script is shown here:

401028: 1
40102b: 1
40102e: 2
401031: 2
401034: 2
401036: 1
40103b: 1

A slight alteration of the preceding example can be used to gather statistics on what types of instructions are executed during the lifetime of a process. The following example shows the modifications required in the first loop to gather instruction-type data rather than address data:

for (code = GetDebuggerEvent(WFNE_ANY | WFNE_CONT, −1); code > 0;
          code = GetDebuggerEvent(WFNE_ANY | WFNE_CONT, −1)) {
        addr = GetEventEa();
       mnem = GetMnem(addr);
       count = GetHashLong(ht, mnem) + 1;
       SetHashLong(ht, mnem, count);
     }

Rather than attempting to classify individual opcodes, we choose to group instructions by mnemonics . Because mnemonics are strings, we make use of the hash-table feature of global arrays to retrieve the current count associated with a given mnemonic and save the updated count back into the correct hash table entry. Sample output from this modified script is shown here:

add:   18
and:   2
call:  46
cmp:   16
dec:   1
imul:  2
jge:   2
jmp:   5
jnz:   7
js:    1
jz:    5
lea:   4
mov:   56
pop:   25
push:  59
retn:  19
sar:   2
setnz: 3
test:  3
xor:   7

In Chapter 25 we will revisit the use of debugger-interaction capabilities as a means to assist in de-obfuscating binaries.

In Chapter 16 you learned that IDA’s SDK offers significant power for developing a variety of compiled extensions that can be integrated into IDA and that have complete access to the IDA API. The IDA API offers a superset of all the capabilities available in IDC, and the debugging extensions are no exception. Debugger extensions to the API are declared in <SDKDIR>/dbg.hpp and include C++ counterparts to all of the IDC functions discussed thus far, along with a complete asynchronous debugger interface capability.

For asynchronous interaction, plug-ins gain access to debugger notifications by hooking the HT_DBG notification type (see loader.hpp). Debugger notifications are declared in the dbg_notification_t enum found in dbg.hpp.

Within the debugger API, commands for interacting with the debugger are typically defined in pairs, with one function used for synchronous interaction (as with scripts) and the second function used for asynchronous interaction. Generically, the synchronous form of a function is named COMMAND(), and its asynchronous counterpart is named request_COMMAND(). The request_XXX versions are used to queue debugger actions for later processing. Once you finish queuing asynchronous requests, you must invoke the run_requests function to initiate processing of your request queue. As your requests are processed, debugger notifications will be delivered to any callback functions that you may have registered via hook_to_notification_point.

Using asynchronous notifications, we can develop an asynchronous version of the address-counting script from the previous section. The first task is to make sure that we hook and unhook debugger notifications. We will do this in the plug-in’s init and term methods, as shown here:

//A netnode to gather stats into
 netnode stats("$ stats", 0, true);

int idaapi init(void) {
   hook_to_notification_point(HT_DBG, dbg_hook, NULL);
   return PLUGIN_KEEP;
}

void idaapi term(void) {
   unhook_from_notification_point(HT_DBG, dbg_hook, NULL);
}

Note that we have also elected to declare a global netnode , which we will use to collect statistics. Next we consider what we want the plug-in to do when it is activated via its assigned hotkey. Our example plug-in run function is shown here:

void idaapi run(int arg) {
     stats.altdel();   //clear any existing stats
    request_enable_step_trace();
    request_step_until_ret();
    run_requests();
  }

Since we are using asynchronous techniques in this example, we must first submit a request to enable step tracing and then submit a request to resume execution of the process being debugged. For the sake of simplicity, we will gather statistics on the current function only, so we will issue a request to run until the current function returns . With our requests properly queued, we kick things off by invoking run_requests to process the current request queue .

All that remains is to process the notifications that we expect to receive by creating our HT_DBG callback function. A simple callback that processes only two messages is shown here:

int idaapi dbg_hook(void *user_data, int notification_code, va_list va) {
     switch (notification_code) {
       case dbg_trace:  //notification arguments are detailed in dbg.hpp
           va_arg(va, thid_t);
          ea_t ea = va_arg(va, ea_t);
           //increment the count for this address
          stats.altset(ea, stats.altval(ea) + 1);
           return 0;
       case dbg_step_until_ret:
           //print results
          for
(nodeidx_t i = stats.alt1st(); i != BADNODE; i = stats.altnxt(i)) {
               msg("%x: %d
", i, stats.altval(i));
           }
           //delete the netnode and stop tracing
          stats.kill();
          request_disable_step_trace();
          run_requests();
           break;
     }
  }

The dbg_trace notification will be received for each instruction that executes until we turn tracing off. When a trace notification is received, the address of the trace point is retrieved from the args list and then used to update the appropriate netnode array index . The dbg_step_until_ret notification is sent once the process hits the return statement to leave the function in which we started. This notification is our signal that we should stop tracing and print any statistics we have gathered. A loop is used to iterate through all valid index values of the stats netnode before destroying the netnode and requesting that step tracing be disabled . Since this example uses asynchronous commands, the request to disable tracing is added to the queue, which means we have to issue run_requests in order for the queue to be processed. An important warning about synchronous versus asynchronous interaction with the debugger is that you should never call the synchronous version of a function while actively processing an asynchronous notification message.

Synchronous interaction with the debugger using the SDK is done in a manner very similar to scripting the debugger. As with many of the SDK functions we have seen in previous chapters, the names of debugger-related functions typically do not match the names of related scripting functions, so you may need to spend some time combing through dbg.hpp in order to find the functions you are looking for. The biggest disparity in names between scripting and the SDK is the SDK’s version of GetDebuggerEvent, which is called wait_for_next_event in the SDK. The other major difference between script functions and the SDK is that variables corresponding to the CPU registers are not automatically declared for you within the SDK. In order to access the values of CPU registers from the SDK, you must use the get_reg_val and set_reg_val functions to read and write registers, respectively.