IAT hooking relies on swapping the function pointers, whereas, in inline hooking, the API function itself is modified (patched) to redirect the API to the malicious code. As in IAT hooking, this technique allows the attacker to intercept, monitor, and block calls made by a specific application, and filter output parameters. In inline hooking, the target API function's first few bytes (instructions) are usually overwritten with a jump statement that re routes the program control to the malicious code. The malicious code can then intercept the input parameters, filter output, and redirect the control back to the original function.

To help you understand, let's suppose that an attacker wants to hook the DeleteFileA() function call made by a legitimate application. Normally, when the legitimate application's thread encounters the call to DeleteFileA(), the thread starts executing from the start of the DeleteFileA() function, as shown here:

To replace the first few instructions of a function with a jump, the malware needs to choose which instructions to replace. The jmp instruction requires at least 5 bytes, so the malware needs to choose instructions that occupy 5 bytes or more. In the preceding diagram, it is safe to replace the first three instructions (highlighted using a different color), because they take up exactly 5 bytes, and also, these instructions do not do much, apart from setting up the stack frame. The three instructions to be replaced in DeleteFileA() are copied, and then replaced with a jump statement of some sort, which transfers control to the malicious function. The malicious function does what it wants to do, and then executes the original three instructions of DeleteFileA() and jumps back to the address that lies below the patch (below the jump instruction), as shown in the following diagram. The replaced instructions, along with the jump statement that returns to the target function, are known as the trampoline:

This technique can be detected by looking for unexpected jump instructions at the start of the API function, but be aware that malware can make detection difficult by inserting the jump deeper in the API function, rather than at the start of the function. Instead of using a jmp instruction, malware may use a call instruction, or a combination of push and ret instructions, to redirect control; this technique bypasses the security tools, which only look for jmp instructions.

With an understanding of inline hooking, let's take a look at an example of malware (Zeus Bot) using this technique. Zeus bot hooks various API functions; one of them is the HttpSendRequestA() in Internet Explorer (iexplore.exe). By hooking this function, the malware can extract credentials from the POST payload. Before hooking, the malicious executable (containing various functions) is injected into the address space of Internet Explorer. The following screenshot shows the address 0x33D0000, where the executable is injected:

After injecting the executable, HttpSendRequestA() is hooked to redirect the program control to one of the malicious functions within the injected executable. Before we look at the hooked function, let's look at the first few bytes of the legitimate HttpSendRequestA() function (shown here):

The first three instructions (occupying 5 bytes, highlighted in the preceding screenshot) are replaced to redirect control. The following screenshot shows the HttpSendRequestA() after hooking. The first three instructions are replaced with the jmp instruction (occupying 5 bytes); note how the jump instruction redirects control to the malicious code at the address 0x33DEC48, which falls within the address range of the injected executable: