Like all software, debuggers contain vulnerabilities, and sometimes malware authors attack them in order to prevent debugging. Here, we present several popular vulnerabilities in the way OllyDbg handles the PE format.
The first technique modifies the Microsoft PE header of a binary executable, causing OllyDbg to crash when loading the executable. The result is an error of “Bad or Unknown 32-bit Executable File,” yet the program usually runs fine outside the debugger.
This issue is due to the fact that OllyDbg follows the Microsoft specifications regarding the
PE header too strictly. In the PE header, there is typically a structure known as the IMAGE_OPTIONAL_HEADER. Figure 16-5 shows a subset of this structure.
The last several elements in this structure are of particular interest. The NumberOfRvaAndSizes field identifies the number of entries in the DataDirectory array that follows. The DataDirectory array indicates where to find other important executable components in the
file; it is little more than an array of IMAGE_DATA_DIRECTORY
structures at the end of the optional header structure. Each data directory structure specifies the
size and relative virtual address of the directory.
The size of the array is set to IMAGE_NUMBEROF_DIRECTORY_ENTRIES, which is equal to 0x10. The Windows loader ignores any NumberOfRvaAndSizes greater than 0x10, because
anything larger will not fit in the DataDirectory array. OllyDbg
follows the standard and uses NumberOfRvaAndSizes no matter what.
As a consequence, setting the size of the array to a value greater than 0x10 (like 0x99) will cause OllyDbg to generate a
pop-up window to the user before exiting the program.
The easiest way to overcome this technique is to manually modify the PE header and set the
NumberOfRvaAndSizes to 0x10
using a hex editor or PE Explorer. Or, of course, you can use a debugger that is not vulnerable to
this technique, such as WinDbg or OllyDbg 2.0.
Another PE header trick involves section headers, causing OllyDbg to crash during loading with
the error “File contains too much data.” (WinDbg and OllyDbg 2.0 are not vulnerable to
this technique.) Sections contain the content of the file, including code, data, resources, and
other information. Each section has a header in the form of an IMAGE_SECTION_HEADER structure. Figure 16-6 shows a subset of this structure.
The elements of interest are VirtualSize and the
SizeOfRawData. According to the Windows PE specification,
VirtualSize should contain the total size of the section when
loaded into memory, and SizeOfRawData should contain the size of
data on disk. The Windows loader uses the smaller of VirtualSize
and SizeOfRawData to map the section data into memory. If the
SizeOfRawData is larger than VirtualSize, only VirtualSize data is copied into
memory; the rest is ignored. Because OllyDbg uses only the SizeOfRawData, setting the SizeofRawData to something
large like 0x77777777, will cause OllyDbg to crash.
The easiest way to overcome this anti-debugging technique is to manually modify the PE header
and set the SizeOfRawData using a hex editor to change the value
to be close to VirtualSize. (Note that, according to the
specification, this value must be a multiple of the FileAlignment
value from the IMAGE_OPTIONAL_HEADER). PE Explorer is a great
program to use for this purpose because it is not fooled by a large value for SizeofRawData.
Malware often attempts to exploit a format string vulnerability in version 1.1 of OllyDbg, by
providing a string of %s as a parameter to OutputDebugString to cause OllyDbg to crash. Beware of suspicious calls
like OutputDebugString ("%s%s%s%s%s%s%s%s%s%s%s%s%s%s"). If this
call executes, your debugger will crash.