There are many ways that malware can transfer execution in addition to the jump and call instructions visible in IDA Pro. It’s important for a malware analyst to be able to figure out how malware could be inducing other code to run. The first and most common way to access code outside a single file is through the use of DLLs.
Dynamic link libraries (DLLs) are the current Windows way to use libraries to share code among multiple applications. A DLL is an executable file that does not run alone, but exports functions that can be used by other applications.
Static libraries were the standard prior to the use of DLLs, and static libraries still exist, but they are much less common. The main advantage of using DLLs over static libraries is that the memory used by the DLLs can be shared among running processes. For example, if a library is used by two different running processes, the code for the static library would take up twice as much memory, because it would be loaded into memory twice.
Another major advantage to using DLLs is that when distributing an executable, you can use DLLs that are known to be on the host Windows system without needing to redistribute them. This helps software developers and malware writers minimize the size of their software distributions.
DLLs are also a useful code-reuse mechanism. For example, large software companies will create DLLs with some functionality that is common to many of their applications. Then, when they distribute the applications, they distribute the main .exe and any DLLs that application uses. This allows them to maintain a single library of common code and distribute it only when needed.
Malware writers use DLLs in three ways:
To store malicious code
Sometimes, malware authors find it more advantageous to store malicious code in a DLL, rather than in an .exe file. Some malware attaches to other processes, but each process can contain only one .exe file. Malware sometimes uses DLLs to load itself into another process.
By using Windows DLLs
Nearly all malware uses the basic Windows DLLs found on every system. The Windows DLLs contain the functionality needed to interact with the OS. The way that a malicious program uses the Windows DLLs often offers tremendous insight to the malware analyst. The imports that you learned about in Chapter 1 and the functions covered throughout this chapter are all imported from the Windows DLLs. Throughout the balance of this chapter, we will continue to cover functions from specific DLLs and describe how malware uses them.
By using third-party DLLs
Malware can also use third-party DLLs to interact with other programs. When you see malware that imports functions from a third-party DLL, you can infer that it is interacting with that program to accomplish its goals. For example, it might use the Mozilla Firefox DLL to connect back to a server, rather than connecting directly through the Windows API. Malware might also be distributed with a customized DLL to use functionality from a library not already installed on the victim’s machine; for example, to use encryption functionality that is distributed as a DLL.
Under the hood, DLL files look almost exactly like .exe files. DLLs use the PE file format, and only a single flag indicates that the file is a DLL and not an .exe. DLLs often have more exports and generally fewer imports. Other than that, there’s no real difference between a DLL and an .exe.
The main DLL function is DllMain. It has no label and is
not an export in the DLL, but it is specified in the PE header as the file’s entry point. The
function is called to notify the DLL whenever a process loads or unloads the library, creates a new
thread, or finishes an existing thread. This notification allows the DLL to manage any per-process
or per-thread resources.
Most DLLs do not have per-thread resources, and they ignore calls to DLLMain that are caused by thread activity. However, if the DLL has
resources that must be managed per thread, then those resources can provide a hint to an analyst as
to the DLL’s purpose.
Malware can also execute code outside the current program by creating a new process or modifying an existing one. A process is a program being executed by Windows. Each process manages its own resources, such as open handles and memory. A process contains one or more threads that are executed by the CPU. Traditionally, malware has consisted of its own independent process, but newer malware more commonly executes its code as part of another process.
Windows uses processes as containers to manage resources and keep separate programs from interfering with each other. There are usually at least 20 to 30 processes running on a Windows system at any one time, all sharing the same resources, including the CPU, file system, memory, and hardware. It would be very difficult to write programs if each program needed to manage sharing resources with all the others. The OS allows all processes to access these resources without interfering with each other. Processes also contribute to stability by preventing errors or crashes in one program from affecting other programs.
One resource that’s particularly important for the OS to share among processes is the system memory. To accomplish this, each process is given a memory space that is separate from all other processes and that is a sum of memory addresses that the process can use.
When the process requires memory, the OS will allocate memory and give the process an address that it can use to access the memory. Processes can share memory addresses, and they often do. For example, if one process stores something at memory address 0x00400000, another can store something at that address, and the processes will not conflict. The addresses are the same, but the physical memory that stores the data is not the same.
Like mailing addresses, memory addresses are meaningful only in context. Just as the address 202 Main Street does not tell you a location unless you also have the ZIP code, the address 0x0040A010 does not tell where the data is stored unless you know the process. A malicious program that accesses memory address 0x0040A010 will affect only what is stored at that address for the process that contains the malicious code; other programs on the system that use that address will be unaffected.
The function most commonly used by malware to create a new process is CreateProcess. This function has many parameters, and the caller has a lot of control
over how it will be created. For example, malware could call this function to create a process to
execute its malicious code, in order to bypass host-based firewalls and other security mechanisms. Or it could create an instance of
Internet Explorer and then use that program to access malicious content.
Malware commonly uses CreateProcess to create a simple
remote shell with just a single function call. One of the parameters to the CreateProcess function, the STARTUPINFO struct,
includes a handle to the standard input, standard output, and standard error streams for a process.
A malicious program could set these values to a socket, so that when the program writes to standard
output, it is really writing to the socket, thereby allowing an attacker to execute a shell remotely
without running anything other than the call to CreateProcess.
Example 7-4 shows how CreateProcess could be used to create a simple remote shell. Prior to this snippet, code
would have opened a socket to a remote location. The handle to the socket is stored on the stack and
entered into the STARTUPINFO structure. Then CreateProcess is called, and all input and output for the process is
routed through the socket.
Example 7-4. Sample code using the CreateProcess call
004010DAmov eax, dword ptr [esp+58h+SocketHandle]004010DE lea edx, [esp+58h+StartupInfo] 004010E2 push ecx ; lpProcessInformation 004010E3 push edx ;lpStartupInfo004010E4 ❶mov [esp+60h+StartupInfo.hStdError], eax 004010E8 ❷mov [esp+60h+StartupInfo.hStdOutput], eax 004010EC ❸mov [esp+60h+StartupInfo.hStdInput], eax 004010F0 ❹mov eax,dword_403098004010F5 push 0 ; lpCurrentDirectory 004010F7 push 0 ; lpEnvironment 004010F9 push 0 ; dwCreationFlags 004010FB mov dword ptr [esp+6Ch+CommandLine], eax 004010FF push 1 ; bInheritHandles 00401101 push 0 ; lpThreadAttributes 00401103 lea eax, [esp+74h+CommandLine] 00401107 push 0 ; lpProcessAttributes 00401109 ❺push eax ; lpCommandLine 0040110A push 0 ; lpApplicationName 0040110C mov [esp+80h+StartupInfo.dwFlags], 101h 00401114 ❻call ds:CreateProcessA
In the first line of code, the stack variable SocketHandle
is placed into EAX. (The socket handle is initialized outside this function.) The lpStartupInfo structure for the process stores the standard output
❷, standard input ❸, and standard error ❶ that will be used
for the new process. The socket is placed into the lpStartupInfo
structure for all three values (❶, ❷, ❸). The access to
dword_403098 at ❹
contains the command line of the program to be executed, which is eventually pushed on the stack as
a parameter ❺. The call to CreateProcess at ❻ has 10 parameters, but all
except lpCommandLine, lpProcessInformation, and lpStartupInfo are either 0
or 1. (Some represent NULL values and others represent flags, but none are interesting for malware
analysis.)
The call to CreateProcess will create a new process
so that all input and output are redirected to a socket. To find the remote host, we would need to
determine where the socket is initialized (not included in Example 7-4). To discover which program will be run, we
would need to find the string stored at dword_403098 by
navigating to that address in IDA Pro.
Malware will often create a new process by storing one program inside another in the resource
section. In Chapter 1, we discuss how the resource section of the PE
file can store any file. Malware will sometimes store another executable in the resource section.
When the program runs, it will extract the additional executable from the PE header, write it to
disk, and then call CreateProcess to run the program. This is
also done with DLLs and other executable code. When this happens, you must open the program in the
Resource Hacker utility (discussed in Chapter 1) and save the
embedded executable file to disk in order to analyze it.
Processes are the container for execution, but threads are what the Windows OS executes. Threads are independent sequences of instructions that are executed by the CPU without waiting for other threads. A process contains one or more threads, which execute part of the code within a process. Threads within a process all share the same memory space, but each has its own processor registers and stack.
When one thread is running, it has complete control of the CPU, or the CPU core, and other threads cannot affect the state of the CPU or core. When a thread changes the value of a register in a CPU, it does not affect any other threads. Before an OS switches between threads, all values in the CPU are saved in a structure called the thread context. The OS then loads the thread context of a new thread into the CPU and executes the new thread.
Example 7-5 shows an example of accessing a local variable and pushing it on the stack.
Example 7-5. Accessing a local variable and pushing it on the stack
004010DE lea ❶edx, [esp+58h]
004010E2 push edxIn Example 7-5, the code at ❶ accesses a local variable (esp+58h) and stores it in EDX, and then pushes EDX onto the stack. Now, if another thread
were to run some code in between these two instructions, and that code modified EDX, the value of
EDX would be wrong, and the code would not execute properly. When thread-context switching is used,
if another thread runs in between these two instructions, the value of EDX is stored in the thread
context. When the thread starts again and executes the push
instruction, the thread context is restored, and EDX stores the proper value again. In this way, no
thread can interfere with the registers or flags from another thread.
The CreateThread function is used to create new
threads. The function’s caller specifies a start address, which is often called the start function. Execution begins at the start address and continues until
the function returns, although the function does not need to return, and the thread can run until
the process ends. When analyzing code that calls CreateThread,
you will need to analyze the start function in addition to
analyzing the rest of the code in the function that calls CreateThread.
The caller of CreateThread can specify the function where
the thread starts and a single parameter to be passed to the start function. The parameter can be any value, depending on the function where the
thread will start.
Malware can use CreateThread in multiple ways, such as the
following:
Malware can use
CreateThreadto load a new malicious library into a process, withCreateThreadcalled and the address ofLoadLibraryspecified as the start address. (The argument passed toCreateThreadis the name of the library to be loaded. The new DLL is loaded into memory in the process, andDllMainis called.)Malware can create two new threads for input and output: one to listen on a socket or pipe and then output that to standard input of a process, and the other to read from standard output and send that to a socket or pipe. The malware’s goal is to send all information to a single socket or pipe in order to communicate seamlessly with the running application.
Example 7-6 shows how to recognize the second technique
by identifying two CreateThread calls near each other. (Only the
system calls for ThreadFunction1 and ThreadFunction2 are shown.) This code calls CreateThread twice. The arguments are lpStartAddress
values, which tell us where to look for the code that will run when these threads start.
Example 7-6. Main function of thread example
004016EE lea eax, [ebp+ThreadId] 004016F4 push eax ; lpThreadId 004016F5 push 0 ; dwCreationFlags 004016F7 push 0 ; lpParameter 004016F9 push ❶offsetThreadFunction1; lpStartAddress 004016FE push 0 ; dwStackSize 00401700 lea ecx, [ebp+ThreadAttributes] 00401706 push ecx ; lpThreadAttributes 00401707 call ❷ds:CreateThread 0040170D mov [ebp+var_59C], eax 00401713 lea edx, [ebp+ThreadId] 00401719 push edx ; lpThreadId 0040171A push 0 ; dwCreationFlags 0040171C push 0 ; lpParameter 0040171E push ❸offsetThreadFunction2; lpStartAddress 00401723 push 0 ; dwStackSize 00401725 lea eax, [ebp+ThreadAttributes] 0040172B push eax ; lpThreadAttributes 0040172C call ❹ds:CreateThread
In Example 7-6, we have labeled the start function
ThreadFunction1
❶ for the first call to CreateThread
❷ and ThreadFunction2
❸ for the second call ❹. To determine the purpose of these two threads, we first navigate to ThreadFunction1. As shown in Example 7-7, the first thread function executes a loop in which it
calls ReadFile to read from a pipe, and then it forwards that
data out to a socket with the send function.
Example 7-7. ThreadFunction1 of thread example
... 004012C5 call ds:ReadFile ... 00401356 call ds:send ...
As shown in Example 7-8, the second thread function
executes a loop that calls recv to read any data sent over the
network, and then forwards that data to a pipe with the WriteFile
function, so that it can be read by the application.
One topic related to threads and processes is mutexes, referred to as mutants when in the kernel. Mutexes are global objects that coordinate multiple processes and threads.
Mutexes are mainly used to control access to shared resources, and are often used by malware. For example, if two threads must access a memory structure, but only one can safely access it at a time, a mutex can be used to control access.
Only one thread can own a mutex at a time. Mutexes are important to malware analysis because they often use hard-coded names, which make good host-based indicators. Hard-coded names are common because a mutex’s name must be consistent if it’s used by two processes that aren’t communicating in any other way.
The thread gains access to the mutex with a call to WaitForSingleObject, and any subsequent threads attempting to gain access to it must
wait. When a thread is finished using a mutex, it uses the ReleaseMutex function.
A mutex can be created with the CreateMutex function.
One process can get a handle to another process’s mutex by using the OpenMutex call. Malware will commonly create a mutex and attempt to open an existing
mutex with the same name to ensure that only one version of the malware is running at a time, as
demonstrated in Example 7-9.
Example 7-9. Using a mutex to ensure that only one copy of malware is running on a system
00401000 push offset Name ; "HGL345" 00401005 push 0 ; bInheritHandle 00401007 push 1F0001h ; dwDesiredAccess 0040100C ❶call ds:__imp__OpenMutexW@12 ; OpenMutexW(x,x,x) 00401012 ❷test eax, eax 00401014 ❸jz short loc_40101E 00401016 push 0 ; int 00401018 ❹call ds:__imp__exit 0040101E push offset Name ; "HGL345" 00401023 push 0 ; bInitialOwner 00401025 push 0 ; lpMutexAttributes 00401027 ❺call ds:__imp__CreateMutexW@12 ; CreateMutexW(x,x,x)
The code in Example 7-9 uses the hard-coded
name HGL345 for the mutex. It first checks to see if there is a
mutex named HGL345 using the OpenMutex call at ❶. If the return value is
NULL at ❷, it jumps (at ❸) over the exit call and continues to execute. If
the return value is not NULL, it calls exit at ❹, and the process will exit. If the code continues to execute,
the mutex is created at ❺ to ensure that additional
instances of the program will exit when they reach this code.
Another way for malware to execute additional code is by installing it as a service. Windows allows tasks to run without their own processes or threads by using services that run as background applications; code is scheduled and run by the Windows service manager without user input. At any given time on a Windows OS, several services are running.
Using services has many advantages for the malware writer. One is that services are normally
run as SYSTEM or another privileged account. This is not a
vulnerability because you need administrative access in order to install a service, but it is
convenient for malware writers, because the SYSTEM account has
more access than administrator or user accounts.
Services also provide another way to maintain persistence on a system, because they can be set to run automatically when the OS starts, and may not even show up in the Task Manager as a process. A user searching through running applications wouldn’t find anything suspicious, because the malware isn’t running in a separate process.
Note
It is possible to list running services using net
start at the command line, but doing so will display only the names of running services.
Programs, such as the Autoruns tool mentioned earlier, can be used to gather more information about
running services.
Services can be installed and manipulated via a few Windows API functions, which are prime targets for malware. There are several key functions to look for:
OpenSCManager. Returns a handle to the service control manager, which is used for all subsequent service-related function calls. All code that will interact with services will call this function.CreateService. Adds a new service to the service control manager, and allows the caller to specify whether the service will start automatically at boot time or must be started manually.StartService. Starts a service, and is used only if the service is set to be started manually.
The Windows OS supports several different service types, which execute in unique ways. The one
most commonly used by malware is the WIN32_SHARE_PROCESS type,
which stores the code for the service in a DLL, and combines several different services in a single,
shared process. In Task Manager, you can find several instances of a process called
svchost.exe, which are running WIN32_SHARE_PROCESS-type services.
The WIN32_OWN_PROCESS type is also used because it stores
the code in an .exe file and runs as an independent process.
The final common service type is KERNEL_DRIVER, which is
used for loading code into the kernel. (We discuss malware running in the kernel later in this
chapter and extensively in Chapter 10.)
The information about services on a local system is stored in the registry. Each service has a
subkey under HKLMSYSTEMCurrentControlSetServices. For example,
Figure 7-2 shows the registry entries for HKLMSYSTEMCurrentControlSetServicesVMware NAT Service.
The code for the VMware NAT service is stored at
C:Windowssystem32vmnat.exe
❶. The type value of 0x10
❸ corresponds to WIN32_OWN_PROCESS, and the start value of 0x02
❷ corresponds to AUTO_START.
The SC program is a command-line tool included with Windows that you can use to investigate
and manipulate services. It includes commands for adding, deleting, starting, stopping, and querying
services. For example, the qc command queries a service’s configuration
options by accessing the same information as the registry entry shown in Figure 7-2 in a more readable way. Example 7-10 shows the SC program in action.
Example 7-10. The query configuration information command of the SC program
C:UsersUser1>sc qc "VMware NAT Service"
[SC] QueryServiceConfig SUCCESS
SERVICE_NAME: VMware NAT Service
TYPE : 10 ❶WIN32_OWN_PROCESS
START_TYPE : 2 AUTO_START
ERROR_CONTROL : 1 NORMAL
BINARY_PATH_NAME : C:Windowssystem32vmnat.exe
LOAD_ORDER_GROUP :
TAG : 0
DISPLAY_NAME : VMware NAT Service
DEPENDENCIES : VMnetuserif
SERVICE_START_NAME : LocalSystemExample 7-10 shows the query configuration
information command. This information is identical to what was stored in the registry for the VMware
NAT service, but it is easier to read because the numeric values have meaningful labels such as
WIN32_OWN_PROCESS
❶. The SC program has many different commands, and
running SC without any parameters will result in a list of the possible commands. (For more about
malware that runs as a service, see Chapter 11.)
The Microsoft Component Object Model (COM) is an interface standard that makes it possible for different software components to call each other’s code without knowledge of specifics about each other. When analyzing malware that uses COM, you’ll need to be able to determine which code will be run as a result of a COM function call.
COM works with any programming language and was designed to support reusable software components that could be utilized by all programs. COM uses an object construct that works well with object-oriented programming languages, but COM does not work exclusively with object-oriented programming languages.
Since it’s so versatile, COM is pervasive within the underlying OS and within most Microsoft applications. Occasionally, COM is also used in third-party applications. Malware that uses COM functionality can be difficult to analyze, but you can use the analysis techniques presented in this section.
COM is implemented as a client/server framework. The clients are the programs that are making use of COM objects, and the servers are the reusable software components—the COM objects themselves. Microsoft provides a large number of COM objects for programs to use.
Each thread that uses COM must call the OleInitialize or
CoInitializeEx function at least once prior to calling any other
COM library functions. So, a malware analyst can search for these calls to determine whether a program is using COM
functionality. However, knowing that a piece of malware uses a COM object as a client does not
provide much information, because COM objects are diverse and widespread. Once you determine that a
program uses COM, you’ll need to find a couple of identifiers of the object being used to
continue analysis.
COM objects are accessed via their globally unique identifiers (GUIDs) known as class identifiers (CLSIDs) and interface identifiers (IIDs).
The CoCreateInstance function is used to get access to COM
functionality. One common function used by malware is Navigate,
which allows a program to launch Internet Explorer and access a web address. The Navigate function is part of the IWebBrowser2 interface, which specifies a list of functions that must be implemented, but
does not specify which program will provide that functionality. The program that provides the
functionality is the COM class that implements the IWebBrowser2 interface. In most cases, the IWebBrowser2 interface is implemented by Internet Explorer. Interfaces are identified
with a GUID called an IID, and classes are identified with a GUID called a CLSID.
Consider an example piece of malware that uses the Navigate
function from the IWebBrowser2 interface implemented by Internet
Explorer. The malware first calls the CoCreateInstance function.
The function accepts the CLSID and the IID of the object that the malware is requesting. The OS then
searches for the class information, and loads the program that will perform the functionality, if it
isn’t already running. The CoCreateInstance class returns a
pointer that points to a structure that contains function pointers. To use the functionality of the
COM server, the malware will call a function whose pointer is stored in the structure returned from
CoCreateInstance. Example 7-11 shows how some code gets access to an IWebBrowser2 object.
Example 7-11. Accessing a COM object with CoCreateInstance
00401024 lea eax, [esp+18h+PointerToComObject] 00401028 push eax ; ppv 00401029 push ❶offset IID_IWebBrowser2 ; riid 0040102E push 4 ; dwClsContext 00401030 push 0 ; pUnkOuter 00401032 push ❷offset stru_40211C ; rclsid 00401037 call CoCreateInstance
In order to understand the code, click the structures that store the IID and CLSID at
❶ and ❷. The
code specifies the IID D30C1661-CDAF-11D0-8A3E-00C04FC9E26E, which represents the IWebBrowser2 interface, and the CLSID 0002DF01-0000-0000-C000-000000000046, which represents Internet Explorer. IDA Pro can
recognize and label the IID for IWebBrowser2, since it’s
commonly used. Software developers can create their own IIDs, so IDA Pro can’t always label
the IID used by a program, and it is never able to label the CLSID, because disassembly
doesn’t contain the necessary information.
When a program calls CoCreateInstance, the OS uses
information in the registry to determine which file contains the requested COM code. The HKLMSOFTWAREClassesCLSID and HKCUSOFTWAREClassesCLSID registry keys store the information about which code to
execute for the COM server. The value of C:Program FilesInternet
Exploreriexplore.exe, stored in the LocalServer32
subkey of the registry key HKLMSOFTWAREClassesCLSID 002DF01-0000-0000-C000-000000000046, identifies the executable that will be loaded when CoCreateInstance is called.
Once the structure is returned from the CoCreateInstance
call, the COM client calls a function whose location is stored at an offset in the structure. Example 7-12 shows the call. The reference to the COM object is stored on the
stack, and then moved into EAX. Then the first value in the structure points to a table of function
pointers. At an offset of 0x2C in the table is the Navigate function that is called.
Example 7-12. Calling a COM function
0040105E push ecx
0040105F push ecx
00401060 push ecx
00401061 mov esi, eax
00401063 mov eax, [esp+24h+PointerToComObject]
00401067 mov edx, [eax]
00401069 mov edx, [edx+❶2Ch]
0040106C push ecx
0040106D push esi
0040106E push eax
0040106F call edxIn order to identify what a malicious program is doing when it calls a COM function, malware
analysts must determine which offset a function is stored at, which can be tricky. IDA Pro stores
the offsets and structures for common interfaces, which can be explored via the structure subview.
Press the INSERT key to add a structure, and then click
Add Standard Structure. The name of the structure to add is
InterfaceNameVtbl. In our Navigate example, we add the IWebBrowser2Vtbl structure. Once the structure is added, right-click the
offset at ❶ in the disassembly to change the label from
2Ch to the function name IwebBrowser2Vtbl.Navigate. Now IDA Pro will add comments to the call instruction and the parameters being pushed onto the stack.
For functions not available in IDA Pro, one strategy for identifying the function called by a
COM client is to check the header files for the interface specified in the call to CoCreateInstance. The header files are included with Microsoft Visual
Studio and the platform SDK, and can also be found on the Internet. The functions are usually
declared in the same order in the header file and in the function table. For example, the Navigate function is the twelfth function in the .h
file, which corresponds to an offset of 0x2C. The first function
is at 0, and each function takes up 4 bytes.
In the previous example, Internet Explorer was loaded as its own process when CoCreateInstance was called, but this is not always the case. Some COM
objects are implemented as DLLs that are loaded into the process space of the COM client
executable. When the COM object is set up to be loaded as a DLL, the registry entry for the CLSID
will include the subkey InprocServer32, rather than LocalServer32.
Some malware implements a malicious COM server, which is subsequently used by other applications. Common COM server functionality for malware is through Browser Helper Objects (BHOs), which are third-party plug-ins for Internet Explorer. BHOs have no restrictions, so malware authors use them to run code running inside the Internet Explorer process, which allows them to monitor Internet traffic, track browser usage, and communicate with the Internet, without running their own process.
Malware that implements a COM server is usually easy to detect because it exports several
functions, including DllCanUnloadNow, DllGetClassObject, DllInstall, DllRegisterServer, and DllUnregisterServer, which all must be exported by COM servers.
Exceptions allow a program to handle events outside the flow of normal execution. Most of the
time, exceptions are caused by errors, such as division by zero. When an exception occurs, execution
transfers to a special routine that resolves the exception. Some exceptions, such as division by
zero, are raised by hardware; others, such as an invalid memory access, are raised by the OS. You
can also raise an exception explicitly in code with the RaiseException call.
Structured Exception Handling (SEH) is the Windows mechanism for handling exceptions. In 32-bit systems, SEH information is stored on the stack. Example 7-13 shows disassembly for the first few lines of a function that has exception handling.
Example 7-13. Storing exception-handling information in fs:0
01006170 push ❶offset loc_10061C0 01006175 mov eax, large fs:0 0100617B push ❷eax 0100617C mov large fs:0, esp
At the beginning of the function, an exception-handling frame is put onto the stack at
❶. The special location fs:0 points to an address on the stack that stores the exception information. On the
stack is the location of an exception handler, as well as the exception handler used by the caller
at ❷, which is restored at the end of the function. When
an exception occurs, Windows looks in fs:0 for the stack location
that stores the exception information, and then the exception handler is called. After the exception
is handled, execution returns to the main thread.
Exception handlers are nested, and not all handlers respond to all exceptions. If the exception handler for the current frame does not handle an exception, it’s passed to the exception handler for the caller’s frame. Eventually, if none of the exception handlers responds to an exception, the top-level exception handler crashes the application.
Exception handlers can be used in exploit code to gain execution. A pointer to exception-handling information is stored on the stack, and during a stack overflow, an attacker can overwrite the pointer. By specifying a new exception handler, the attacker gains execution when an exception occurs. Exceptions will be covered in more depth in the debugging and anti-debugging chapters (Chapter 8–Chapter 10, Chapter 15, and Chapter 16).