One of the most popular anti-debugging tricks is to use SEH to pass code execution. It is popular trick used in Windows computer viruses.  But before we discuss how this trick is used for anti-debugging, let us discuss how SEH works a little.

Exceptions are usually triggered from errors, such as reading bytes from inaccessible memory regions, or by something as simple as division by zero. They can also be triggered by debugger interrupts, INT 3 and INT 1. When an exception occurs, the system jumps right to the exception handler. Normally, the exception handler's job is to do something about the error.

Usually, this job gives an error message notification, leading to a graceful termination of the program. In programming terms, this is try-except or try-catch handling. The following is an example of exception handling in Python programming:

try:
    print("Hello World!")
    
except:
    print("Hello Error!")

An SEH record contains two elements: the address of the exception handler and the address of the next SEH record. The next SEH record contains the address of the SEH record next to it. Overall, the SEH records are chained to each other. This is called the SEH chain. If the current handler was not able to handle the exception, then the next handler takes over. A program crash can happen if ever the SEH records were exhausted. This process is shown here:

As we can see, the last SEH record contains a -1 (0xFFFFFFFF for 32-bit address space) value at the SEH record pointer field.

Now that we know how SEH works, how can this be abused for anti-debugging? Using our try-except Python code, abusing it would look something like this:

x = 1
try:
    x = x / 0
    print("This message will not show up!")
except:
    print("Hello World!")

What we did was force an error (a division-by-zero error, to be precise) to cause an exception. The exception handler displays the Hello World! message. But how does it work in x86 assembly language?

To set up our new SEH, we need to first identify where the current SEH is. For every process, there is an SEH chain set up by the Windows OS. The current SEH record can be retrieved from offset 0 of TIB, as denoted by the FS segment register.

The following assembly code retrieves the address of the current SEH record to the eax register:

mov eax, dword ptr FS:[0]

To change the handler, we can simply change the address of the current SEH record at FS:[0] with our SEH record. Let's assume that the handling code's address will be at 0x00401000, and that the current SEH record, is located at 0x00200000 has these values in it:

The next thing to do is build our SEH record, which we can store in the stack. With FS:[0] returning the 0x00200000 value, and our handler located at 0x00401000, here's a way to build the SEH record from the stack:

push 0x00401000
push dword ptr FS:[0]

The stack should look like something like this:

All we need to do is update the value of FS:[0] to the address of this SEH record, which is the register ESP register (that is, top of the stack):

mov dword ptr FS:[0], esp

The preceding code should add our SEH to the SEH chain.