In this section, we will learn about assembly language. Our objective is to take C code, translate it to an assembly language, and take a look at what happens.

The following is the sample C code that we will be loading and using in order to learn about assembly language:

We will run this piece of code in the immunity debugger, compiling it to a file called Bufferoverflow.exe. Let's start by opening it with the immunity debugger:

Note that at the top right, we have a Registers section. The first register, EAX, is the accumulator. In a computer's CPU, an accumulator is a register in which the intermediate arithmetic and the logic results are stored. In the top left, we have the actual assembly code, while in the bottom left, we get the memory dump used by the program. The bottom right contains the stack area of the program that we are inspecting.

If we scroll down to position 00401290, we can see the PUSH command. We can also see the ASCII string Functionfunction, then the integer hexadecimal value. This is in reverse order, as the processor here is an Intel processor which uses little -endian notation, whereby  the lower order byte comes first:

The previous screenshot shows the stack/code part of our functionFunction function, and each statement of this segment represents a statement of the original code that we had.

If we scroll down a little further, we will see the actual main method and the function calls made from there. This is shown next. In the highlighted area is the function call to the actual functionFunction function:

The main function returns 0, which is what is shown by the assembly-level language as we are moving 0 to the EAX register. Similarly, in the previous screenshot, we were moving the value 1 to EAX.

Let's now go to Debug and click on Arguments. From here, we will supply the command-line argument to the assembly code so that we can run it without any errors in the debugger:

We then need to set certain break points to understand the debugger, the program control, and the sequence flow more thoroughly. We will put a break point at the beginning of the main method, as specified by the code shown here:

The break point is highlighted in the following screenshot:

Note that once we run the application, the code actually stops when it hits this line. This is what is meant by a break point:

At the bottom right of the screen, the area we see is the stack area. As we know, every method has a dedicated area of execution, where all the local parameters are stored and where the code is executed. This is the area that we define as a stack. The first statement of the stack points towards the place where the program control is supposed to return after the successful execution of the whole method block. Note that we have four options on the top of the screen, which are Step over, Step onto, Trace onto, and Trace over. We will explore these options as we progress. Let's go ahead and call step into, and see what happens with the stack and the debuggers:

Calling the step into function actually took the control to next line on the debugger. While that is happening, different values are added to the program variables. Note that the following line is going to call the functionFunction function, as specified:

Notice that the previous address at which the function call from the main function to the functionFunction function will happen is from the 004012EA memory address of the main function. When the function is called, the stack allocated to functionFunction must contain the return address, such that once it finishes its execution, it knows where exactly it is supposed to return:

It can be seen on the right that the EIP register is holding the 00401EA address. Note that at the bottom right, the address of the statement itself is 0060FD0 over on the stack. Let's hit next and see what happens:

It can be seen that the moment the function is called, its stack gets updated and says that the code should return to the 004012EF address after the execution. The 004012EF address is the next instruction address of the main function, functionFunction. Note that since the IP contains the address of the next instruction to be executed, it now contains the 00401290 address, which is the starting address of the Functionfunction function. Once it finishes its execution, the contents from the top of the stack will be popped (004012EF) and the IP will be updated with this address so that the program execution is retrieved from where it stopped last.

After clicking on next twice, we see that the first statement, assigning the integer value to a variable in our functionFunction method, will get executed. Finally, when we hit or reach the return statement or the end of the functionFunction method, we will see that the stack top will contain the return address shown in the following screenshot:

We can hit next until the program exits from the main method. This is how the program will execute under normal circumstances, which we call behaved execution. In the next section, we'll see how to make the program misbehave.

Let's see what would happen at the code level of the assembly language when we overflow the buffer by providing an argument that exceeds the expected length. We'll add more than nine characters in the following code:

We will now keep the break point at the main method, as we had earlier. We will reach the break point when we run the code, as shown here:

In the next line, we will copy the value 112233 to the local variable. We will then call the Functionfunction function where the bufferoverflow actually takes place when we do a strcpy on the supplied argument to a local buffer of a size of 10:

As shown in the preceding screenshot, the string that we passed is placed in the register and will be passed to functionFunction. The line after the highlighted line is the actual function call:

It can be seen in the highlighted line that the operation being executed is strcpy(Localstring2,param), which means the value of the EAX register will be moved to the location SS:[EBP +8]. The moment we execute the preceding command, we will notice that the large value we gave will be loaded at the stack. We can see this at the bottom right of the following screenshot:

Now, the next line that will be executed will be the strcpy function after the one that is currently highlighted. We can see the stack of the strcpy function at the bottom right:

There are a few buffers and memory locations in the strcpy function. When we write the value to a buffer of a length of 10, the buffer overflows and the remainder of the value gets spilled and written to the other memory locations of the stack. In other words, the other memory locations in the stack get overwritten by the spilled content. In this case, the memory location that contained the return address of the stack (once the execution was finished) would get overwritten and thus the code will end with an exception. This is what actually happens behind the scenes, as demonstrated in the following screenshot. At the bottom of the screenshot, we can see the access violation exception depicting this: