In a single byte XOR, each byte from the plaintext is XORed with the encryption key. For example, if an attacker wants to encrypt plaintext cat with a key of 0x40, then each character (byte) from the text is XORed with 0x40, which results in the cipher-text #!4. The following diagram displays the encryption process of each individual characters:

Another interesting property of XOR is that when you XOR the ciphertext with the same key used to encrypt, you get back the plain text. For example, if you take the ciphertext #!4 from the previous example and XOR it with 0x40 (key), you get back cat. This means that if you know the key, then the same function can be used to both encrypt and decrypt the data. The following is a simple python script to perform XOR decryption (the same function can be used to perform XOR encryption as well):

def xor(data, key):
    translated = ""
    for ch in data:
        translated += chr(ord(ch) ^ key)
    return translated

if __name__ == "__main__":
   out = xor("#!4", 0x40)
   print out

With an understanding of the XOR encoding algorithm, let's look at an example of a keylogger, which encodes all the typed keystrokes to a file. When this sample is executed, it logs the keystrokes, and it opens a file (where all the keystrokes will be logged) using the CreateFileA() API, as shown later. It then writes the logged keystrokes to the file using the WriteFile() API. Note how the malware calls a function (renamed as enc_function) after the call to CreateFileA() and before the call to WriteFile(); this function encodes the content before writing it to the file. The enc_function takes two arguments; the ^1st argument is the buffer containing the data to encrypt, and the 2^nd argument is the length of the buffer:

Examining the enc_function shows that the malware uses single byte XOR. It reads each character from the data buffer and encodes with a key of 0x5A, as shown here. In the following XOR loop, the edx register points to the data buffer, the esi register contains the length of the buffer, and the ecx register acts as an index into the data buffer that is incremented at the end of the loop, and loop is continued as long as the index value (ecx) is less than the length of the buffer (esi):