The Windows API is a broad set of functionality that governs the way that malware interacts with the Microsoft libraries. The Windows API is so extensive that developers of Windows-only applications have little need for third-party libraries.
The Windows API uses certain terms, names, and conventions that you should become familiar with before turning to specific functions.
Much of the Windows API uses its own names to represent C types. For example, the DWORD and WORD types represent 32-bit
and 16-bit unsigned integers. Standard C types like int, short, and unsigned int are not
normally used.
Windows generally uses Hungarian notation for API function identifiers.
This notation uses a prefix naming scheme that makes it easy to identify a variable’s type.
Variables that contain a 32-bit unsigned integer, or DWORD, start
with dw. For example, if the third argument to the VirtualAllocEx function is dwSize, you
know that it’s a DWORD. Hungarian notation makes it easier
to identify variable types and to parse code, but it can become unwieldy.
Table 7-1 lists some of the most common Windows API types (there are many more). Each type’s prefix follows it in parentheses.
Table 7-1. Common Windows API Types
Type and prefix | Description |
|---|---|
| A 16-bit unsigned value. |
| A double- |
Handles ( | A reference to an object. The information stored in the handle is not
documented, and the handle should be manipulated only by the Windows API. Examples include |
Long Pointer ( | A pointer to another type. For example, |
Callback | Represents a function that will be called by the Windows API. For example,
the |
Handles are items that have been opened or created in the OS, such as a window, process, module, menu, file, and so on. Handles are like pointers in that they refer to an object or memory location somewhere else. However, unlike pointers, handles cannot be used in arithmetic operations, and they do not always represent the object’s address. The only thing you can do with a handle is store it and use it in a later function call to refer to the same object.
The CreateWindowEx function has a simple example of a
handle. It returns an HWND, which is a handle to a window.
Whenever you want to do anything with that window, such as call DestroyWindow, you’ll need to use that handle.
One of the most common ways that malware interacts with the system is by creating or modifying files, and distinct filenames or changes to existing filenames can make good host-based indicators.
File activity can hint at what the malware does. For example, if the malware creates a file and stores web-browsing habits in that file, the program is probably some form of spyware.
Microsoft provides several functions for accessing the file system, as follows:
CreateFile
This function is used to create and open files. It can open existing files, pipes, streams, and I/O devices, and create new files. The parameter
dwCreationDispositioncontrols whether theCreateFilefunction creates a new file or opens an existing one.
ReadFile and WriteFile
These functions are used for reading and writing to files. Both operate on files as a stream. When you first call
ReadFile, you read the next several bytes from a file; the next time you call it, you read the next several bytes after that. For example, if you open a file and callReadFilewith a size of 40, the next time you call it, it will read beginning with the forty-first byte. As you can imagine, though, neither function makes it particularly easy to jump around within a file.
CreateFileMapping and MapViewOfFile
File mappings are commonly used by malware writers because they allow a file to be loaded into memory and manipulated easily. The
CreateFileMappingfunction loads a file from disk into memory. TheMapViewOfFilefunction returns a pointer to the base address of the mapping, which can be used to access the file in memory. The program calling these functions can use the pointer returned fromMapViewOfFileto read and write anywhere in the file. This feature is extremely handy when parsing a file format, because you can easily jump to different memory addresses.
Note
File mappings are commonly used to replicate the functionality of the Windows loader. After obtaining a map of the file, the malware can parse the PE header and make all necessary changes to the file in memory, thereby causing the PE file to be executed as if it had been loaded by the OS loader.
Windows has a number of file types that can be accessed much like regular files, but that are not accessed by their drive letter and folder (like c:docs). Malicious programs often use special files.
Some special files can be stealthier than regular ones because they don’t show up in directory listings. Certain special files can provide greater access to system hardware and internal data.
Special files can be passed as strings to any of the file-manipulation functions, and will operate on a file as if it were a normal file. Here, we’ll look at shared files, files accessible via namespaces, and alternate data streams.
Shared files are special files with names that start with \serverNameshare or \?serverNameshare. They access directories or files in a shared folder stored on a network. The \? prefix tells the OS to disable all string parsing, and it allows access to longer filenames.
Additional files are accessible via namespaces within the OS. Namespaces can be thought of as a fixed number of folders, each storing different types of objects. The lowest level namespace is the NT namespace with the prefix . The NT namespace has access to all devices, and all other namespaces exist within the NT namespace.
Note
To browse the NT namespace on your system, use the WinObj Object Manager namespace viewer available free from Microsoft.
The Win32 device namespace, with the prefix \., is often used by malware to access physical devices directly, and read and write to them like a file. For example, a program might use the \.PhysicalDisk1 to directly access PhysicalDisk1 while ignoring its file system, thereby allowing it to modify the disk in ways that are not possible through the normal API. Using this method, the malware might be able to read and write data to an unallocated sector without creating or accessing files, which allows it to avoid detection by antivirus and security programs.
For example, the Witty worm from a few years back accessed DevicePhysicalDisk1 via the NT namespace to corrupt its victim’s file system. It would open the DevicePhysicalDisk1 and write to a random space on the drive at regular intervals, eventually corrupting the victim’s OS and rendering it unable to boot. The worm didn’t last very long, because the victim’s system often failed before the worm could spread, but it caused a lot of damage to the systems it did infect.
Another example is malware usage of DevicePhysicalMemory in order to access physical memory directly, which allows user-space programs to write to kernel space. This technique has been used by malware to modify the kernel and hide programs in user space.
The Alternate Data Streams (ADS) feature allows additional data to be added to an existing file within NTFS, essentially adding one file to another. The extra data does not show up in a directory listing, and it is not shown when displaying the contents of the file; it’s visible only when you access the stream.
ADS data is named according to the convention normalFile.txt:Stream:$DATA, which allows a program to read and write to a stream. Malware authors like ADS because it can be used to hide data.