Process Address Spaces

Although there are typically many processes running at any one time, each process is unaware of the other processes running on the system. In fact, without explicit code, one process cannot interact or influence the behavior of another process.

The operating system provides each process with a range of memory within which it is allowed to operate; this is known as the process's address space. The address space is dynamic and changes during execution as a process allocates memory. If a process attempts to read or write to a memory address outside of its address space, the operating system typically terminates it, and the user informed that the application has crashed.

Although protected memory is not new, it is only within the last decade that it has been found on consumer desktop systems. Prior to Mac OS X, a process running under Mac OS 9 was able to read or write to any memory address, even if that address corresponded to a buffer that was allocated by another process or belonged to the operating system itself.

Without memory protection, applications were able to bypass the operating system and implement their own inter-process communication schemes based on directly modifying the memory and variables of a different process, with or without the consent of that process. This was also true for operating system structures. For example, Mac OS 9 had an internal global variable that contained a linked list of every GUI window that was open. Although this linked list was nominally owned and manipulated by the operating system, applications were able to walk and modify the list without making any calls to the operating system.

Without memory protection, an operating system is susceptible to bugs in user applications. An application running on a system with memory protection can, at worst, corrupt its own memory and structures, but the damage is localized to the application itself. On a system without memory protection, such as Mac OS 9, a bug in an application could potentially overwrite the internal structures of the operating system, which could cause the system to crash entirely and require a reboot to recover.

It is worth noting that on a modern operating system such as Mac OS X, the kernel has an address space of its own. This allows the kernel to operate independently of all running processes. On Mac OS X, a single address space is used for both the kernel and all kernel extensions that are loaded. This means that there is nothing protecting core operating system structures from being inadvertently overwritten by a buggy driver. Unlike a user process, which can simply be aborted, if this situation occurs in the kernel, the entire system is brought down and the computer must be rebooted. This type of error presents itself as a kernel panic on Mac OS X, or the “blue screen of death” on Windows. For this reason, developers of kernel extensions need to be careful with memory management to ensure that all memory accesses are valid.