The Core Functions of an Operating System

The OS is the software program that really understands the hardware and knows how to work with it. Because the OS is so knowledgeable, the programs, the hardware devices, and the user don't have to be as aware. Table 5.1 provides an overview of the five key functions of the OS. Each will be covered in turn.

Table 5.1. Operating system functions

Operating system function	Example
System supervision	manages memory, starts and controls processes
Services to hardware	drivers for various devices (drives, displays, etc.)
Services to software	support to other programs such as the user inter-face (UI), file systems, etc.
Communications services	redirects requests for information from the local system to an external one; connects to network device drivers (e.g. Ethernet board)
Security	controls user access to directories and files

System Supervision

One way to think of the OS is as a manager. Of all the programs in operation, the OS is the one in charge.

Managing Memory

The most important of its supervisory functions is control of memory. This will be easier to understand if you think of memory as the tables in a large restaurant. If you have one place where reservations are taken and tables are assigned, all should work smoothly. But if people are constantly coming in at different doors and asking the waiters to seat them or take reservations, things can get out of control quickly. The OS, then, is like a maître d', checking off table assignments, keeping a master list of reservations, and so on. To see why memory management is important, remember that standard computers run at least two programs at any given time: an application program and the OS (the OS is also a program; the word application is normally used to describe software that performs tasks directly for users). Both need to use memory, and there is danger of fatal conflict if they attempt to access the same area. The original way for an OS to cope with this was to create partitions. The two programs then each have an area of memory set aside for their exclusive use. Figure 5.1 shows how memory could be partitioned in the one megabyte available to the PC's original operating system, MS-DOS.

Physical Memory

It helps to understand this if you think of a computer's memory as a lot of pigeonholes on a rolltop desk. If software needs to store something temporarily, it puts it in hole number 32 (or whatever number is available); it then keeps track of that number and when it needs the information back, knows exactly where to find it. This is described as working with physical memory—each program gets a set of addresses that correspond to actual places on DRAM chips and that only it can use.

Partitioning physical memory hampers system effectiveness when more than one program is running. If we assume that physical memory is limited, just dividing it into lots of partitions will mean that each program is going to get a pretty small slice (as in Figure 5.1). Moreover, such a static approach to partitioning isn't fair—at any given time it's likely that one program will be very active and struggling for as much memory as it can get, while the other is just sitting there, not doing any work but occupying valuable memory space. Obviously, a dynamic approach to partitioning, in which memory is allocated and reallocated according to need, is preferable to static allocation. Modern OSs are able to do this, largely because they use a trick called virtual memory.

Virtual Memory

Dynamic allocation of memory has the potential to be much more effective than the static approach. For example, when the OS wants the word processor to yield to the database, it simply swaps the word processor out of memory to disk and moves the database from disk to memory. The challenge in this strategy occurs when, as must happen frequently (lots of times per second in an interactive system), the process is reversed. Because of changes that occur naturally in the normal execution of a program, the shape of physical memory might no longer be the same when a program is restarted—items would then be returned to different pigeonholes. This could cause big problems if the word processor, when it resumed operating, went to pigeonhole 32 for something that was now to be found in pigeonhole 49.

IBM solved this problem for its mainframes by creating a system of memory addresses that was independent of physical memory. Called virtual memory (see Figure 5.2), this approach sets up a numbering scheme that is typically much larger than physical memory can accommodate—say 16 million vs. 1 million. Because there is now an almost limitless number of pigeonholes, all operating programs get unique addresses—if the word processor is assigned virtual location 32 for the storage of some piece of information, it is the only program to get it. But this number no longer represents an actual place; instead, it is a tag that identifies the information.

In order to keep track of where virtual addresses are to be found in physical memory, the system now includes a layer, the memory management system, that translates between virtual and physical memory as things are moved back and forth. Memory is now much easier to manage, and more and larger programs can be accommodated because the virtual memory is so much larger than the physical memory (virtual addresses that can't be kept in physical memory are held in disk storage). While virtual memory is a function of the OS, implementing it efficiently requires changes to hardware as well. CPUs without attached or on-chip memory management systems can't realistically handle virtual memory. Chips—CPUs or chipsets—now also include translation look aside buffers that keep a table of virtual-physical memory conversions close at hand. A well-designed translation look aside buffer (no acronym!) can make memory management work much faster.

An important point about memory management is that virtual memory systems don't move memory around one byte or even one word at a time. Instead, memory is partitioned into blocks of around 2 to 4 kilobytes each, called pages, that are shifted all at once. So, when the memory management system identifies a virtual address that needs to be shifted back from disk to memory, it moves the entire page that the address is on (and if memory is full, swaps another page from memory back to disk in order to make room for it). When the memory management system discovers that a requested address is not in physical memory, it issues a page fault, which is the command to have it returned. Programmers, as you would guess, spend a lot of time thinking about how to organize their programs in a way that avoids page faults. One last piece of memory jargon: when a program gets confused and causes lots of page faults to occur in sequence, it is said to be thrashing—programmers really want to prevent this.

The OS's supervisory role includes more than memory management. The OS also has to keep track of multiple programs. This function, called multitasking, is discussed next.

Multitasking

The ability of a computer to handle two or more programs at once is a key distinction of an OS. For quite some time, this was a way of separating the big guys from the little ones—mainframe, mini, and workstation systems were multitasking and microcomputer systems were not. While it would have been theoretically possible to put a multitasking OS (such as Unix, see Chapter 6) on a chip like an Intel 8088, the size of the OS, not to mention that of multiple programs, would have brought operations to a crawl. By about 1985, and with the appearance in the microcomputer world of systems based on the Intel 80386 for the PC and the Motorola 68020 for the Mac, this had changed. A workable version of Unix was ported to the Intel platform and a form of multitasking was added to the Mac OS. Windows 1.0 was multitasking too, and supposedly even on a 286, but rumor has it that various religious cults were using these Windows/286 systems as a way of teaching extreme patience to their disciples. Since about 1990, all of the major microcomputer platforms have supported multitasking and supported it well.

From Batch Processing to Multitasking Early programs ran on the computer in what was known as batch mode. This meant that once the program was loaded, it kept running until it had completed its batch of instructions. No user intervention was possible and no other program could run at the same time. After some effort, designers were able to implement multiprogramming, which allowed two or more programs to share access to the system in batch mode. This greatly improved efficiency. The final step, allowing multiple interactive programs, took longer to develop even though it was a key goal of the computing community. The first of these term "time sharing" systems were available commercially in the late 1960s. The term time sharing has now gone out of use and has been replaced by multitasking.

It's important to mention that there are two basic types of multitasking: cooperative and preemptive. In cooperative multitasking, two or more programs are running and the one with control of system resources gets to keep control until it has to wait for something—receiving input from the keyboard, for example. The other can then jump in and stay in control until it yields (some timing modifications are possible). Rules for who can go when have to be defined in advance, and for best functioning, are designed into the programs. The problem with cooperative multitasking is that one program that is working intensely can control the system and lock another out. That is a particular problem when the program not in control needs to respond to something that is time-dependent, such as information coming in on a serial port.

Preemptive multitasking means that the OS allocates time between running programs. The OS switches back and forth quickly and regularly so that each task gets a little bit of the CPU's time. The OS can also intervene if an external event (e.g., that pesky serial port again) requires that one task get higher priority than the other. The preemptive approach is more efficient and is strongly favored by the most advanced OSes.

Multithreading

Until now, we have used program to describe software that is executing on a computer. The correct technical term for an executing program, however, is process. When running concurrently on a computer (multitasking), processes can be defined by the fact that they are allocated separate virtual memory "spaces" (a space in this case is a range of addresses). A thread, which is a relatively new term, is a subprocess. One process can have a number of different threads that execute tasks independently. For example, a database program (a process with its own memory space) can use threads to analyze a query and prepare it for sorting at the same time (or virtually so) that it is managing a sort from a previous query and formatting the one before that to send to the user. Threads are different from processes in that they are subject to the control of the parent process and share its memory space rather than having their own. A multitasking OS doesn't necessarily support multiple threads, though most now do. At the end of the 1990s, few microcomputer programs were written to take advantage of multithreading, but increased OS support, together with the development of microprocessor architectures that are thread-friendly, has started a stampede to this new type of software.

Since system designers are increasingly finding that adding multiple CPUs is a good way to increase performance, the modern OS has to assume the complex burden of managing what is called multiprocessing. We'll cover this topic at the end of the chapter.

Services to Hardware

The OS supports hardware in a variety of ways; we'll talk here about the most important, the device driver, which refers to software that connects programs to hardware. A device driver isn't typically a full program, so it attaches to and becomes a part of the OS, as shown in Figure 5.3. This usually means that after a driver is installed, the computer has to be restarted so that the OS can reformat itself to accommodate the internal connections, memory needs, and other requirements of the driver. However, the drivers for some newer devices, such as the Universal Serial Bus (see Chapter 2), have been constructed in a way that allows a host driver to manage the connection of subsidiary devices without the need for rebooting. This is called hot swapping.

To illustrate the role of drivers, we'll look at two different ways of handing printing.

Application-specific Drivers

In the old way of doing things under Microsoft's DOS, an application program, say Lotus 1-2-3, had the responsibility of providing a software interface, known as a driver, that spoke to the printer. The driver would do such things as adjust the spreadsheet to accommodate the printer's font selection and tell the printer when to start a new page. In other words, the driver translated between Lotus 1-2-3's commands and those of the printer. Since a user might choose from many different printers, Lotus had to provide drivers for all of them. This was a lot of effort (drivers are among the toughest pieces of software to write). Worse, the work was never done because Lotus also had to deal with new printers as they appeared, and it had to provide updated drivers for everybody when it came out with new and improved versions of 1-2-3. This was a major pain in the neck for Lotus and for users. In fact, the pain for printer manufacturers could become terminal if Lotus didn't get around to writing drivers for their equipment. For a long time, 1-2-3 was the major reason people bought computers; if Lotus didn't support a printer, people wouldn't buy it. The role of the OS was limited to providing memory for the driver and handling communications within software and to the printer's hardware port. As suggested in Figure 5.3, this type of driver just passes information through the OS.

Generic Translation Drivers

In the Windows approach (see Figure 5.4), Lotus only has to provide an interface to a software component of Windows called the GDI (graphics device interface). On the "other side" of Windows, there is a device driver that connects the GDI to a specific printer. Described another way, Lotus speaks GDI's language to Windows, and Windows' GDI layer in turn uses drivers to talk to the printer. Actually, the GDI is even more generic than shown here because it allows applications to speak to all display devices—not just to printers but also to graphics cards. Each of these external devices needs a driver to talk to the GDI, but the individual programs now only need to be written for one external interface—the GDI. In this scheme, either Microsoft or the hardware vendor is responsible for providing the drivers. This is still a chore, but a much more manageable one than in the old approach. The GDI, which has to be a very sophisticated piece of software to accommodate such a large number of devices with such a considerable array of capabilities, illustrates the increasingly complex services that modern OSes provide to applications.

Services to Software

The OS assists other software programs in a number of ways. We'll discuss three key aspects: the user interface, the application programming interface (API), and the file system.

The User Interface

In the early days of computing, each program was responsible for presenting itself to the user. This meant not only the appearance of the screen, but also the entire set of available commands. By the late 1970s, operating systems had taken a part of this burden on themselves. To illustrate, we'll use the familiar example of Microsoft's Windows. You know that Windows gives you a startup screen, the Desktop, which you use to find and launch programs. In addition, Windows has standardized much of the menu system. For example, most programs use the same "File" and "Edit" menu settings at the left of the top bar. Windows also provides some consistent functions, such as "cut," "copy," and "paste," that are available to all applications and normally allow movement of information from one program to another. Of course, Windows doesn't control the whole thing. The actual menu settings will vary according to the functions it provides to the user.

The first user interfaces (UIs) were character-based and command-driven. If you've seen MS-DOS, you know what this means. If not, think of a blank screen with just the prompt "C:>" or something similar somewhere on the left. Once this appeared, it was your responsibility to know what to do next. With limited exceptions, today's UIs are graphical and menu-driven. They are often called GUIs (for graphical user interface), but this acronym will be avoided for aesthetic reasons. In some OSes, such as Windows and the Mac, the UI is embedded in the OS's core software. In others, such as Unix and its popular sibling Linux, the user can choose between a variety of different UIs (see Chapters 6 and 7 for descriptions of these OSes).

Application Programming Interface (API)

The user isn't the only one who needs help in working with the OS; programs require services and assistance as well. Just as the user sees the UI as a map to the services of the OS, so the programmer has a set of APIs (Application Programming Interfaces) that provide a similar, but more comprehensive, map. The set of APIs published for an OS provides a library of functions that a programmer can call to do such things as open a new window or provide a menu bar. The term API is now used rather widely, often referring to any request for a function (an action) going from any program to any other.

APIs need not be static; an OS vendor can develop and publish new ones when the software is rewritten. For example, Microsoft has added TAPI (telephony API) to its newest OSes so that they can be used to support and manage a wide variety of telephone-related applications. To illustrate, programmers now have access to APIs that allow them to route a call from a company's telephone switching system (usually a PBX, or private branch exchange) directly to a PC on a local area network. From there it can be passed to an application that records it and makes it available as a substitute for an answering machine. To illustrate further, other program-to-program APIs would also make it possible for a voice recognition program to transcribe the voice message, put it in an e-mail, and send it to the user's palmtop. Without APIs, programmers wouldn't be able to take advantage of the services of the OS, and other programs and would have to create everything on their own.

Show Us Your Documents! A hot issue in computing has been "undocumented APIs." As the name suggests, these are known to the developer of the software but not to others. Critics have accused Microsoft of using undocumented APIs in Windows as a way of giving an advantage to Microsoft's applications (Word, Excel, etc.). Microsoft has denied this, saying that there is a "Chinese Wall" between its OS and applications groups. Whatever the truth of this issue, there are more important reasons for the success of Microsoft's applications (see Chapter 9).

File Systems

Since the concept of files is a fairly basic one, it won't be covered in depth here. Essentially all computers use a hierarchical file system, with directories and files branching off from the center like roots on a tree. The way in which files are structured and organized is often an intrinsic feature of the OS; DOS has a file system of its own, so do IBM mainframes, and so forth. Some OSs, such as Windows NT, Windows 2000, and most mainframe systems, can support multiple file systems at once.

Communications Services

The OS manages not only physical connections to external devices via drivers, but also logical linkages in networked environments. This is illustrated by the simple process of saving a file. If you are on a network, and want to save a file to a drive on the server, for example, on Drive M, your application program will save it just as if it was to the local Drive C. In a networked system, the OS recognizes that M is not a local drive, and instead of sending it to a disk, forwards the file information to the section of the OS that manages communications. Once there, the file is packaged in the appropriate network protocol, given the correct address, and sent to the right output port (for example, an Ethernet card; network protocols and connections are described in detail in Part 3). Older OSs, such as MS-DOS, could be modified with driver-like software that allowed them to function on networks; an example is Novell's "requestor" modules. Modern OSs, however, have this capability built in.

Security

Security isn't much of a concern in a single-user operating system, but quickly becomes a big deal when multiple users are sharing files and services. As a consequence, we'll digress for a moment to talk about multiuser systems.

Multiuser Operating Systems

A true multiuser OS is one that has only one copy of the OS that is shared by multiple users. These systems are also called terminal-host. As we'll see in Part 3 when we talk about networking, in terminal-host systems, only the host has any intelligence. The terminal has no CPU, little or no memory, and no storage. Terminal-host computing is the exclusive province of mainframe and minicomputer OSs (these go by confusing names such as VM and MVS from IBM, and VMS from DEC; see Chapter 6). These OSes will allow many users, perhaps thousands, to be sharing system resources at any given time. Most versions of Unix are also multiuser, and some are designed to support hundreds of simultaneous terminal users.

Multiuser operating systems are necessarily distinguished by the degree of security they provide. The level of interaction varies, but the typical system is one in which many people or client computers are using the host computer or server at the same time. Each of these users is interacting with his or her process, or they are using separate threads in the same process, or both. This level of interaction makes some kind of traffic control mandatory. Processes and threads have to be identified with their owners and, equally important, files and collections of files (directories) have to be identified in a way that makes it clear who has the right to access and change which piece of information.

Multiuser systems normally give each user a separate identification (user ID), that tracks processes and threads, as well as files. Normally, a user will see not the ID but a username, the system automatically handles the conversion from name to number. Users have to get these usernames and IDs from a system administrator. User IDs are normally associated with specific levels of access. For example, a member of the accounting staff would have access only to materials in the section of the computer set aside for the accounting department. Those files that she "owns" personally (usually because she created them) can be accessed and modified at will (typically called read/write access). By contrast, she might only have the right to look at, but not modify, files that are in a common area for the department (read-only access). Of course, she would not be able to see, much less read or modify, files in other individuals' areas.

Some other dimensions of OS-based security include internal auditing to see who used what and for how long, controls to make sure that deleted files are erased so they can't be viewed by snoopers, and other such functions. Finally, to extend the example above, none of the individuals in accounting or any of the other user departments would have any kind of access to "system" files (those that belong to the operating system itself) or to files that run system programs (such as one that shuts down the computer), etc. Access to these files is limited to system administrators (some OSs refer to them as superusers) and they are often protected by multiple levels of passwords.

The OS ensures that a user is who she says she is by means of passwords. A user logs in to the computer, is asked to give the password, and, if the user ID and password can be verified by the computer, is given access to whatever resources the holder of that ID is allowed to access. There are various ways of handling passwords. Unix, which is fairly secure, takes the password from the keyboard, runs it through an encryption algorithm, then compares it to the encrypted original that is in the file with the user's ID. If there is a match, the system provides access. Next, it uses information in the file to determine what files and services the user may work with. As you can see, it is no trivial matter for an OS to support multiple simultaneous users. Such systems must be designed as multiuser from the ground up if they are to be effective.

Single-User Systems in a Networked Environment

Single-user computers can be set up to facilitate connection to a multiple user host (a server) in one of two basic ways. In the first case, add-on software (normally provided by the maker of the server OS) allows the single-user system (the client) to be authenticated by the host. In the second and now standard approach, the single-user OS is designed from the beginning to function as a network client and comes out of the box with provision for password logins, file access restrictions, etc. Windows NT and Windows 95+ support this approach. Unix also has these capabilities, but doesn't fit in this category because both client and server are full-blown multiple user OSs (to be fair, this is sort of true for Windows NT and Windows 2000 as well, since even the client version will support peer-to-peer networking; see chapters 6 and 7).

Directories

Our networked world would be significantly hampered if an individual user could only connect to and get services from one server. Such an approach would mean either that a particular user would be limited to the services available from a single server, or that needed information on every server on the network would have to be copied to that user's designated server. Whatever the choice, the attendant limitations or inefficiencies would defeat much of the purpose of a network. The solution to this problem, of course, is some system that easily supports users having privileges on multiple computers. It turns out that it is very hard to make this easy. Databases that manage users and resources across a network are called directories. They are fairly simple for small, local area networks but become extremely challenging to design and manage for large, disparate networks with users accessing resources in a dynamic way. The problem of smoothly implementing directory services across the Internet has not yet been solved.