25. Case Study: IBM Operating System/360
Peach and cherries, a metaphor for the OS/360 big control program and the several smaller and independent language compilers and utilities that complete the OS/360 support package
© P. Desgrieux/photocuisinet/CORBIS. All rights reserved.
The central tension in the software process comes from the fact that we must go from an informally identified need that exists in-the-world to a formal model that operates in-the-computer.
BRUCE BLUM [1996], BEYOND PROGRAMMING
Highlights and Peculiarities1
Bold Decision
Develop one software package: one operating system and set of compilers and utilities for an entire range of computers and I/O configurations. It can be generated to fit and exploit a variety of memory sizes and I/O configurations.
Bold Decision
Mandate a random-access device for operating systems residence.
Bold Decision
Do not require an operator. Design the operating system so that it can run the computer system without manual input or intervention. Operators serve as the computer’s hands and feet for mounting disks, tapes, card decks, and printer paper. Alternatively, the same operating system can be configured to be fully controlled by a human operator.
Bold Decision
Incorporate multitasking for the concurrent safe execution of jobs and programs not specifically designed to be run concurrently.
Device-Independent I/O
Programs are written for abstract I/O datatypes, called access methods. I/O device types, specific devices, and space on them are automatically allocated when a job is scheduled for execution. The same merge-sort program, for example, can be run disk-to-disk for one run and tape-to-tape for another. Whether output is printed or stored can readily be changed at run time, without program alteration.
Industrial-Strength
OS/360 was an industrial-strength operating system, designed to run 24/7, to log and restart automatically after failures. Through its generations this characteristic has been strengthened, so that its descendants are still widely used for 24/7 database systems.
Teleprocessing
The system powerfully supports remote access for real-time database access and batch job execution.
Primitive Time-sharing
The system is not designed for interactive terminal programming and debugging, so it supports it in less efficient ways.
Virtual Memory Added Somewhat Later
As originally shipped, S/360 computers and the OS/360 package did not provide virtual memory. Both were changed in the first subsequent iteration, and all versions had virtual memory by 1970.
Assembler versus High-Level Languages
Although by 1961 probably a majority of computer programming was done in the high-level languages Fortran, COBOL, and Report Program Generator, assembler-language thinking affects some parts of the OS/360 design. A powerful macro-assembler, reflecting the quite different macro use traditions of the scientific-computing and the commercial data-processing communities, was provided as one of the language packages. By 1966, measurements of some large installations showed that application programs written in assembler accounted for only about 1 percent of computer time.
Introduction and Context
From its inception in 1961 until mid-1965, I had the once-in-a-lifetime opportunity to manage the IBM System/360 Computer Family Project—first the hardware and then the Operating System/360 software package. This computer family, announced on April 7, 1964, and first shipped in February 1965, defined “third-generation” computers and introduced (semi)-integrated-circuit technology into off-the-shelf computer products.
Just as the first-generation operating systems were developed for second-generation computers, so OS/360 is the first of the second-generation software support packages, developed for the first of the third-generation computers. There were few precedents of integrated operating systems.
System/360’s strict binary compatibility enabled us to design a single software support package that would support the whole product family and could be cost-shared across the entire family, with its large combined market forecast. This in turn enabled building a software support package of unprecedented richness and completeness. I shall describe the OS/360 software package in the present tense, since its linear descendants are still major players in the mainframe world.
The term Operating System/360, or OS/360, is used ambiguously to describe both an entire software support package—operating system proper, language compilers, and utilities—and, more narrowly, just the operating system itself. As the chapter frontispiece suggests, our team sometimes thought of the entire package as a big peach and a lot of smaller, distinct cherries. I shall usually use the term to describe only the operating system proper.
Besides the OS/360 support package, there was initially planned and delivered a Basic Tape Support package, including a Fortran compiler, for small-memory systems with no disk, and a Basic Punched Card Support Package. The OS/360 package was originally targeted for all systems with 16K of memory or more. We couldn’t fit that size with even minimal function, so we raised our minimum memory requirement to 64K. Concern for small-system customers and OS/360 delays led the company to initiate an entire separate support package optimized for smaller memory sizes, known as Disk Operating System/360, or DOS/360.2 It, too, evolved, and its descendants are alive today.
The System/360 Computer Family
Chapter 24 describes the market context for the computer family and gives its chief architectural properties. A radical conceptual innovation was that all of the models (except the cheapest, Model 20) would be logically identical, upward- and downward-compatible implementations of a single architecture. Blaauw and I define a computer architecture to be precisely the set of computer properties visible to the programmer, not including speed.3 In software engineering terms, a computer architecture in our precisely restricted sense is equivalent to an abstract datatype. It defines the sets of valid data, their abstract representations, and the syntax and semantics of the set of operations proper to those datasets. Each computer implementation then is an instance of that type. So is each emulator, or each simulator. In practice, our first hardware implementations ranged from 8 to 64 bits wide in dataflow and had various memory and circuit speeds.
Software Context as of 1961
Operating Systems
First-generation operating systems were sharply differentiated as to whether they were designed for scientific computing or for commercial data processing. They were batch operating systems, designed to control the sequential processing of a stream of independent jobs.
Each had three components that had evolved separately. A supervisor, resident in memory all the time, had evolved from earlier interruption-handling routines. A data-management component had evolved into a standard library of input-output routines that were linked into the application program. A scheduler, typically resident on tape, was rolled into memory between jobs to specify the mounting of tape (and card) files and the disposition of the output produced. Operating systems provided for the use of disk files, but generally the operating system itself was resident on tape.
Late-first-generation IBM operating systems provided for Simultaneous Peripheral Operation On Line (SPOOL), so that at any given time a second-generation computer could be executing one main application and several card-to-tape/disk, tape/disk-to-card, tape/disk-to-printer utilities. These latter were “trusted programs,” carefully written so as not to corrupt or intrude on the main application, which usually ran tape-to-tape or disk-to-disk. Thus a computer would be preparing tapes for the next job, running a main job, and printing output from the previous job, all at once.
Language Compilers
IBM customers were using a wide variety of high-level languages, and IBM was committed to providing compilers for those. Most popular were Fortran and COBOL. ALGOL was popular in Europe. At the lower end of the spectrum, Report Program Generator (RPG) was popular with those converting from punched-card installations.
The evolution of assembler programs was technically interesting. The classical two-pass assembler had sprouted two preliminary passes that constituted a macro-operation generator with rich compile-time capabilities, including branching and looping. Such macro assemblers were used in two quite different ways.
The scientific computing community typically used programmer-written macros as open subroutines for frequent higher-level operations, such as matrix operations. Macros gave not only coding ease but run-time speed, avoiding the overhead of subroutine calls.
In contrast, many business data-processing shops had evolved the practice of having a small group of gurus write a “house” macro library that essentially defined new datatypes, with data structures and operations defining a specialized programming language for that firm’s business practices. The larger body of programmers just used this macro library, typically not creating any new macros of their own.
Utilities
A variety of utilities, hardly noticed but necessary and nontrivial, completed each computer software package: sort program generators, media translators, format translators, debugging aids, memory dumps.
Free Software
In those days, manufacturers gave away operating systems and compilers to stimulate the sale and use of hardware. Hence the cost of the software packages had to be built into the prices of the hardware.4
Challenges Accepted
The occasion to do an all-new software support package brought forward many challenges of what would constitute the “next step” in software support. Some were accepted, others rejected.
Universal Applicability
Whereas previous-generation software support packages were sharply differentiated by application areas and performance level, the OS/360 package was designed to cover the entire spectrum of applications. The very name System/360 had been chosen to indicate an “all-around computer system.” It was also designed to cover a very large performance range, from a modest 64K-memory system to the most elaborate supercomputer system or massive database configuration.
The response to this challenge mostly affected the languages and their compilers. A new, general-purpose programming language, PL/I, was developed in cooperation with the scientific and commercial IBM user associations. Multiple compilers optimized for different memory sizes were built for each of Fortran, COBOL, Assembler, and PL/I. The teams responsible for the compilers or utilities assessed the ideas in their several user communities, and each incorporated advances over the previous-generation products. Here I will discuss only the most innovative component, the operating system proper.
Disk Residence
The new availability of an inexpensive disk drive, the IBM 2311, with its then-immense capacity of 7MB, meant that we could design the operating system to assume operating system residence on a “random-access” device rather than on magnetic tape. This made the biggest single difference in the design concepts. Operating system modules could be quickly rolled into memory as needed, and they could be made small and function-specific.
A new, word-parallel magnetic drum provided low-latency, high-data-rate operating systems residence for higher-performance computer systems.
Multiprogramming
OS/360 made the big leap to concurrent operation of independent, untrusted programs—a leap made possible by the hardware supervisory capabilities of the System/360 architecture. Early OS/360 versions supported multiple tasks of fixed size, for which memory allocation was straightforward. Within two years the MVS version supported multiprogramming in full generality. This proved much more difficult than we expected.
The OS, Not the Operator, in Control
A key new concept, now routine, is that the OS, not the operator, controls the computer. As late as 1987, some supercomputers such as the Control Data Corporation’s spin-off ETA’s ETA 10, were still running under manual operator control. A corollary of OS control, pioneered in Stretch, and routine today, is that the keyboard or console is just another I/O device, with very few buttons that directly do anything (for example, Power, Restart).
Teleprocessing, but Not Time-sharing
OS/360 was designed from the ground up as a teleprocessing system, but not really a terminal-based time-sharing system. This concept contrasts with that of the contemporary MIT Multics System. OS/360 was designed for industrial-strength scientific and data-processing applications of all sizes; Multics was designed as an exploratory system, primarily for program development.
24/7 Robust Operation
OS/360 is designed to provide checkpoint-restart points automatically, to sense hardware errors, and to restart after either a hardware or a software failure. When used in a multiprocessor configuration, diagnostics enable a well processor to sideline a sick one and to assume its workload. From the beginning, OS/360 was intended to be usable around the clock, although it took some evolutionary steps to get there.
Design Decisions5
System Structure
In OS/360, the three independent streams of control program evolution come together. The Supervisor evolved from early interrupt-handling routines; the Scheduler, from earlier tape-based job schedulers; the Data Management System, from earlier packages of I/O subroutines. The system structure mirrors this diverse ancestry.
The Supervisor
Whereas original supervisors handled only program interruptions, and thereby allocated the processor’s instruction counter among tasks, a multiprogramming supervisor must allocate main memory space as well. The OS/360 Supervisor allocates memory blocks and computer cycles among tasks according to priority.
The OS/360 Supervisor keeps control of the computer by controlling the instruction counter. It lends that control to one program at a time. Any program fault, including attempted violation of any of the system protection mechanisms, causes an interruption, giving the instruction counter back to the Supervisor. Asynchronous event reports from I/O devices, such as operation completions, do the same. Moreover, the Supervisor controls a protected elapsed-time clock that interrupts, so it can seize control after any specified interval, thus stopping endless loops in buggy programs. Only the Supervisor can set the various memory and other protections and perform other privileged operations such as input-output control.
When an ordinary application program wants a service from the Supervisor, such as an additional memory block, it makes its request by a Supervisor Call hardware operation. This is an intentional interruption, with the instruction carrying parameters to the Supervisor. So the only access to the Supervisor is a humble access, on the Supervisor’s terms.
The Supervisor also provides mechanisms for mutually unaware programs to communicate with each other at run time.
The Scheduler
The OS/360 Scheduler prepares the concurrent execution of independent “jobs” and then manages the sequential execution of “tasks” within each job, such as compilation, linking to libraries, execution, output transformation. When a job is ready to be scheduled, the Scheduler checks the job priority, allocates any needed I/O devices, gives operator instructions for the mounting of any off-line data volumes, and enqueues the job for execution. The Supervisor then allocates initial memory and initiates the first task. As output is produced, the Scheduler manages its disposition and the dismounting of any finished data volumes.
OS/360, more explicitly than any of its predecessors, recognizes scheduling time as a binding occasion distinct from compile time with its rigidities and run time with its overheads. Not only are separately compiled program modules bound to each other at scheduling time by a Linker, but dataset names are bound to particular datasets on particular devices only at scheduling time. This binding is specified by the Job Control Language, which is executed by the Scheduler.
Data Management
Although strict program compatibility was the most distinctive new concept of the System/360 computer family, the rich set of I/O devices was its most important system attribute in terms of application breadth, configuration flexibility, and performance enhancement. The single standard mechanical, electrical, and logical I/O interface radically reduced the engineering cost for new I/O devices, radically simplified system configuration, and radically eased configuration growth and change.
The crucial software innovation complementing and exploiting the standard hardware I/O interface was a standard software interface—a single system of I/O control and data management for all kinds of I/O devices. I consider it the most important innovation in OS/360.
A resulting new feature was device-independent input-output. The application programmer wrote in terms of dataset names. Bindings to particular datasets, to particular reels of tape, to tape versus disk, to disk versus communication line or printer, all were usually deferred to scheduling time.
Four access methods were designed especially for exploitation of the fleet of new disk types, across the range of disk applications. These embodied different trade-offs between dynamic flexibility and maximum-performance buffered or block transfer:
• Sequential access method—tape-like, buffered Example: for sorting (works for tapes, printers, and card decks, as well as for disks)
• Direct access method—pure random access to a record Example: for airline reservations
• Partitioned access method—fast fixed-block transfer Example: for operating system modules
• Indexed sequential access method—sequential, buffered, but rapidly handling random queries Example: for utility billing
Two other access methods were designed especially to provide full flexibility and ease of use for both terminals and high-speed telecommunications.
Of all the I/O devices, the check sorters alone, curiously enough, posed a rigid constraint on operating system performance—the time of a paper check’s flight between the reading head and the sorter pocket is fixed and short. In banks’ check-routing and check-processing facilities, these machines read the magnetic-ink numerals along the bottom of the check at a reading station and then route the check into one of some 24 pockets, at a rate of up to 40 checks/second.6
Assessment
The Successes
Full Function, Universal Applicability
OS/360 established a new baseline for operating system function. It did indeed support a surprisingly wide range of applications, system configurations, and performance.
Robustness
The level of robustness has become peerless. It is the industrial-strength operating system and has become a standard for the massive database applications that consume most mainframe cycles.
Data Management System
Device-independent input-output was a major simplification of the programming task and a major flexibility for data center operation and evolution. Weekend reconfigurations of processors and I/O devices are routinely performed. After such reconfigurations, most applications can still be run without recompilation.
Teleprocessing Support
OS/360 became the basis for wide networks of terminals for banking, retail, and most other industries.
Accommodated Virtual Memory
When IBM adopted virtual memory on the System/370 successor line, OS/360 served as the base for OS/360 Multiple Virtual Storage (MVS), an extension, but not a total rewrite.
Amdahl, Hitachi, and Fujitsu
Most manufacturers of S/360 plug-compatible computers did not undertake software systems but used the OS/360 package.
Weaknesses in the Design
The System
OS/360 is too rich. Systems residence on a disk removed the size constraint that had disciplined earlier OS designers—we put in many functional goodies of marginal usefulness.7 Featuritis is even now not yet dead in the software community.
Two quite different debugging systems are provided, one conceived for interactive use from terminals with rapid recompilation, the other conceived for batch operation. It was the best batch debugging system ever designed, yet totally obsolete from birth.
The system-generation process of OS/360 is wondrously flexible and wondrously onerous. We should have configured a small number of standard packages to meet the needs of most users and offered these to supplement the fully flexible configuration process.
Control Blocks
Communication among modules is by system-wide shared control blocks, each with a structured set of variables read by and written by the several modules. Every programmer had access to all the control blocks. Had we understood and adopted in 1963 the information-hiding strategy Parnas set forth in 1971, we would have avoided much grief in original construction and all subsequent maintenance. Object-oriented programming is today’s embodiment of information hiding; we all recognize its superiority.
Virtual Memory
As discussed in Chapter 24, we missed the early boat on virtual memory in the initial processors and had to retrofit it just a few years later. The required extension to OS/360 was more difficult and costly than if it had been originally designed in.
The Scheduler’s Job Control Language
The Job Control Language is the worst programming language ever designed by anybody anywhere—it was designed under my management. The very concept is wrong; we did not see it as a programming language but as “a few control cards to precede the job.” I have elaborated on its flaws in Chapter 14.
Complexities in the Data Management System
We should have cut loose from the count-key-data variable-length block structure established for IBM’s earlier disks and designed for one or two sizes of fixed-length blocks on all random-access devices.
The I/O device-control unit-channel attachment tree is unnecessarily complex.8 We should have specified one (probably virtual) channel per device and one (probably virtual) control unit per device.
I believe we could have invented one sequential disk access method that would have combined the optimizations of the three: SAM, PAM, ISAM.
Weaknesses in the Process
I have treated this topic at length in The Mythical Man-Month. Here I will highlight only two points.
I am firmly convinced that if we had built the whole thing in PL/I, the best high-level language available at the time, the OS would have been just as fast, far cleaner and more reliable, and built more swiftly. In fact, it was built in PLS, a syntactically sugared assembler language. Using PL/I (or any high-level language) would have indeed required careful training of our force in how to write good PL/I code, PL/I source that would compile to fast run-time code.
We should have maintained rigid architectural control over all the interfaces, insisting that all declarations of external variables be included from libraries, not crafted anew in each instance. Many bugs would have been prevented.
The Designers
About 1,000 people worked on the entire OS/360 software package. Here I identify both the teams and those individuals who contributed most to the conceptual structure.
Key Players
Labs: Poughkeepsie, Endicott, San Jose, New York City, Hursley (UK), La Gaude (France)
OS/360 Architect: Martin Belsky
Key: Bernie Witt, George Mealy, William Clark
Control Program Manager: Scott Locken
Compilers, Utilities Manager: Dick Case
OS/360 Assistant Project Manager: Dick Case
OS/360 Manager from 1965: Fritz Trapnell
The best single document is the Concepts and Facilities manual, written by Bernard Witt.9,10
General Lessons Learned
1. Give the system architect full authority over the design (Chapter 19). This “multimillion-dollar mistake” is more fully discussed in The Mythical Man-Month, pages 47ff.
2. Take the time necessary to do a sound design and prototyping, whatever the schedule pressures. The project will be completed sooner, not later, because of time so invested. Chapters 21–24 illustrate the benefits of enough design time; this one illustrates the opposite.
Notes and References
1. This essay is derived from Brooks [2002], “The history of IBM Operating System/360,” in Broy and Denert [2002], Software Pioneers. The material is taken largely from IBM Systems Journal 5, no. 1 [1966].
2. http://en.wikipedia.org/wiki/DOS/360_and_successors, accessed August 2009.
3. Blaauw and Brooks [1997], Computer Architecture, Section 1.1.
4. Grad [2002], “A personal recollection,” describes the 1969 unbundling of software and hardware.
5. Pugh [1991], IBM’s 360 and Early 370 Systems, gives a detailed history of the OS/360 initial development.
6. More information and a photo of a successor machine is at http://www.thegalleryofoldiron.com/3890.HTM, accessed August 2009.
7. Brooks [1995], The Mythical Man-Month, Chapter 5.
8. Blaauw and Brooks [1997], Computer Architecture, Section 8.22.
9. IBM Corp. and Witt [1965], IBM Operating System/360, Concepts and Facilities, Form C28-6535-0.
10. Witt [1994], Software Architecture and Design, elaborates on the design concepts and approaches.