The Korn shell is almost completely backward-compatible with the Bourne shell. The only significant feature of the latter that the Korn shell doesn’t support is ^ (caret) as a synonym for the pipe (|) character.^[131] This is an archaic feature that the Bourne shell includes for its own backward compatibility with earlier shells. No modern Unix version has any shell code that uses ^ as a pipe.

To describe the differences between the Bourne shell and the Korn shell, we’ll go through each chapter of this book and enumerate the features discussed in the chapter that the Bourne shell does not support.

Perhaps the most obvious shell with which to compare ksh93 is ksh88, the 1988 version of the Korn shell. This section briefly describes those features of ksh93 that are either different or nonexistent in ksh88. As with the presentation for the Bourne shell, the topics are covered in the same order as they’re presented in the rest of the book.

There have been many attempts to standardize Unix. Hardware companies’ monolithic attempts at market domination, fragile industry coalitions, marketing failures, and other such efforts are the stuff of history — and the stuff of frustration.

Only one standardization effort has not been tied to commercial interests: the Portable Operating System Interface, known as POSIX. This effort started in 1981 with the /usr/group (now UniForum) Standards Committee, which produced the /usr/group Standard three years later. The list of contributors grew. Eventually, the effort to create a formal standard moved under the umbrella of the Institute of Electrical and Electronic Engineers (IEEE) and the International Organization for Standardization (ISO).

The first POSIX standard was published in 1988 and revised in 1996. This one, called IEEE Standard 1003.1, covered low-level issues at the system call level. IEEE Standard 1003.2, covering the shell, utility programs, and user interface issues, was ratified in September 1992 after a six-year effort. In September 2001, a joint revision of both standards was approved. The new standard, covering all the material in the two earlier separate documents, is known as IEEE Standard 1003.1-2001.

The POSIX standards were never meant to be rigid and absolute. The committee members certainly weren’t about to put guns to the heads of operating system implementers and force them to adhere. Instead, the standards are designed to be flexible enough to allow for both coexistence of similar available software, so that existing code isn’t in danger of obsolescence, and the addition of new features, so that vendors have the incentive to innovate. In other words, they are supposed to be the kind of third-party standards that vendors might actually be interested in following.

As a result, most Unix vendors currently comply with both standards. The Korn shell is no exception; it is intended to be 100% POSIX-compliant. It pays to be familiar with what’s in the standard if you want to write code that is portable to different systems.

The shell part of the standard describes utilities that must be present on all systems, and others that are optional, depending upon the nature of the system. One such option is the User Portability Utilities option, which defines standards for interactive shell use and interactive utilities like the vi editor. The standard — on the order of 2000 pages — is available through the IEEE; for information, contact the IEEE:

[email protected] http://www.standards.ieee.org/catalog/ordering.html

The committee members had two motivating factors to weigh when they designed the shell standard. On the one hand, the design had to accomodate, as much as possible, existing shell code written under various Bourne-derived shells (the Version 7, System V, BSD, and Korn shells). These shells are different in several extremely subtle ways, most of which have to do with the ways certain syntactic elements interact with each other.

It must have been quite difficult and tedious to spell out these differences, let alone to reach compromises among them. Throw in biases of some committee members towards particular shells, and you might understand why it took six years to ratify the first 1003.2 standard and another five years to merge the two standards.

On the other hand, the shell design had to serve as a standard on which to base future shell implementations. This implied goals of simplicity, clarity, and precision — objectives that seem especially elusive in the context of the above problems.

The designers found one way of ameliorating this dilemma: they decided that the standard should include not only the features included in the shell, but also those explicitly omitted and those included but with unspecified functionality. The latter category allows some of the existing shells’ innovations to “sneak through” without becoming part of the standard, while listing omitted features helps programmers determine which features in existing shell scripts won’t be portable to future shells.

The POSIX standard is primarily based on the System V Bourne shell. Therefore you should assume that Korn shell features that aren’t present in the Bourne shell also aren’t included in the POSIX standard.

However, the Korn shell did contribute a few of its features to the POSIX standard, including:

The following Korn shell features are left “unspecified” in the standard, meaning that their syntax is acceptable but their functionality is not standardized:

The POSIX standard supports functions, but the semantics are weaker than the Korn shell’s function-style functions: functions do not have local traps or options, and it is not possible to define local variables. (For this reason, ksh93 has two different syntaxes for defining functions, with different semantics.)

Code blocks ({...; }) are supported. For maximum portability, when you want literal curly braces, you should quote them (for reasons too complicated to go into here).

The POSIX standard introduced the following features, which are different from traditional Bourne shell behavior. ksh93 supports them all:

Finally, because the POSIX standard is meant to promote shell script portability, it explicitly avoids mention of features that only apply to interactive shell use — including aliases, editing modes, control keys, and so on. The User Portability Utilities option covers these. It also avoids mentioning certain key implementation issues: in particular, there is no requirement that multitasking be used for background jobs, subshells, etc. This was done to allow portability to non-multitasking systems like MS-DOS, so that, for example, the MKS Toolkit (see later in this appendix) can be POSIX compliant.

The Desk Top Korn Shell (dtksh) is a standard part of the Common Desktop Environment (CDE), available on commercial Unix systems such as Solaris, HP-UX, and AIX. It is based on a somewhat older version of ksh93. It evolved from the earlier program wksh, the Windowing Korn shell, released by Unix System Laboratories in late 1992. It’s a full Korn shell, compatible with the version that this book describes,^[135] that has extensions for graphical user interface (GUI) programming in the X Window System environment. It is typically found in /usr/dt/bin/dtksh.

dtksh supports the OSF/Motif graphical Toolkit by making its routines available as built-in commands. This allows programmers to combine the Korn shell’s strength as a Unix systems programming environment with the power and abstraction of the Toolkit. The result is a unified environment for quick and easy development of graphics-based software.

There are various GUI development tools that allow you to construct user interfaces with a graphics-based editor rather than with programming language code. But such tools are typically huge, expensive, and complex. dtksh, on the other hand, is inexpensive and unbeatable for the smoothness of its integration with Unix — it’s the only such tool that you can use as your login shell! (Well, almost; see the next section.) It is a definite option for systems programmers who use X-based workstations and need a rapid prototyping tool.

To give you the flavor of dtksh code, here is a script that implements the canonical “Hello World” program by displaying a small box with an “OK” button. It is from the article Graphical Desktop Korn Shell, in the July, 1998 issue of Linux Journal, by George Kraft IV. (See http://www.linuxjournal.com/article.php?sid=2643.) This code should hold no surprises for X and Motif programmers:

#!/usr/dt/bin/dtksh

XtInitialize TOPLEVEL dtHello DtHello "$@"

XmCreateMessageDialog HELLO $TOPLEVEL hello 
            dialogTitle:"DtHello" 
            messageString:"$(print "Hello
World")"
XmMessageBoxGetChild HELP $HELLO 
        DIALOG_HELP_BUTTON
XtUnmanageChild $HELP
XmMessageBoxGetChild CANCEL $HELLO 
        DIALOG_CANCEL_BUTTON
XtUnmanageChild $CANCEL
XtAddCallback $HELLO okCallback exit
XtManageChild $HELLO
XtMainLoop

http://www.cactus.org/~gk4/kraft/george/papers/dtksh/ is Mr. Kraft’s web presentation on dtksh.

The following book is devoted to dtksh: Desktop KornShell Graphical Programming by J. Stephen Pendergast, Jr., published by Addison-Wesley, 1995 (ISBN: 0-201-63375-2). Examples from the book are online at ftp://ftp.aw.com/aw.prof.comp.series/pendergrast.examples.tar.Z. Also available is ftp://ftp.aw.com/aw.prof.comp.series/pend.dtksh1, a text file that provides an overview of the book. Unfortunately, as of this writing, this book is out of print.

Back in 1996, while a computer science graduate student at Princeton University, Dr. Jeffrey L. Korn^[136] wrote tksh. This is an integration of ksh93 with Tcl/Tk. The following quote (from Dr. Korn’s research web page) summarizes it well:

The tksh home page is still at Princeton: http://www.cs.princeton.edu/~jlk/tksh/. It has links to papers and documentation that can be downloaded and printed. However, the link to tksh executables is out-of-date. The tksh source is available from AT&T Research as part of the ast-open package, which also contains ksh93 and reimplementations of many other Unix tools. See Appendix C for more information.

The following example script, from the USENIX paper on tksh, is called watchdir:

# Tksh Demo
# Jeff Korn
#
# This script keeps track of visited directories and shows the files
# in the current directory.   You can double-click on files and 
# directories.  The script should be used interactively, so to run:
#   $ tksh
#   $ . scripts/watchdir

function winsetup
{
    pack $(frame .f)
    frame .f.dirname -relief raised -bd 1
    pack .f.dirname -side top -fill x
    pack $(frame .f.ls) $(frame .f.dirs) -side left 
    label .f.dirname.label -text "Current directory: "
    label .f.dirname.pwd -textvariable PWD
    pack .f.dirname.label .f.dirname.pwd -side left

    scrollbar .f.ls.scroll -command ".f.ls.list yview"
    listbox .f.ls.list -yscroll ".f.ls.scroll set" -width 20 -setgrid 1
    pack $(label .f.ls.label -text "Directory Contents") -side top
    pack .f.ls.list .f.ls.scroll -side left -fill y -expand 1

    scrollbar .f.dirs.scroll -command ".f.dirs.list yview"
    listbox .f.dirs.list -yscroll ".f.dirs.scroll set" -width 20 -setgrid 1
    pack $(label .f.dirs.label -text "Visited Directories") -side top
    pack .f.dirs.list .f.dirs.scroll -side left -fill y -expand 1
    bind .f.dirs.list "<Double-1>" 'cd $(selection get)'
    bind .f.ls.list "<Double-1>" 'tkfileselect $(selection get)'
}

function tkfileselect
{
    [[ -d "$1" ]] && tkcd "$1"
    [[ -f "$1" ]] && ${EDITOR-${VISUAL-emacs}} "$1"
}

function tkcd
{
    cd $1 > /dev/null || return
    .f.ls.list delete 0 end
    set -o markdirs
    .f.ls.list insert end .. *
    [[ ${VisitedDir["$PWD"]} == "" ]] && .f.dirs.list insert end "$PWD"
    VisitedDir["$PWD"]=1
}

typeset -A VisitedDir
winsetup > /dev/null
alias cd=tkcd
tkcd .

What’s nice about tksh, besides the interesting blend of complementary technologies, is that it brings Tk programming out to the shell level. Graphics programming with Tk is much higher level than with the Motif toolkit; thus the learning curve is easier to climb, and the scripts are easier to read and write.

Many of the Open Source Unix-like systems, such as GNU/Linux, come with the Public Domain Korn Shell, pdksh. pdksh is available as source code; start at its home page: http://www.cs.mun.ca/~michael/pdksh/. It comes with instructions for building and installing on various Unix platforms.

pdksh was originally written by Eric Gisin, who based it on Charles Forsyth’s public-domain clone of the Version 7 Bourne shell. It is mostly compatible with the 1988 Korn shell and POSIX, with some extensions of its own.

Its emacs editing mode is actually more powerful than that of the 1988 Korn shell. Like the full Emacs editor, you can customize the keystrokes that invoke editing commands (known as key bindings in Emacs terminology). Editing commands have full names that you can associate with keystrokes by using the bind command.

For example, if you want to set up CTRL-U to do the same thing as CTRL-P (i.e., go back to the previous command in the history file), you could put this command in your .profile:

bind '^U'=up-history

You can even set up two-character escape sequences, which (for example) allow you to use ANSI arrow keys as well as control characters, and you can define macros, i.e., abbreviations for sequences of editing commands.

The public domain Korn shell’s additional features include alternation wildcards (borrowed from the C shell) and user-definable tilde notation, in which you can set up ~ as an abbreviation for anything, not just user names. There are also a few subtle differences in integer expression evaluation and aliasing.

pdksh lacks the following features of the official version:

Although it lacks most ksh93 extensions, pdksh is a worthwhile alternative to the C and Bourne shells.

bash is probably the most popular “third-party” shell that is available for Unix and other systems. In particular, it is the default shell on GNU/Linux systems. (It also comes on the “freeware” CD with Solaris 8.) You can get it from the Internet, via anonymous FTP to ftp.gnu.org in the directory /pub/gnu/bash. You can also order it from its maker at the address listed here.

Bash was written by Brian Fox and Chet Ramey. Chet Ramey currently maintains it. Its name is in line with the FSF’s penchant for bad puns: it stands for Bourne-Again Shell. Although bash is easily available and you don’t have to pay for it (other than the cost of media, phone calls, etc.), it’s not really public domain software. While public domain software doesn’t have licensing restrictions, the FSF’s software does. But those restrictions are diametrically opposed to those in a commercial license:^[137] instead of agreeing not to distribute the software further, you agree not to prevent it from being distributed further! In other words, you enjoy unrestricted use of the software as long as you agree not to inhibit others from doing the same. Richard Stallman, the founder of the FSF, invented this intriguing and admirable concept.

These days, the ideals of the Free Software and Open Source movements, the GNU project, and the quality of GNU software are all well known. The most popular GNU system is GNU/Linux, which uses the Linux kernel and GNU utilities to make a complete, fully functional, Unix- and POSIX-compatible computing environment.

bash is fully compatible with the 1992 POSIX standard. It has several of the most important Korn shell features and the C shell features that the Korn shell has appropriated, including aliases, functions, tilde notation, emacs and vi editing modes, arithmetic expressions, job control, etc.

The overlap of features between bash and the Korn shell has increased in recent years. It includes many ksh93 features. But it is not an exact ksh clone. The bash FAQ, published monthly by Chet Ramey, lists the following differences between bash and ksh93. Items enclosed in square brackets ([...]) are listed in this book, but not in the FAQ.

The following new things in ksh93 are not available in bash 2.05:

The following new things in ksh93 are present in bash 2.05:

bash has many features of its own that make it a very powerful and flexible environment. Here are some of the highlights:

We’re compelled to say that many users prefer bash to the Korn shell. With the increasing popularity of GNU/Linux and various BSD-derived systems, it’s not clear which shell has the larger user base. In any case, bash is definitely an excellent choice. We recommend the book Learning the bash Shell by Cameron Newham and Bill Rosenblatt, published by O’Reilly & Associates. (It is based on the first edition of this book.)

zsh is a powerful interactive shell and scripting language with many features found in ksh, bash, and tcsh, as well as several unique features. zsh has most of the features of ksh88 but few of ksh93. It is freely available and should compile and run on just about any modern version of Unix. Ports for other operating systems are also available. The zsh home page is http://www.zsh.org. The current version is 4.0.2.

In this section we cover:

grep foo **/*.c

print **/core

print *(.)

print *(/)

print **/core(.)

print /tmp/**/*(U)

print /tmp/**/*(U=)

print ~/**/*(Lm+10)

print ~/**/*(UW/)

autoload -U compinit
compinit

g@sifl:pts/7% man zsh[TAB]
zsh           zshcompctl    zshcontrib    zshmodules    zshzftpsys  
zshall        zshcompsys    zshexpn       zshoptions    zshzle      
zshbuiltins   zshcompwid    zshmisc       zshparam

g@sifl:pts/2% kill [TAB]
 9652 pts/2    00:00:00 zsh                                                    
 9653 pts/2    00:00:00 ps

g@sifl:pts/7% finger o[TAB]
odin	omg	oof        operator	orb

g@sifl:pts/7% finger oof@[TAB]
brak	localhost 	sifl 	zorak

compdef _pids pstack

g@sifl:pts/7% pstack [TAB]
13606 pts/7    00:00:00 zsh                                                    
13607 pts/7    00:00:00 ps

g@sifl:pts/7% tar --v[TAB]
--verbose      --verify       --version      --volno-file

g@sifl:pts/2% RPROMPT='%~'
g@sifl:pts/2%                                    ~/src/xemacs-21.1.14

autoload -U promptinit
promptinit

g@sifl:pts/2% prompt -l                                             ~
Currently available prompt themes:
adam1 adam2 bart bigfade clint elite2 elite fade fire off oliver
redhat suse zefram

g@sifl:pts/7% prompt -s bart                                        ~
sifl [prompt -s bart] ~                              01-10-04 11:58PM
g@sifl:pts/7%                                                       ~

The proliferation of the Korn shell has not stopped at the boundaries of Unix-dom. Many programmers who got their initial experience on Unix systems and subsequently crossed over into the PC world wished for a nice Unix-like environment (especially when faced with the horrors of the MS-DOS command line!), so it’s not surprising that several Unix shell-style interfaces to small-computer operating systems have appeared, Korn shell emulations among them.

In the past several years, not just shell clones have appeared, but entire Unix “environments.” Two of them use shells that we’ve already discussed. Two others provide their own shell reimplementations. Providing lists of major and minor differences is counterproductive. Instead, this section describes each environment in turn (in alphabetical order), along with contact and Internet download information.