The spawn command follows the “usual” rules in finding programs to invoke. Both relative and absolute filenames are acceptable. If a filename is specified with no directory at all, the value of the environment variable PATH is treated as a list of directories and each directory is searched until the given file is found. This searching is performed by the operating system and behaves identically to the way that programs are found from shells such as the Bourne shell and the C shell.

spawn /bin/passwd                            ;# absolute
spawn passwd                            ;# relative

In some cases, naming programs absolutely is a good idea. In other cases, relative names make more sense. If you do use relative names, it is a good idea to set PATH explicitly. For example:

set env(PATH) "/bin:/usr/bin"

Setting the path avoids the possibility of picking up local versions of utilities that users might otherwise have on their paths. Users at other sites may need to change the path, but the single definition at the top of a script makes the path easy to change.

While resetting the path is easy enough, there are circumstances when it makes more sense to refer to programs absolutely. For example, your system may have several versions of a utility (e.g., BSD, SV, POSIX, GNU). Naming a utility absolutely may be safer than using the path.

Rather than embedding a name literally, use variables so that all the references in a script can be changed easily if necessary. For example, you might localize a particular telnet as:

set telnet /usr/ucb/telnet

Later in the script, telnet would then be started this way:

spawn $telnet

Sets of programs that live in different subdirectories under a common directory could be localized with separate variables such as in this example:

set root "/usr/kidney"
set bindir "$root/bin"
set testdir "$root/test/bin"
set demoprog "$bindir/nephron-demo"

Scripts that use these initializations would have spawn commands such as:

spawn $bindir/control
spawn $testdir/simulate
spawn $demoprog

If you have a number of scripts using these definitions, they can be stored in a common file that is sourced by the scripts at start-up.

Like Tcl’s exec command and the C-shell, spawn also supports the tilde notation. A tilde preceeding the first component of a filename is understood to mean the home directory corresponding to the user named by that component. For example, the following command spawns the pike program from the files of a user named shaney:

spawn ~shaney/bin/pike

The previous chapter demonstrated how to open files or devices and send commands to them with puts. This technique calls upon another program to do the handling of the device. At first glance, you might consider this inefficient. Why run two programs when one will do? Consider that reusing another program allows you to isolate all the device-specific problems in one place. If the tip program already knows about serial devices and how, for example, to choose among baud rates, why burden other programs with the same knowledge?

Ideally, if you just had one program that knew everything about, say, your serial devices, you would not need any others. Or perhaps, other programs could call upon your serial device program when they needed to access a serial device. This is analogous to the concept of device drivers. Unfortunately, real device drivers are not high-level enough to isolate out the device dependencies for our purposes.

It is all too common to have numerous programs (kermit, procomm, tip, cu, etc.) that all do the same thing. The reason for having multiple programs to do the same thing is that one has features that another does not have and vice versa. So you keep them all around. Not only is this a problem when you require both features at the same time, but it is a problem when you upgrade or modify your serial device. For example, if you change some phone numbers in your kermit scripts, you also have to change them in any procomm scripts. Some of these programs use a database but none use the same one, so you have to change multiple databases, too.

Expect tries to avoid this quagmire by reusing programs and their knowledge. Expect does not have to be told any device-specific information—it relies entirely upon the device-specific program. If you want to communicate with a serial device, Expect can spawn tip (or kermit, etc.). To communicate with sockets, Expect can spawn telnet. And so on. This works because each of these devices is controlled by interactive programs which in turn can be controlled by Expect.

If you have a device unique to your machine with its own interactive interface, Expect can use it. By unburdening Expect from a lot of device specific information, it is simpler to use, plus you get to use the interface with which you are already familiar.

In UNIX, devices are files. Or at least, it is convenient to think that they are. Devices appear in the file system, and file-like operations can be performed on them. Expect uses these beliefs to support operations like expect and send.

In fact, these can be applied to files as well. In the previous chapter, I described how you can make any file look like a spawned process—for example, by spawning a process to read it with cat. A more direct way is possible.

Tcl’s open command is capable of opening files. The open command returns a file identifier which can be used as an argument to gets, puts, etc. Using the -open argument of spawn, the file id can be turned into a spawn id.

set file [open "/etc/passwd" r]
spawn -open $file
expect . . .

This example opens the /etc/passwd file and searches through it until the given pattern appears. After the spawn, the file behaves exactly like a process. While there is no real process, for consistency, the file must be waited for after being closed. The value returned by wait always indicates a normal exit and can be ignored. This allows you to write code in which it does not matter whether a real process was spawned or not.

The first two commands in the fragment above can be condensed to one:

spawn -open [open "/etc/passwd" r]

Once the file has been passed to spawn, it should not be accessed by gets, puts, or any other Tcl function. Expect and Tcl do not share buffers internally, so joint access should generally be avoided. The file will be closed automatically when the spawn id is closed. If the file must be left open after the spawn id is closed, use -leaveopen instead of -open.

The spawn id in this example cannot be written to (with send) because open only opened the file for reading. To open for writing, the w flag should be used. "w+" allows reading and writing. Several variations on this exist. Read Tcl’s open documentation for the complete details.

While the spawn command can convert a Tcl file to a spawn id, it is also possible to do the opposite. On page 303, I will describe how to convert a spawn id to a Tcl file identifier.

Normally, Expect calls upon tty-aware programs such as tip or cu to interact with ttys. However, using the technique described in the previous section, it is possible to open ttys or other devices directly. For example, if you have a tty port named /dev/ttya, you can open it as follows:

spawn -open [open /dev/ttya w+]

Unfortunately, that is not the whole story. Tty devices must be initialized. This is something that a program such as tip would do for you. Without such a program, you must do it yourself. In the interest of generality, Expect performs no special handling depending upon the type of file it has been handed. That is up to you.

You can initialize a tty using stty with the appropriate arguments. The only hard part is figuring out the arguments. There is no standard, and the parameters can vary depending on your task.

For most tasks, you want to disable special character processing. You also want to disable echo. These are accomplished as follows:

stty raw -echo < /dev/ttya

Flags such as -echo apply to the tty input. Unlike a traditional tty to which you log in, this tty is being used to “go out”. So the tty’s input (in the sense that stty cares) is provided by the remote side. For example, if you use the tty to connect to another serial port, then the output of that serial port is the input of this one.

If the serial port generates parity, you may need to handle that either by disabling it or telling stty to ignore it. Another stty command suffices:

stty istrip < /dev/ttya

The istrip flag strips off the parity bit. If you need an eight-bit connection, you can modify the terminal modes of both the tty and the remote serial port after you are logged in.

Note that raw is a conglomeration of other stty modes. This includes parity on some systems, so you may have to issue the stty commands separately as I have shown here.

The dozens of flags supported by stty vary from system to system. If the stty man page is unenlightening, examine your tty modes while your are using tip (or whatever program you are trying to simulate). This will tell you what the correct configuration should be.

On some systems, bugs in the operating system prevent correct operation of "spawn -open [open . . .]" on some special files. For example, on SunOS, Expect cannot detect an eof from a fifo. Yet another operating system bug prevents AIX 3.2 from detecting input from physical tty devices.^[46]

Fortunately, OS bugs like these are infrequent but Expect does a number of unusual things and you must gird yourself to work around such bugs. In the next section, I will describe a workaround for the fifo problem. A similar workaround can be applied to the other problem.

The spawn command does not provide any facility for redirection or pipelines because the need for it is almost non-existent. Automating interactive programs virtually always require much more sophisticated handling of input and output, such as looking at the output before deciding on the next input.

Rather than burden the spawn command with features that are almost never used, it is easier to call upon existing programs in those rare occasions. There are a variety of programs that support redirection including Tcl’s open command. Tcl’s open command is also capable of building pipelines.

In the previous section, I noted that fifos cannot be handled with spawn on some operating systems. The following command interposes a cat process. The "|" indicates that the following file should be started as a process rather than a file that is simply read or written.

spawn -open [open "|cat -u $fifo" r]

While seemingly redundant, the fifo eof is now handled by cat which converts this to an eof that is detectable by Expect.^[47] The same solution works with other devices that are not supported by select or poll.^[48]

Additional processes can be strung together in the first argument to open by separating them with "|" symbols. Bidirectional processes and process pipelines can be generated. They require the w+ flag to indicate that they will be both read and written.

All of the files, processes, and pipelines opened by open are opened without a terminal interface. For many programs, this is a problem. They expect to be dealing with a terminal. For example, they may want to change the terminal modes. This will fail if there is no terminal involved.

In contrast, each spawned process has a terminal interface associated with it. To programs, the interface “feels” like a real tty is behind it. The program can tell the tty to echo or not echo, for example. But the tty does not physically exist. Rather, it is simulated, and for this reason is known as a pseudoterminal or pty (pronounced “pity”) for short. With a pty, interactive programs work properly.

When spawn is called with -open, no pty is provided. In the rare cases that a pty is needed along with a process pipeline or redirection, /bin/sh can be spawned before invoking whatever is needed. For example, in Chapter 7 (p. 174), I described how spawn normally combines the standard output and standard error. In contrast, the following command separates the standard error of a process so that it is written to a file.

spawn /bin/sh -c "exec 2> error.out; exec prog"

It works as follows: /bin/sh is spawned. The -c flag tells /bin/sh to execute the following argument as a command. The argument is composed of two commands that will be executed sequentially.

The first command is "exec 2> error.out“. This command directs the shell to associate file descriptor 2 (i.e., the standard error) with the file error.out.

The next command is "exec prog“. This command runs prog. Its input and output are still associated with the input and output of the shell (from /bin/sh) which in turn are associated with the spawn id. But the standard error remains tied to the file error.out. The exec in front of prog tells the shell that prog can take over the process resources of the shell. In effect, the shell exits leaving prog in its place.

While initially this may look rather confusing and complex, the result effectively leaves only the intended process running. The shell goes away after setting up the indirection. More complex redirection and pipelines can be constructed but they usually share the same underlying ideas.

Expect normally works with programs that read either from the standard input or /dev/tty. Some programs do not read their input in this way. A good example is the xterm program. xterm is an X Window System client that provides a shell in a terminal emulator. In this section, I will describe three different ways to control xterm.

The xterm program reads user input from a network socket. The standard input and /dev/tty are both ignored. Hence, spawning xterm in the usual way is fruitless.

spawn xterm                   ;# WRONG

Interacting in this way—with no special knowledge of xterm—requires a program that can drive X applications the way Expect drives character-oriented programs. Such programs exist. However, discussion of them is beyond the scope of this book.

Instead of attempting to control an xterm, it often suffices to have an xterm execute an Expect script. For example, suppose you want to be able to pop up a window that automatically runs the chess script defined in Chapter 10 (p. 234). The following command would suffice:

xterm -e chess.exp

The xterm continues to take input in the usual way. Therefore it is even possible to have scripts that accept user input. For example, the auto-ftp script defined in Chapter 3 (p. 83) could be started up with the following command. Once running, it is controllable from the keyboard just the way an xterm normally is.

xterm -e aftp.exp

Both of these examples give up the possibility of controlling the xterm from another script. It is possible to do this by having xterm run an open-ended script such as kibitz. I will present an example of this in Chapter 16 (p. 355).

A third way to control xterm is to spawn it so that the script replaces the process that xterm normally spawns internally. When xterm starts, it no longer starts a new process but talks to Expect. Expect reads what the user types and tells the xterm what to display.

This is a little more complicated, but it allows a script the ability to start multiple xterms and interact with them. Rather than the xterm driving Expect, Expect drives the xterm.

In order to talk to an xterm in this way, Expect must obtain a pty and pass it to the xterm. Expect will communicate using one end of it (the master), and the xterm will communicate using the other end (the slave).

When a process is spawned, Expect allocates the pty, creates the process, and arranges for the process to use the pty for its standard input among other things. However, as I mentioned earlier, xterm does not read its standard input. xterm normally spawns its own process. It is possible to start xterm so that it does not spawn a process but instead interacts with an existing one.

To do this, xterm requires that the pty name and file descriptor be passed to it when it is invoked. Expect normally allocates the pty as the process is created, but this is too late for xterm. xterm wants the pty name on the command line. The -pty flag causes the spawn command to generate a pty with no new process.

spawn -pty

During the spawn command, the name of the slave end of the pty is written to the variable spawn_out(slave,name). This occurs whenever a process is spawned, whether or not the -pty flag is present. (In Chapter 14 (p. 315), I will show another use for this variable.)

The pty must be initialized to raw mode and have echoing disabled.

stty raw -echo < $spawn_out(slave,name)

In X11R5 and earlier versions, the flag to run xterm in slave mode is rather peculiar. The flag is -S. It is followed by two characters denoting the suffix of the pty name and an integer representing the file descriptor. For example, if the slave is named /dev/ttyp0 and the file descriptor is 6, xterm is started with the flag "-Sp06“.

The two-character format does not support all pty names and because of this, many vendors have modified xterm. For example, some versions of xterm use a 0 to pad suffixes that would otherwise be one character. The following code generates the two-character suffix, padding if necessary:

regexp ".*(.)(.)" $spawn_out(slave,name) dummy c1 c2
if {[string compare $c1 "/"] == 0} {
    set c1 "0"
}

There is no backward-compatible solution for ptys that use more than three characters for identification. However, as of X11R5, xterm does not actually use the information. So the code above suffices (unless your vendor has made radical changes).^[49]

xterm also requires the open file descriptor corresponding to the slave. This information is written to the variable spawn_out(slave,fd) by the spawn command.

Now the xterm can be started. It is not necessary to use spawn since the pty has already been allocated. The exec command is appropriate. An ampersand forces it to run in the background so the script can go on to do other things.

exec xterm -S$c1$c2$spawn_out(slave,fd) &

Like spawn, the exec command returns the process id of the xterm.

Once the xterm is running, Expect should close its copy of the slave file descriptor. This is done by invoking the close command with the -slave argument.

close -slave

When xterm starts this way, it immediately sends back an X window id on a line by itself. Extensions such as TkSteal can use the X window id to provide reparenting, allowing an xterm to appear in a Tk widget hierarchy. If you do not want the X window id, just discard it.

expect "
"       ;# match and discard X window id

At this point, the xterm can now be controlled. The send command will print strings on the xterm display. The expect command will read input from the user (including insertions made using the mouse).

For example, the following code spawns a shell and lets the user interact in the xterm until X is pressed. Then the user is prompted for a return, after which the xterm is killed and the script exits. (The "interact -u" ties the xterm and the shell together—this will be explained further in Chapter 16 (p. 350).)

# assume xterm is initialized, spawn id is in $xterm,
# and xterm pid is in $xterm_pid

spawn $env(SHELL)

interact -u $xterm "X" {
    send −i $xterm "Press return to go away: "
    set timeout −1
    expect −i $xterm "
" {
        send −i $xterm "Thanks!
"
        exec kill $xterm_pid
        exit
    }
}

A real example that is more sophisticated than this one will be shown in Chapter 16 (p. 361).

All of the examples so far have assumed that spawn always succeeds. The bad news is that spawn does not always succeed. The good news is that it only fails in peculiar environments or in peculiar situations. In this section, I will describe the meaning of “peculiar” and how to check whether spawn succeeded or not.

The spawn command normally returns the process id of the newly spawned process.^[50] This is generally of little value since spawned processes are more easily manipulable by their spawn ids. However, it is occasionally useful to be able to kill a process using its process id rather than going through some long interaction.

set pid [spawn program]
. . .
# some time later
exec kill $pid

Once killed, the process connection should be recycled by calling close and wait.

Running out of various system resources can cause spawn to fail. For example, spawn allocates dynamic memory as well as a logical terminal interface. Failures like this can be caught using Tcl’s catch command:

if [catch "spawn program" reason] {
    send_user "failed to spawn program: $reason
"
    exit 1
}

Even if spawn does not return an error, that is not a guarantee that it was entirely successful. To understand why, it is necessary to explain a little of how spawn is implemented.

The spawn command follows the traditional UNIX paradigm for running a new program. First, Expect forks. Forking is the UNIX way of generating a new process. Initially, the new process is still running Expect code. This allows Expect to prepare the environment appropriately for the new program. The last step is for Expect (in the new process) to overlay itself with the new program. At this point, the original Expect process is still running, and the new process is running the requested program.

This last step of loading the program can fail if, for example, the program does not exist. If it does not exist, the new process must communicate this back to the Expect process. Ironically, the failure of the program to be found can be communicated but not its success. The reason is that the very act of successfully running the program removes any functionality of the earlier program (i.e., Expect). Thus, the new program has no idea how to signal success or even that it should.

Because of this, the original Expect process cannot wait around for a possible failure. The spawn command returns immediately. If the process does fail however, the new process sends back an error report in such a way that the Expect process hears it at the next convenient moment—the first expect command.

Here is an example of interactively running Expect and attempting to spawn a non-existent program:

% expect
expect1.1> spawn noprog
spawn noprog
18961
expect1.2> expect -re .+
noprog: No such file or directory
expect1.3> puts "expect_out = <$expect_out(buffer)>
"
expect_out = <noprog: No such file or directory>

The error message is returned exactly the way any other output from the spawned process is—via expect_out. Differentiating between an error from the shell and real program output from the process may be difficult, if not impossible. The recognition problem is identical to what a real human faces when using interactively starting programs from the shell. How one differentiates between an error message and real program output is left to the user.

The format of the error message is as shown above. It is the program name, followed by a colon and space, followed by your particular system’s standard error message. Other messages are possible in other scenarios, such as if the file exists but is not executable.

Checking the return value of spawn (as shown above with catch) is a good idea if you want your code to be bulletproof. These kinds of errors are often due to transient conditions that may go away if the operation is retried, such as a lack of memory.

On the other hand, checking spawn’s success via the first expect is less valuable. For example, if a standard utility such as /bin/sh is being spawned, there is little point in checking if it succeeded. If it did not, the computer has such severe problems that few programs will be able to continue to run.

The primary circumstance in which to check the first expect after a spawn is when the program is unknown at the time the script is written. For example, if a user can type in arbitrary command names dynamically, these names should be checked. Note that using "file executable" is a reasonable test but it is not guaranteed since there is a window between the time the file can be tested and the time it is executed, during which the file can change.

In the previous example, all of the commands were entered interactively. When this is done, the return values of all commands are automatically printed by Expect. In the case of the spawn command, the return value was the process id. In that example, the process id was 18961. The command also echoed itself as a side effect. This is not the return value. If a spawn command appears in a script, the process id will no longer be printed to the standard output, but the command itself still echoes.

This echoing is intended as a convenience for simple scripts, much as the echoing performed by the expect command itself is. Both of these can be disabled with the log_user command. However, the log_user command disables all of the spawned program’s output. To disable just the echoing produced by the spawn command, use the -noecho flag. This flag affects nothing else. Here is the previous example repeated using that flag.

% expect
expect1.1> spawn -noecho noprog
18961
expect1.2> expect -re .+
noprog: No such file or directory
expect1.3> puts "expect_out = <$expect_out(buffer)>
"
expect_out = <noprog: No such file or directory>

Here is the same example, but using "log_user 0" instead of -noecho. Notice that both spawn and expect no longer echo anything.

% expect
expect1.1> log_user 0
expect1.2> spawn noprog
18961
expect1.3> expect -re .+
expect1.4> puts "expect_out = <$expect_out(buffer)>
"
expect_out = <noprog: No such file or directory>

In all cases, spawn still produces a return value. This and all other return values disappear if run from a script.

Most non-interactive programs behave differently depending on whether their output goes to the terminal or is redirected. In particular, output to a terminal normally appears as soon as a full line is produced. In contrast, output that is redirected to a file or a process is buffered by much larger amounts in the name of efficiency. This difference in buffering is automatically chosen by the UNIX stdio system.

Unfortunately, this means that some simple UNIX commands do not work as nicely as you might expect. For example, suppose a slow source is sending output to a fifo called /tmp/fifo and you want to read it using od and then pipe it into a pager such as more. The obvious shell command to do this is:

od -c /tmp/fifo | more

Alas, the stdio system compiled into od sees that its output is a pipe so the output is automatically buffered. Even if od receives a complete line, od does not send anything down the pipe until the buffer has been filled.

There is no way to fix od short of modifying and recompiling it. However, by using Expect, it is possible to make od think that its output is destined for a terminal. Since Expect connects processes to a pty, this is sufficient to satisfy the stdio system, and it changes to line-buffered I/O.

A script to do this is simple. All it has to do is spawn the process and wait for it to finish. Here is a script which does this, called unbuffer:

#/usr/local/bin/expect —
# Name: unbuffer
# Description: unbuffer stdout of a program

eval spawn -noecho $argv
set timeout −1
expect

The original command can now be rewritten to use unbuffer:

unbuffer od -c /tmp/fifo | more

Most other non-interactive UNIX utilities share the problem exhibited here by od. Dealing with the stdio system is one of the few times where it makes sense to run Expect on non-interactive processes.

Historically, the console was a dedicated terminal to which critical messages were sent concerning the status of the computer. The idea was that a person would be watching at all times and could take immediate action if necessary. With modern workstations, there is no physical console with a dedicated operator. Instead, the console is simulated with a dedicated window. For example, in the X window system, the command "xterm -C" starts an xterm window and tells the operating system to send all console messages to it.

Expect can do the same thing with the spawn command. The -console flag redirects all console messages so that they appear to be generated from a spawned process. It is sufficient to spawn any process. Even cat will do.

spawn -console cat

A simple use for this flag is to watch for errors from device drivers. For example, when performing backups, errors writing to the backup media may be sent to the console rather than the backup program. This is a consequence of the way certain drivers are written and is surprisingly common.

By spawning the backup program using the -console flag, it is possible to catch problems with the backup that might not otherwise be reported. In Chapter 17 (p. 380), I will describe how to make an Expect script actively look for a skilled user to fix any problems encountered, and initiate a session for the user connected to the spawned process automatically.

The -console feature can only be used by one program at a time. It is also a relatively recent addition to UNIX. Therefore, it is not yet supported by all systems. The -console flag is ignored on systems that do not support the ability to redirect console output.

Pty modes can have a big effect on scripts. For example, if a script is written to look for echoing, it will misbehave if echoing is turned off. Suppose a script is driving a shell that prompts with a bare "%“. If the script sends the command "whoami “, the shell might return "whoami don %“. In this case, the response could be matched with:

expect -re "
(.*)
% "

If the shell did not echo its input, the shell would return "don %“. But the expect command just shown fails to match this output.

For this reason, Expect forces “sane” pty modes by default. In fact, the sane flag is known to stty, the program which configures ttys and ptys. The particulars of sane differ from system to system; however, sanity typically implies characteristics as echoing, and recognition of erase and kill characters. Expect invokes stty to set the pty, so you can be assured that Expect’s version of sanity is just what your local stty thinks. If for some reason you believe stty’s understanding of sane is flawed and you are not in the position to change it (i.e., you do not have the source), you can redefine it when installing Expect on your system. This is covered in the installation procedure.

Unfortunately, one program’s sanity is another program’s gibberish. Some programs have special demands. As an example, it is possible to interact with a shell from inside of the Emacs editor (this has nothing to do with Expect so far). The shell session appears as a normal file (or “buffer” in Emacs-speak) except that when you press return, the current line is sent as a command to the shell and the results are appended to the end of the buffer. This has many benefits. For example, with an Emacs shell session, you can use Emacs commands to directly edit the input and output.

To make the Emacs shell-session work similarly to a session outside Emacs, Emacs changes the pty modes. For example, echoing is disabled so that you can edit the command line before passing it to the shell. Also, newlines produced by programs are no longer translated to carriage-return linefeed sequences. Instead, newlines remain as newlines.

Expect scripts written for the “normal” pty modes could fail if they were to only use Emacs’ idea of pty modes. To avoid this, Expect performs a three-step pty initialization which leaves the pty with a suitable mixture of Emacs and user pty characteristics.

The first step initializes the pty so that it is configured just like the user’s terminal. Next, the pty is forced into a sane state (as I described earlier). In most cases, this changes nothing; however, anything too unusual is reset. This is also important when the process is running from cron where there is no terminal from which to copy attributes in the first place. Finally, any other pty modes are changed according to the requirements of the script.

Each of these steps is controllable. The first step, copying the user’s terminal modes, is done unless spawn is invoked with the -nottycopy flag. The second step, forcing the pty into a sane state, is done unless spawn is invoked with the -nottyinit flag. The third step is only done if the stty_init variable is defined, in which case it is passed as arguments to the stty program.

The order that the flags are given to spawn is irrelevant, but they must appear before the program name. Here are several examples. In each case, prog stands for a program to be spawned.

spawn -nottyinit prog
spawn -nottyinit -nottycopy prog
set stty_init "kill ! susp ?"
spawn prog

The last example sets the kill character to “!” and the suspend character to “?”. Conceivably, this could be useful or necessary for running or testing a particular program. The spawn command does not enforce any kind of pty initialization. It is possible to use -nottycopy and -nottyinit and not define stty_init but this is not a good idea. Ptys are not otherwise initializated by most systems.

These options may seem complex, but in most cases they are not necessary. Going back to Emacs for a moment, the default behavior of spawn allows the correct functioning of Expect scripts. Expect scripts may “look funny” inside of Emacs with respect to character echoing, but then, so do commands such as telnet and rlogin. If you absolutely have to have Emacs look correct, use the -nottyinit flag. However, you must then go to extra effort to make your scripts avoid any dependencies on echoing, line termination characters, and anything else that the Emacs terminal modes affect.

Another example of how stty can introduce unexpected results is with the line kill character. On some UNIX implementations, stty believes the @ is the default line kill character (i.e., pressing it removes all previous characters typed on the line). The @ was a popular convention many years ago. Now, it is just archaic and ^U is much more common. On such archaic systems, sending strings such as "user@hostname " ends up sending only "hostname “.

Yet another problem that occasionally crops up is what to do with parity. On some UNIX implementations, stty believes that parity should be disabled. This confuses programs that work with 8-bit character sets. If you can not fix your local stty, work around the problem by using the -nosttyinit flag or by setting stty_init to -istrip.

Historically, UNIX systems have provided a fixed number of ptys, pre-allocating filenames in the file system for each one. Most versions of UNIX no longer do this, but there are still some that do. With a static set of ptys, it is necessary to search through the list of files. Expect performs several tests on each pty before using it. These tests ensure that no other process is still using the pty.

Usually these tests are very quick, but programs that have misbehaved and are sloppy in their pty allocation and deallocation can force Expect to take up to ten seconds, waiting for a response from a pty that is still in use.^[51] Normally, Expect goes on and continues trying other ptys until it finds one that can be allocated; however, such ptys can cause problems for most other programs. For example, programs that use ptys, such as xterm and Emacs, simply give up when encountering such a pty. If you see this happening, you can try spawning a process with Expect’s diagnostic mode enabled. Expect will then report the ptys it is ignoring and you can verify that each one is in use by a functioning program. In some cases, the program may have exited but left the pty in a bizarre state. Expect’s thorough pty-initialization procedure will reset the pty so that other processes can use it.

You can take advantage of Expect’s ability to fix ptys with the following script called ptyfix.

#!/usr/local/bin/expect--
spawn cat

Or even simpler, just put the following shell command in an alias or menu:

expect -c "spawn cat"

There is no explicit restriction on spawning multiple processes—any number of processes may be running under control of Expect. However, some old—perhaps archaic is a better word—systems do not provide a facility for listening from multiple processes simultaneously. When Expect is installed, it looks for the presence of the select or poll system call. Either of these usually indicates that Expect can listen to multiple processes simultaneously.

Some systems provide select or poll but do not allow them to be used the way Expect needs. In this case, Expect simulates this functionality using the read system call with alarms. When using read, Expect has one major restriction. Only one process can be listened to (with either expect or interact) at a time.

Fortunately, such systems are rare and growing rarer.^[52] Although you cannot run Expect with all of its power on them, you can still get useful work done even by automating one application at a time.

While the spawn command returns a process id, Expect can provide this information at any time by using the exp_pid command. With no arguments, exp_pid returns the process id of the currently spawned process. The process id of a particular spawn id can be returned by using a −i flag. For example:

expect1.1> exp_pid −i $shell
20004

Do not confuse this command with pid. The pid command is a built-in Tcl command that returns the process id of the Expect process itself.

You cannot directly read from or write to spawned processes with puts and gets. In general, there is little need for it because you can emulate the behavior with suitable send and expect commands. Nonetheless, it may be convenient to do so at times.

Earlier, I showed how the -open flag of the spawn command converts a file identifier to a spawn id. The exp_open command does the opposite. It converts a spawn id to a file identifier that may be used with gets and puts. The file identifier will be open for both reading and writing. If exp_open is called with no arguments, it converts the spawn id of the currently spawned process. If called with a −i argument, exp_open converts the given spawn id.

By default, after calling exp_open, the spawn id can no longer be accessed using send and expect. It becomes owned entirely by Tcl and should eventually be closed in the Tcl style and without doing a wait. On some systems, processes return spurious error indications during a close operation. Expect knows to ignore these errors; however, you may have to explicitly catch them from Tcl.

spawn /bin/csh
set file [exp_open]
catch {close $file}

You may have to call flush explicitly after I/O operations because the file commands normally buffer internally. Process output that does not terminate with a newline may be impossible to read unless you disable buffering or read it explicitly with read. In the following example, the first output from telnet is read using read since it does not end with a newline.

% expect
expect1.1> spawn -noecho telnet
4484
expect1.2> exp_open
file5
expect1.3> read file5 7
telnet>

The spawn id can be left open by calling exp_open with the -leaveopen flag. In this case, both the file and the spawn id must be closed explicitly. A wait must be executed. As with the -leaveopen flag in the spawn command, alternation of Tcl and Expect commands is best avoided because Tcl and Expect do not share buffers internally.