I initially did not view Expect as something that would last very long. It struck me as solving a very simple problem but not in the best way possible. Indeed, I originally wrote Expect as an experiment to demonstrate the need for a general way of handling interaction automation. I expected that, having shown the utility of it, the popular shells of the day would all soon incorporate these functions, allowing Expect-like things to be accomplished from the shells without requiring the use of another tool.

But to my surprise that has not been the case. Some shells (e.g., Korn shell, Z shell) provide co-processes but offer no access to pseudo-terminals and no in-line stream pattern matching. Most shells don’t even provide co-processes. Thus, Expect remains very important to shell programmers.

Expect is also useful with environments that have limited or baffling Expect-like functions. For example, Emacs, a popular editor, has actually had the ability to do Expect-like processing for years. However, Emacs is a fairly unusual programming environment and few people do real Emacs programming. Perl is another unusual programming environment, again, with its own Expect-like functionality; however, the implementation is difficult to use and many Perl programmers find it easier to call Expect scripts for these tasks. Perl is also a large language, which makes its use for Expect-like programming all the more formidable for the casual user.^[2]

By comparison, Expect is simple. It really only does one thing. But it does it very well. Everything in Expect is optimized to help you automate interactive programs.

That doesn’t mean it can’t do other things. These other things simply aren’t the focus of Expect. You can use Expect to work with non-interactive applications. Expect rests on top of Tcl which provides a very pleasant environment in which to work. I have implemented and used a number of complex software packages that use Tcl as a scripting language, and I look forward to using any other application that does similarly.

Underneath Expect is Tcl, a small and simple language designed to be embedded in applications. Much of the reason Expect appears so coherent and well thought out is actually due to Tcl.

Tcl is a tour de force. It is powerful yet elegant, drawing a fine line between primitives and extensibility, and between simplicity and overkill. Tcl allowed me to concentrate on the application requirements of Expect. Tcl will allow you to call and mix the Expect primitives in all sorts of interesting ways to control your applications.

The intent behind Tcl matches Expect’s philosophy perfectly. Expect doesn’t need a specialized language. Any generic but extensible control language would have sufficed. However, at the time that I was thinking about writing Expect, no such extensible control language existed. I was irritated at the thought of having to create a language specifically for such a simple application. There was simply no justification for designing yet another language.

I had been thinking about writing an Expect-like program after helping Scott Paisley write a program to automate the initial login and command in a telnet session. The program understood only a few simple commands. For instance, find waited for a single fixed string to arrive. Every session ended in a permanent sort of interact with a single telnet process. There were no variables and no flow control commands such as if. And the program used pipes instead of ptys. The program solved our immediate problem, but it had a lot of special-case coding and I thought it could be generalized. This was in 1987.

For a long time I considered borrowing a shell, integrating the Expect primitives into it, and then re-releasing it. However, shells are not intended to be used this way and I did not have any interest in maintaining a shell once I had stuck my hands in it. At that time, shells were renowned for being messy beasts. For example, the C shell was well known for not having consistent and robust parsing. And the Bourne shell had stimulated the Obfuscated C Code Contest—a contest that actually celebrates and revels in torturous code. I wanted something else.

I was fortunate to be able to attend the 1990 Winter USENIX Conference in Washington DC (January 22-26). I had been thinking about writing something like Expect for several years, and I decided to go there and ask some wizards about what they did for portable language facilities. To my delight, there was a talk at the conference addressing that very topic.

John Ousterhout, a professor at the University of California at Berkeley, had designed a language for embedding into applications—my very need! By the middle of the talk, I knew I wanted to try using it. At the end of talk, when he said it was freely available, I swooned.

Four days later, I had a copy of Tcl. I couldn’t believe it. It was not only everything John had promised, it was also easy to use and well documented. By February 7 (eight days after first downloading Tcl), I had a working, albeit primitive, Expect. It had the core of what Expect has today: send, expect, spawn, interact, plus a logging function. At that time, the compiled Tcl code was about 48K, while Expect added on another 12K. The idea of a control language being larger than the application seemed peculiar, but it worked too well to go back to the old ways of designing ad hoc interfaces.

Even with minimal functionality, it clearly suggested some interesting uses and I thought that it might be nice to inform others. I had a USENIX Call-For-Papers hanging next to my workstation. It said:

It was February 8, one day after the deadline. I sat down, banged out the requested extended abstract, and sent it in on the 9th.^[3] It was accepted with top marks by all the referees and appeared at the very next conference. No person was more astonished by this rapid publication describing a Tcl-based application than John. (John published another Tcl-related paper at the 1991 Winter USENIX Conference—leading to Tcl papers at three consecutive USENIX conferences and a high profile very early in its life.)

During 1990 and 1991, my local ftp server distributed over 2500 copies of Tcl, all for the purpose of running Expect. I like to think that Expect was a catalyst in the success of Tcl, but even without Expect, Tcl would have caught on eventually. Tcl is now used by thousands of applications and millions of users. Some of the Tcl extensions (Tk, in particular) stand on their own, and like Expect, allow people to do things more easily than before and in many cases things that they would never even have tried.

Since writing Expect, I’ve used Tcl to build several other applications and extensions. While none of them are as general purpose as Expect, the results are wonderful. Even for building specialized tools, Tcl is a joy to work with. I hope never to go back to yacc and lex again.

While the focus here is Expect, not Tcl, I believe that this book is worth reading even if you are learning Tcl for other reasons—perhaps contemplating putting it to use in your own applications. Expect is a good example of what can be accomplished with Tcl. Plus there are many techniques—such as the debugger and the signal handler—that while invented for Expect, can be applied to just about any other Tcl application. By reusing my efforts (and in some cases, avoiding my mistakes), it will be that much easier when designing your own Tcl-based applications.

I owe a large amount to the many people that used Expect and gave me excellent feedback. There are literally hundreds of people who made contributions. Many suggestions, although cavalierly made, developed into important aspects of Expect. Some people donated sizable chunks of code while yet others debated with me on philosophical aspects.

Thanks to Scott Paisley who initially sowed the seeds for the idea and then ended up listening to me rave on about it for several years before I took any action. Scott also wrote the dialback script that appears in Chapter 1 (p. 4).

I would also like to thank (in no particular order) John Conti, Mike Gourlay, Frank Terhaar-Yonkers, Jerry Friesen, Brian Fitzgerald, Mark Diekhans, Kevin Kenny, Rob Nagler, Terrence Brannon, Ken Manheimer, Steve Clark, Keith Eberhardt, Brian Woodson, Pete Termaat, Sandy Ressler, Hal Peterson, Mike O’Dell, Steve Legowik, Scott Hess, Mark Diekhans, David Vezie, Keith Hanlan, Rick Lyons, Jeff Okamoto, Tom Tromey, Phil Shepherd, Michael Grant, Bob Proulx, Marty Leisner, Ting Leung, Karl Lehenbauer, Przemek Klosowski, Bill Tierney, Steve Summit, Rainer Wilcke, Alon Albert, Jay Schmidgall, Corey Satten, Konrad Haedener, Tor Lillqvist, Pasi Kaara, Kartik Subbarao, Steve Ray, Karl Vogel, Seth Perlman, Dave Mielke, Brian Bebeau, Bob Bagwill, Dan Bernstein, Pascal Meheut, Rusty Wilson, Jonathan Kamens, Bill Houle, Todd Richmond, Mark Weissman, Chris Matheus, Lou Salkind, Bud Bach, R.K. Lloyd, Chip Rosenthal, David D ‘Zoo’ Zuhn, John Rouillard, and numerous others whose names litter my log books and change files. Of special note are Rob Savoye, who automated the Expect configuration procedure; the people who wrote Autoconf on which Rob’s work rests; Henry Spencer, who wrote the regular expression engine used by Expect and Tcl; and Arnold Robbins, who wrote the POSIX date formatter.

And thanks to John Ousterhout for Tcl and for Tk. It is difficult to describe the amount of work that he has put into making them possible, making them so usable, and then making them freely available. John is a renaissance computer scientist. He programs with great style—his code is a pleasure to read, and it is documented in a thoughtful and readable fashion. Tcl is a great accomplishment, yet it is only one of many for John. John is a continuing source of excellent ideas followed up by gargantuan amounts of effort.

A significant amount of support for the development of Expect was provided by the National Institute of Standards and Technology (NIST). The development of Expect fit in well with the charter of NIST, an agency of the U.S. Department of Commerce, whose goal is, after all, to promote commerce and assist industry in the development of technology. Even though Expect was never specifically funded as a separate project at NIST, I was permitted to work on Expect to the extent that it directly aided projects at NIST because it kept solving problems that would crop up. Thanks to Cita Furlani, Selden Stewart, Jeane Ford, and Howard Bloom for allowing the flexibility and freedom in the technical work to explore Expect.

Much of the funding for Expect actually came out of my own pocket. But the person who really paid the steepest price is Susan Mulroney. She has been especially patient (“Isn’t that thing done yet?!”), allowing me to give up lots of quality time (“Please come home, the toilet is overflowing!”) even after I decided the software wasn’t enough and I had to write a book on it. (“What? Another book!?”) Even worse, she has to put up with my sense of humor.

Writing a book is no simple task. Getting permission from the government alone took eleven months. (Getting permission for the previous sentence took three of them.) With a full-time job in the way (and fixing bugs, making enhancements, and answering everyone’s questions—all after hours), the writing took another two years. This was also delayed by changes in Tcl, Tk, and Expect themselves.

The secret to successful writing is getting the most dedicated, anal, and brutally honest reviewers. My readers nitpicked the manuscript to death, pointing out flaws and incorrect or missing reasoning or explanations. I thank the readers for finding all these problems and keeping me on my toes. Thanks to W. Richard Stevens, Henry Spencer, Brian Kernighan, Gerard Holzmann, Thomas Accardo, Bennett Todd, Miguel Angel Bayona, Tim O’Reilly, Frank Willison, Brent Welch (now writing a Tcl book of his own!), Danny Faught, Paul Kinzelman, Barry Johnston, Rob Huebner, Todd Bradfute, David Dodd, Will Morse, Thomas Brown, Sam Shen, Adrian Mariano, Ken Lunde, Marc Rovner, Jeffrey Friedl, David Rosenfeld, Stavros Macrakis, Jeremy Mathers, Larry Virden, Keith Neufeld, Nelson Beebe, and Jeff Moore. Thanks also to Lennie, Sol, and Susan Libes for proofreading yet another manuscript, evidently under the belief that due to a freak biological accident many years ago, they are guaranteed the right to be my critics forever. Due to a more recent biological event, I must also thank Sue Baughman and Lenore Mulroney for babysitting the Kenna.

Finally, thanks to Sheryl Avruch, Frank Willison, and Clairemarie Fisher O’Leary for production help, Edie Freedman for typography advice as well as a most appropriate and appealing cover, Tanya Herlick and Jessica Perry Hekman for system administration, Brian Erwin and Linda Walsh for creative and inspired sales and marketing (yes, they do more than make t-shirts), Seth Maislin for copyediting, Mike Sierra for FrameMaker assistance, Jennifer Niederst for Nutshell design standards, Michael Kalantarian and Celinda Bormeth for numerous sordid little details, and Tim O’Reilly, who listened patiently to all my questions and solved them with wisdom and common sense.

We have tested and verified all of the information in this book to the best of our ability, but you may find that features have changed (or even that we have made mistakes!). Please let us know about any errors you find, as well as your suggestions for future editions, by writing:

You can also send us messages electronically. To be put on the mailing list or request a catalog, send email to:

To ask technical questions or comment on the book, send email to: