The sys
module is also the
place where the standard input, output, and error streams of your Python programs live:
>>>for f in (sys.stdin, sys.stdout, sys.stderr): print f
...
<open file '<stdin>', mode 'r' at 762210>
<open file '<stdout>', mode 'w' at 762270>
<open file '<stderr>', mode 'w' at 7622d0>
The standard streams are simply preopened Python file objects
that are automatically connected to your program’s standard streams
when Python starts up. By default, all of them are tied to the console
window where Python (or a Python program) was started. Because the
print
statement and raw_input
functions are really nothing more
than user-friendly interfaces to the standard output and input
streams, they are similar to using stdout
and stdin
in sys
directly:
>>>print 'hello stdout world'
hello stdout world >>>sys.stdout.write('hello stdout world' + ' ')
hello stdout world >>>raw_input('hello stdin world>')
hello stdin world>spam
'spam' >>>print 'hello stdin world>',; sys.stdin.readline( )[:-1]
hello stdin world>eggs
'eggs'
Technically, standard output (and print
) text appears in the console window
where a program was started, standard input (and raw_input
) text comes from the keyboard,
and standard error text is used to print Python error messages to
the console window. At least that’s the default. It’s also possible
to redirect these streams both to
files and to other programs at the system shell, as well as
to arbitrary objects within a Python script. On most systems, such
redirections make it easy to reuse and combine general-purpose
command-line utilities.
Redirection is useful for things like canned (precoded) test inputs: we can apply a single test script to any set of inputs by simply redirecting the standard input stream to a different file each time the script is run. Similarly, redirecting the standard output stream lets us save and later analyze a program’s output; for example, testing systems might compare the saved standard output of a script with a file of expected output to detect failures.
Although it’s a powerful paradigm, redirection turns out to be straightforward to use. For instance, consider the simple read-evaluate-print loop program in Example 3-6.
Example 3-6. PP3ESystemStreams eststreams.py
# read numbers till eof and show squares def interact( ): print 'Hello stream world' # print sends to sys.stdout while 1: try: reply = raw_input('Enter a number>') # raw_input reads sys.stdin except EOFError: break # raises an except on eof else: # input given as a string num = int(reply) print "%d squared is %d" % (num, num ** 2) print 'Bye' if _ _name_ _ == '_ _main_ _': interact( ) # when run, not imported
As usual, the interact
function here is automatically executed when this file is run, not
when it is imported. By default, running this file from a system
command line makes that standard stream appear where you typed the
Python command. The script simply reads numbers until it reaches
end-of-file in the standard input stream (on Windows, end-of-file
is usually the two-key combination Ctrl-Z; on Unix, type Ctrl-D
instead[*]):
C:...PP3ESystemStreams>python teststreams.py
Hello stream world Enter a number>12
12 squared is 144 Enter a number>10
10 squared is 100 Enter a number> Bye
But on both Windows and Unix-like platforms, we can redirect
the standard input stream to come from a file with the <
filename
shell syntax. Here is a command session in a DOS console box on
Windows that forces the script to read its input from a text file,
input.txt. It’s the same on Linux, but
replace the DOS type
command
with a Unix cat
command:
C:...PP3ESystemStreams>type input.txt
8 6 C:...PP3ESystemStreams>python teststreams.py < input.txt
Hello stream world Enter a number>8 squared is 64 Enter a number>6 squared is 36 Enter a number>Bye
Here, the input.txt file automates the
input we would normally type interactively—the script reads from
this file rather than from the keyboard. Standard output can be
similarly redirected to go to a file with the >
filename
shell syntax. In fact, we can combine input and output redirection
in a single command:
C:...PP3ESystemStreams>python teststreams.py < input.txt > output.txt
C:...PP3ESystemStreams>type output.txt
Hello stream world Enter a number>8 squared is 64 Enter a number>6 squared is 36 Enter a number>Bye
This time, the Python script’s input and output are both mapped to text files, not to the interactive console session.
On Windows and Unix-like platforms, it’s also
possible to send the standard output of one program to the
standard input of another using the |
shell character between two commands.
This is usually called a “pipe” operation because the shell
creates a pipeline that connects the output and input of two
commands. Let’s send the output of the Python script to the
standard more
command-line
program’s input to see how this works:
C:...PP3ESystemStreams>python teststreams.py < input.txt | more
Hello stream world
Enter a number>8 squared is 64
Enter a number>6 squared is 36
Enter a number>Bye
Here, teststreams
’s
standard input comes from a file again, but its output (written by
print
statements) is sent to
another program, not to a file or window. The receiving program is
more
, a standard command-line
paging program available on Windows and Unix-like platforms.
Because Python ties scripts into the standard stream model,
though, Python scripts can be used on both ends. One Python
script’s output can always be piped into another Python script’s
input:
C:...PP3ESystemStreams>type writer.py
print "Help! Help! I'm being repressed!" print 42 C:...PP3ESystemStreams>type reader.py
print 'Got this" "%s"' % raw_input( ) import sys data = sys.stdin.readline( )[:-1] print 'The meaning of life is', data, int(data) * 2 C:...PP3ESystemStreams>python writer.py | python reader.py
Got this" "Help! Help! I'm being repressed!" The meaning of life is 42 84
This time, two Python programs are connected. Script
reader
gets input from script
writer
; both scripts simply
read and write, oblivious to stream mechanics. In practice, such
chaining of programs is a simple form of cross-program
communications. It makes it easy to reuse
utilities written to communicate via stdin
and stdout
in ways we never anticipated. For
instance, a Python program that sorts stdin
text could be applied to any data
source we like, including the output of other scripts. Consider
the Python command-line utility scripts in Examples 3-7 and 3-8 that sort and sum lines in
the standard input stream.
Example 3-7. PP3ESystemStreamssorter.py
import sys # or sorted(sys.stdin) lines = sys.stdin.readlines( ) # sort stdin input lines, lines.sort( ) # send result to stdout for line in lines: print line, # for further processing
Example 3-8. PP3ESystemStreamsadder.py
import sys sum = 0 while True: try: line = raw_input() # or call sys.stdin.readlines( ) except EOFError: # or for line in sys.stdin: break # raw_input strips at end else: sum += int(line) # was sting.atoi( ) in 2nd ed print sum
We can apply such general-purpose tools in a variety of ways at the shell command line to sort and sum arbitrary files and program outputs (Windows note: on my XP machine, I have to type “python file.py” here, not just “file.py”; otherwise, the input redirection fails):
C:...PP3ESystemStreams>type data.txt
123 000 999 042 C:...PP3ESystemStreams>python sorter.py < data.txt
sort a file 000 042 123 999 C:...PP3ESystemStreams>python sorter.py < data.txt
sum file 1164 C:...PP3ESystemStreams>type data.txt | python adder.py
sum type output 1164 C:...PP3ESystemStreams>type writer2.py
for data in (123, 0, 999, 42): print '%03d' % data C:...PP3ESystemStreams>python writer2.py | python sorter.py
sort py output 000 042 123 999 C:...PP3ESystemStreams>python writer2.py | python sorter.py | python adder.py
1164
The last command here connects three Python scripts by standard streams—the output of each prior script is fed to the input of the next via pipeline shell syntax.
A few coding pointers here: if you look closely,
you’ll notice that sorter
reads
all of stdin
at once with the
readlines
method, but adder
reads one line at a time. If the
input source is another program, some platforms run programs
connected by pipes in parallel. On such
systems, reading line by line works better if the data streams
being passed about are large because readers don’t have to wait
until writers are completely finished to get busy processing data.
Because raw_input
just reads
stdin
, the line-by-line scheme
used by adder
can always be
coded with sys.stdin
too:
C:...PP3ESystemStreams>type adder2.py
import sys
sum = 0
while True:
line = sys.stdin.readline( )
if not line: break
sum += int(line)
print sum
This version utilizes the fact that the int
allows the digits to be surrounded
by whitespace (readline
returns
a line including its
, but we
don’t have to use [:-1]
or
rstrip( )
to remove it for
int
). In fact, we can use
Python’s more recent file iterators to achieve the same effect—the
for
loop, for example,
automatically grabs one line each time through when we iterate
over a file object directly (more on file iterators in the next
chapter):
C:...PP3ESystemStreams>type adder3.py
import sys
sum = 0
for line in sys.stdin: sum += int(line)
print sum
Changing sorter
to read
line by line this way may not be a big performance boost, though,
because the list sort
method
requires that the list already be complete. As we’ll see in Chapter 20, manually coded sort
algorithms are likely to be much slower than the Python list
sorting method.
Interestingly, these two scripts can also be coded in a much
more compact fashion in Python 2.4 by using the new sorted
function, list comprehensions,
and file iterators. The following work the same way as the
originals:
C:...PP3ESystemStreams>type sorter24.py
import sys for line in sorted(sys.stdin): print line, C:...PP3ESystemStreams>type adder24.py
import sys print sum(int(line) for line in sys.stdin)
The latter of these employs a generator expression, which is much like a list comprehension, but results are returned one at a time, not in a physical list. The net effect is space optimization.
At the start of the last section, we piped
teststreams.py output into the standard
more
command-line program with
a command similar to this one:
C:...PP3ESystemStreams>python teststreams.py < input.txt | more
But since we already wrote our own “more” paging utility in
Python near the start of this chapter, why not set it up to accept
input from stdin
too? For
example, if we change the last three lines of the
more.py file listed earlier in this chapter
to this...
if _ _name_ _ == '_ _main_ _': # when run, not when imported if len(sys.argv) == 1: # page stdin if no cmd args more(sys.stdin.read( )) else: more(open(sys.argv[1]).read( ))
...it almost seems as if we should be able to redirect the standard output of teststreams.py into the standard input of more.py:
C:...PP3ESystemStreams>python teststreams.py < input.txt | python ..more.py
Hello stream world
Enter a number>8 squared is 64
Enter a number>6 squared is 36
Enter a number>Bye
This technique generally works for Python scripts. Here, teststreams.py takes input from a file again. And, as in the last section, one Python program’s output is piped to another’s input—the more.py script in the parent (..) directory.
But there’s a subtle problem lurking in the preceding
more.py
command. Really,
chaining worked there only by sheer luck: if the first script’s
output is long enough that more
has to ask the user if it should continue, the script will utterly
fail. The problem is that the augmented
more.py uses stdin
for two disjointed purposes. It
reads a reply from an interactive user on stdin
by calling raw_input
, but now it
also accepts the main input text on stdin
. When the stdin
stream is really redirected to an
input file or pipe, we can’t use it to input a reply from an
interactive user; it contains only the text of the input source.
Moreover, because stdin
is
redirected before the program even starts up, there is no way to
know what it meant prior to being redirected in the command
line.
If we intend to accept input on stdin
and use the
console for user interaction, we have to do a bit more. Example 3-9 shows a modified
version of the more script that pages the
standard input stream if called with no arguments but also makes
use of lower-level and platform-specific tools to converse with a
user at a keyboard if needed.
Example 3-9. PP3ESystemmoreplus.py
############################################################# # split and interactively page a string, file, or stream of # text to stdout; when run as a script, page stdin or file # whose name is passed on cmdline; if input is stdin, can't # use it for user reply--use platform-specific tools or GUI; ############################################################# import sys def getreply( ): """ read a reply key from an interactive user even if stdin redirected to a file or pipe """ if sys.stdin.isatty( ): # if stdin is console return raw_input('?') # read reply line from stdin else: if sys.platform[:3] == 'win': # if stdin was redirected import msvcrt # can't use to ask a user msvcrt.putch('?') key = msvcrt.getche( ) # use windows console tools msvcrt.putch(' ') # getch( ) does not echo key return key elif sys.platform[:5] == 'linux': # use linux console device print '?', # strip eoln at line end console = open('/dev/tty') line = console.readline( )[:-1] return line else: print '[pause]' # else just pause--improve me import time # see also modules curses, tty time.sleep(5) # or copy to temp file, rerun return 'y' # or GUI pop up, tk key bind def more(text, numlines=10): """ split multiline string to stdout """ lines = text.split(' ') while lines: chunk = lines[:numlines] lines = lines[numlines:] for line in chunk: print line if lines and getreply( ) not in ['y', 'Y']: break if _ _name_ _ == '_ _main_ _': # when run, not when imported if len(sys.argv) == 1: # if no command-line arguments more(sys.stdin.read( )) # page stdin, no raw_inputs else: more(open(sys.argv[1]).read( )) # else page filename argument
Most of the new code in this version shows up in its
getreply
function. The file’s
isatty
method tells us whether
stdin
is connected to the
console; if it is, we simply read replies on stdin
as before. Unfortunately, there is
no portable way to input a string from a console user independent
of stdin
, so we must wrap the
non-stdin
input logic of this
script in a sys.platform
test:
On Windows, the built-in msvcrt
module supplies low-level
console input and output calls (e.g., msvcrt.getch( )
reads a single key
press).
On Linux, the system device file named /dev/tty gives access to keyboard input (we can read it as though it were a simple file).
On other platforms, we simply run a built-in time.sleep
call to pause for five
seconds between displays (this is not at all ideal, but it is
better than not stopping at all and it serves until a better
nonportable solution can be found).
Of course, we have to add such extra logic only to scripts
that intend to interact with console users
and take input on stdin
. In a GUI application, for
example, we could instead pop up dialogs, bind keyboard-press
events to run callbacks, and so on (we’ll meet GUIs in Chapter 8).
Armed with the reusable getreply
function, though, we can safely
run our moreplus
utility in a
variety of ways. As before, we can import and call this module’s
function directly, passing in whatever string we wish to
page:
>>>from moreplus import more
>>>more(open('System.txt').read( ))
This directory contains operating system interface examples. Many of the examples in this unit appear elsewhere in the examples distribution tree, because they are actually used to manage other programs. See the README.txt files in the subdirectories here for pointers.
Also as before, when run with a command-line argument, this script interactively pages through the named file’s text:
C:...PP3ESystem>python moreplus.py System.txt
This directory contains operating system interface examples. Many of the examples in this unit appear elsewhere in the examples distribution tree, because they are actually used to manage other programs. See the README.txt files in the subdirectories here for pointers. C:...PP3ESystem>python moreplus.py moreplus.py
############################################################# # split and interactively page a string, file, or stream of # text to stdout; when run as a script, page stdin or file # whose name is passed on cmdline; if input is stdin, can't # use it for user reply--use platform-specific tools or GUI; ############################################################# import sys, string def getreply( ): ?n
But now the script also correctly pages text redirected into
stdin
from either a
file or a command pipe,
even if that text is too long to fit in a single display chunk. On
most shells, we send such input via redirection or pipe operators
like these:
C:...PP3ESystem>python moreplus.py < moreplus.py
############################################################# # split and interactively page a string, file, or stream of # text to stdout; when run as a script, page stdin or file # whose name is passed on cmdline; if input is stdin, can't # use it for user reply--use platform-specific tools or GUI; ############################################################# import sys, string def getreply( ): ?n
C:...PP3ESystem>type moreplus.py | python moreplus.py
############################################################# # split and interactively page a string, file, or stream of # text to stdout; when run as a script, page stdin or file # whose name is passed on cmdline; if input is stdin, can't # use it for user reply--use platform-specific tools or GUI; ############################################################# import sys, string def getreply( ): ?n
This works the same way on Linux, but, again, use the
cat
command rather than
type
. Finally, piping one
Python script’s output into this script’s input now works as
expected, without botching user interaction (and not just because
we got lucky):
......SystemStreams>python teststreams.py < input.txt | python ..moreplus.py
Hello stream world
Enter a number>8 squared is 64
Enter a number>6 squared is 36
Enter a number>Bye
Here, the standard output of one Python script is fed to the standard input of another Python script located in the parent directory: moreplus.py reads the output of teststreams.py.
All of the redirections in such command lines work only
because scripts don’t care what standard input and output really
are—interactive users, files, or pipes between programs. For
example, when run as a script, moreplus.py
simply reads stream sys.stdin
;
the command-line shell (e.g., DOS on Windows, csh on Linux)
attaches such streams to the source implied by the command line
before the script is started. Scripts use the preopened stdin
and stdout
file objects to access those
sources, regardless of their true nature.
And for readers keeping count, we have run this single
more
pager script in four
different ways: by importing and calling its function, by passing
a filename command-line argument, by redirecting stdin
to a file, and by piping a
command’s output to stdin
. By
supporting importable functions, command-line arguments, and
standard streams, Python system tools code can be reused in a wide
variety of modes.
All of the previous standard stream redirections work
for programs written in any language that hooks into the standard
streams and rely more on the shell’s command-line processor than on
Python itself. Command-line redirection syntax like <
filename
and |
program
is evaluated by the shell, not by
Python. A more Pythonesque form of redirection can be done within
scripts themselves by resetting sys.stdin
and sys.stdout
to file-like objects.
The main trick behind this mode is that anything that looks like a file in terms of methods will work as a standard stream in Python. The object’s interface (sometimes called its protocol), and not the object’s specific datatype, is all that matters. That is:
Any object that provides file-like
read methods can be assigned to sys.stdin
to make input come from that
object’s read methods.
Any object that defines file-like
write methods can be assigned to sys.stdout
; all standard output will
be sent to that object’s methods.
Because print
and raw_input
simply call the write
and readline
methods of whatever objects
sys.stdout
and sys.stdin
happen to reference, we can use
this technique to both provide and intercept standard stream text
with objects implemented as classes.
Such plug-and-play compatibility is usually called polymorphism—i.e., it doesn’t matter what an object is, and it doesn’t matter what its interface does, as long as it provides the expected interface. This liberal approach to datatypes accounts for much of the conciseness and flexibility of Python code. Here, it provides a way for scripts to reset their own streams. Example 3-10 shows a utility module that demonstrates this concept.
Example 3-10. PP3ESystemStreams edirect.py
############################################################################# # file-like objects that save standard output text in a string and provide # standard input text from a string; redirect runs a passed-in function # with its output and input streams reset to these file-like class objects; ############################################################################# import sys # get built-in modules class Output: # simulated output file def _ _init_ _(self): self.text = '' # empty string when created def write(self, string): # add a string of bytes self.text = self.text + string def writelines(self, lines): # add each line in a list for line in lines: self.write(line) class Input: # simulated input file def _ _init_ _(self, input=''): # default argument self.text = input # save string when created def read(self, *size): # optional argument if not size: # read N bytes, or all res, self.text = self.text, '' else: res, self.text = self.text[:size[0]], self.text[size[0]:] return res def readline(self): eoln = self.text.find(' ') # find offset of next eoln if eoln == -1: # slice off through eoln res, self.text = self.text, '' else: res, self.text = self.text[:eoln+1], self.text[eoln+1:] return res def redirect(function, args, input): # redirect stdin/out savestreams = sys.stdin, sys.stdout # run a function object sys.stdin = Input(input) # return stdout text sys.stdout = Output( ) try: function(*args) # was apply(function, args) except: sys.stderr.write('error in function! ') sys.stderr.write("%s, %s " % tuple(sys.exc_info( )[:2])) result = sys.stdout.text sys.stdin, sys.stdout = savestreams return result
This module defines two classes that masquerade as real files:
Output
Provides the write method interface (a.k.a. protocol) expected of output files but saves all output in an in-memory string as it is written.
Input
Provides the interface expected of input files, but provides input on demand from an in-memory string passed in at object construction time.
The redirect
function at
the bottom of this file combines these two objects to run a single
function with input and output redirected entirely to Python class
objects. The passed-in function to run need not know or care that
its print
statements, raw_input
calls and stdin
and stdout
method calls, are talking to a
class rather than to a real file, pipe, or user.
To demonstrate, import and run the interact
function at the heart of the
teststreams
script of Example 3-6 that we’ve been
running from the shell (to use the redirection utility function, we
need to deal in terms of functions, not files). When run directly,
the function reads from the keyboard and writes to the screen, just
as if it were run as a program without redirection:
C:...PP3ESystemStreams>python
>>>from teststreams import interact
>>>interact( )
Hello stream world Enter a number>2
2 squared is 4 Enter a number>3
3 squared is 9 Enter a number >>>
Now, let’s run this function under the control of the
redirection function in redirect.py and pass in
some canned input text. In this mode, the interact
function takes its input from the
string we pass in ('4
5
6
'
—three lines with explicit
end-of-line characters), and the result of running the function is a
string containing all the text written to the standard output
stream:
>>>from redirect import redirect
>>>output = redirect(interact, ( ), '4 5 6 ')
>>>output
'Hello stream world Enter a number>4 squared is 16 Enter a number> 5 squared is 25 Enter a number>6 squared is 36 Enter a number>Bye '
The result is a single, long string containing the
concatenation of all text written to standard output. To make this
look better, we can split it up with the string object’s split
method:
>>>for line in output.split('
'): print line
...
Hello stream world
Enter a number>4 squared is 16
Enter a number>5 squared is 25
Enter a number>6 squared is 36
Enter a number>Bye
Better still, we can reuse the more.py
module we saw earlier in this
chapter; it’s less to type and remember, and it’s already known to
work well:
>>>from PP3E.System.more import more
>>>more(output)
Hello stream world Enter a number>4 squared is 16 Enter a number>5 squared is 25 Enter a number>6 squared is 36 Enter a number>Bye
This is an artificial example, of course, but the techniques
illustrated are widely applicable. For example, it’s straightforward
to add a GUI interface to a program written to interact with a
command-line user. Simply intercept standard output with an object
such as the Output
class shown
earlier and throw the text string up in a window. Similarly,
standard input can be reset to an object that fetches text from a
graphical interface (e.g., a popped-up dialog box). Because classes
are plug-and-play compatible with real files, we can use them in any
tool that expects a file. Watch for a GUI stream-redirection module
named guiStreams
in Chapter 11.
Notice the function(*args)
syntax in the redirect
function of Example 3-10. In the prior
edition of this book, this was a built-in function call, apply(function, args)
, but the apply
built-in has been marked as
deprecated since that edition (in fact, it’s not even listed in
the functions section of the library manual). It’s unclear whether
apply
will ever be removed
completely, but the new call syntax is more general and concise,
and it should be preferred today. The following equivalent calls,
for instance, are more complex with apply
; the new syntax allows normal
arguments to be mixed with argument collection objects, but
apply
must merge
manually:
>>>def m(self, a, b, c): print self, a, b, c
>>>m(1, *(2, 3, 4))
1 2 3 4 >>>apply(m, (1,) + (2, 3, 4))
1 2 3 4
This becomes more useful as call signatures grow more complex:
>>>a=1; b=2; c=3; d=4; e=5
>>>def func(a, *ps, **ks): print a, ps, ks
>>>func(a, b, c=1, *(d, e), **{'f':2})
1 (2, 4, 5) {'c': 1, 'f': 2} >>>kargs = {'f':2}
>>>kargs.update({'c':1})
>>>apply(func, (a, b) + (d, e), kargs)
1 (2, 4, 5) {'c': 1, 'f': 2}
The prior section’s technique of redirecting streams to objects proved so handy that now a standard library automates the task. It provides an object that maps a file object interface to and from in-memory strings. For example:
>>>from StringIO import StringIO
>>>buff = StringIO( )
# save written text to a string >>>buff.write('spam ')
>>>buff.write('eggs ')
>>>buff.getvalue( )
'spam eggs ' >>>buff = StringIO('ham spam ')
# provide input from a string >>>buff.readline( )
'ham ' >>>buff.readline( )
'spam ' >>>buff.readline( )
''
As in the prior section, instances of StringIO
objects can be assigned to
sys.stdin
and sys.stdout
to redirect streams for
raw_input
and print
and can be passed to any code that
was written to expect a real file object. Again, in Python, the
object interface, not the concrete datatype, is
the name of the game:
>>>from StringIO import StringIO
>>>import sys
>>>buff = StringIO( )
>>>temp = sys.stdout
>>>sys.stdout = buff
>>>print 42, 'spam', 3.141
# or print >> buff,...
>>>sys.stdout = temp
# restore original stream >>>buff.getvalue( )
'42 spam 3.141 '
We’ve been focusing on stdin
and stdout
redirection, but stderr
can be similarly reset to files,
pipes, and objects. This is straightforward within a Python script.
For instance, assigning sys.stderr
to another instance of a class
such as Output
or a StringIO
object in the preceding section’s
example allows your script to intercept text written to standard
error too.
Python itself uses standard error for error message text (and
the IDLE GUI interface intercepts it and colors it red by default).
However, no higher-level tools for standard error do what print
and raw_input( )
do for the output and input
streams. If you wish to print to the error stream, you’ll want to
call sys.stderr.write( )
explicitly or read the next section for a print
statement trick that makes this a
bit simpler.
Redirecting standard errors from a shell command line is a bit
more complex and less portable. On most Unix-like systems, we can
usually capture stderr
output by
using shell-redirection syntax of the form command > output 2>&1
. This may
not work on some flavors of Windows platforms, though, and can even
vary per Unix shell; see your shell’s manpages for more
details.
Because resetting the stream attributes to new objects
was so popular, as of Python 2.0 the print
statement is also extended to
include an explicit file to which output is to be sent. A statement
of the form:
print >> file, stuff # file is an object, not a string name
prints stuff
to file
instead of to stdout
. The net effect is similar to
simply assigning sys.stdout
to an
object, but there is no need to save and restore in order to return
to the original output stream (as shown in the section on
redirecting streams to objects). For example:
import sys print >> sys.stderr, 'spam' * 2
will send text the standard error stream object rather than
sys.stdout
for the duration of
this single print statement only. The next normal print statement
(without >>
) prints to
standard output as usual.
Earlier in this chapter, we studied the built-in os.popen
function, which provides a way to
redirect another command’s streams from within a Python program. As
we saw, this function runs a shell command line (e.g., a string we
would normally type at a DOS or csh
prompt) but returns a Python file-like
object connected to the command’s input or output stream.
Because of that, the os.popen
tool is another way to redirect
streams of spawned programs, and it is a cousin to the techniques we
just met: its effect is much like the shell |
command-line pipe syntax for redirecting
streams to programs (in fact, its name means “pipe open”), but it is
run within a script and provides a file-like interface to piped
streams. It’s similar in spirit to the redirect
function, but it’s based on
running programs (not calling functions), and the command’s streams
are processed in the spawning script as files (not tied to class
objects). That is, os.popen
redirects the streams of a program that a script starts instead of
redirecting the streams of the script itself.
By passing in the desired mode flag, we redirect a spawned program’s input or output streams to a file in the calling scripts:
C:...PP3ESystemStreams>type hello-out.py
print 'Hello shell world' C:...PP3ESystemStreams>type hello-in.py
input = raw_input( ) open('hello-in.txt', 'w').write('Hello ' + input + ' ') C:...PP3ESystemStreams>python
>>>import os
>>>pipe = os.popen('python hello-out.py')
# 'r' is default--read stdout >>>pipe.read( )
'Hello shell world ' >>>pipe = os.popen('python hello-in.py', 'w')
>>>pipe.write('Gumby ')
# 'w'--write to program stdin >>>pipe.close( )
# at end is optional >>>open('hello-in.txt').read( )
'Hello Gumby '
The popen
call is also
smart enough to run the command string as an independent process on
platforms that support such a notion. It accepts an optional third
argument that can be used to control buffering of written
text.
Additional popen
-like tools
in the Python library allow scripts to connect to more than one of
the commands’ streams. For instance, the os.open2
call includes functions for
hooking into both a command’s input and output
streams:
childStdIn, childStdout = os.popen2('python hello-in-out.py')
childStdin.write(input)
output = childStdout.read( )
os.popen3
is similar, but
it returns a third pipe for connecting to standard error as well. A
related call, os.popen4
, returns
two pipe file objects; it’s like os.popen3
, but the output and error
streams are tied together into a single pipe:
childStdin, childStdout, childStderr = os.popen3('python hello-in-out.py') childStdin, childStdout_and_err = os.popen4('python hello-in-out.py')
The os.popen2/3/4
variants
work much like os.popen
, but they
connect additional streams and accept an optional second argument
that specifies text or binary-mode data (t
or b
—more on the distinction in the next
chapter).
The os.popen
calls are also
Python’s portable equivalent of Unix-like shell syntax for
redirecting the streams of spawned programs. The Python versions
also work on Windows, though, and are the most platform-neutral way
to launch another program from a Python script. The command-line
strings you pass to them may vary per platform (e.g., a directory
listing requires an ls
on Unix
but a dir
on Windows), but the
call itself works on all major Python platforms.
On Unix-like platforms, the combination of the calls os.fork
, os.pipe
, os.dup
, and some os.exec
variants can be used to start a
new independent program with streams connected to the parent
program’s streams. As such, it’s another way to redirect streams and
a low-level equivalent to tools such as os.popen
.
As of this writing, the os.fork
call does not work on the standard
version of Python for Windows, however, because it is too much at
odds with that system’s process model. See Chapter 5 for more on all of these
calls, especially its section on pipes, as well its sidebar on
Cygwin, a third-party package that includes a library for use on
Windows that adds Unix calls such as fork and a version of Python
that contains such tools.[*]
In the next chapter, we’ll continue our survey of Python system interfaces by exploring the tools available for processing files and directories. Although we’ll be shifting focus somewhat, we’ll find that some of what we’ve learned here will already begin to come in handy as general system-related tools. Spawning shell commands, for instance, provides ways to inspect directories, and the file interface we will expand on in the next chapter is at the heart of the stream processing techniques we have studied here.
[*] Notice that raw_input
raises an exception to signal end-of-file, but file read
methods simply return an empty string for this condition.
Because raw_input
also
strips the end-of-line character at the end of lines, an empty
string result means an empty line, so an exception is
necessary to specify the end-of-file condition. File read
methods retain the end-of-line character and denote an empty
line as
instead of
""
. This is one way in
which reading sys.stdin
directly differs from raw_input
. The latter also accepts a
prompt string that is automatically printed before input is
accepted.
[*] More historical anecdotes for users of older releases: as
of Python 2.0, the popen2
and
popen3
calls are made
available in the os
module,
and this subsumes the older popen2
module. For example, os.popen2
is the same as the older
popen2.popen2
except that the
order of stdin
and stdout
in the call’s result tuple is
swapped.
3.137.41.205