In Chapter 5, we learned about threads and the queue mechanism that threads typically use to communicate with each other. We also described the application of those ideas to GUIs in the abstract. Now that we’ve become fully functional GUI programmers, we can finally see what these ideas translate to in terms of code. If you skipped the related material in Chapter 5, you should probably go back and take a look first; we won’t be repeating the thread or queue background material here.
The application to GUIs, however, is straightforward. Recall that long-running operations must generally be run in parallel threads, to avoid blocking the GUI from updating itself. In our packing and unpacking examples earlier in this chapter, for instance, we noted that the calls to run the actual file processing should generally run in threads so that the main GUI thread is not blocked until they finish.
In the general case, if a GUI waits for anything to finish, it will be completely unresponsive during the wait—it can’t be resized, it can’t be minimized, and it won’t even redraw itself if it is covered and uncovered by other windows. To avoid being blocked this way, the GUI must run long-running tasks in parallel, usually with threads. That way, the main GUI thread is freed up to update the display while threads do other work.
Because only the main thread should generally update a GUI’s display, though, threads you start to handle long-running tasks or to avoid blocking input/output calls cannot update the display with results themselves. Rather, they must place data on a queue (or other mechanism), to be picked up and displayed by the main GUI thread. To make this work, the main thread typically runs a counter loop that periodically checks the thread for new results to be displayed. Spawned threads produce data but know nothing about the GUI; the main GUI thread consumes and displays results but does not generate them.
As a more concrete example, suppose your GUI needs to display telemetry data sent in real time from a satellite over sockets (a network interface we’ll meet later). Your program has to be responsive enough to not lose incoming data, but it also cannot get stuck waiting for or processing that data. To achieve both goals, spawn threads that fetch the incoming data and throw it on a queue, to be picked up and displayed periodically by the main GUI thread. With such a separation of labor, the GUI isn’t blocked by the satellite, nor vice versa—the GUI itself will run independently of the data streams, but because the data stream threads can run at full speed, they’ll be able to pick up incoming data as fast as it’s sent. GUI event loops are not generally responsive enough to handle real-time inputs. Without the data stream threads, we might lose incoming telemetry; with them, we’ll receive data as it is sent and display it as soon as the GUI’s event loop gets around to picking it up off the queue—plenty fast for the real human user to see. If no data is sent, only the spawned threads wait, not the GUI itself.
In other scenarios, threads are required just so that the GUI remains active during long-running tasks. While downloading a reply from a web server, for example, your GUI must be able to redraw itself if covered or resized. Because of that, the download call cannot be a simple function call; it must run in parallel with the rest of your program—typically, as a thread. When the result is fetched, the thread must notify the GUI that data is ready to be displayed; by placing the result on a queue, the notification is simple—the main GUI thread will find it the next time it checks the queue. For example, we’ll use threads and queues this way in the PyMailGUI program in Chapter 15, to allow multiple overlapping mail transfers to occur without blocking the GUI itself.
Whether your GUIs interface with satellites, web sites, or
something else, this thread-based model turns out to be fairly simple
in terms of code. Example
11-15 is the GUI equivalent of the queue-based threaded program
we met earlier in Chapter 5. In
the context of a GUI, the consumer thread becomes the GUI itself, and
producer threads add data to be displayed to the shared queue as it is
produced. The main GUI thread uses the Tkinter after
method to check the queue for
results.
Example 11-15. PP3EGuiToolsxd5 ueuetest-gui.py
import thread, Queue, time dataQueue = Queue.Queue( ) # infinite size def producer(id): for i in range(5): time.sleep(0.1) print 'put' dataQueue.put('producer %d:%d' % (id, i)) def consumer(root): try: print 'get' data = dataQueue.get(block=False) except Queue.Empty: pass else: root.insert('end', 'consumer got: %s ' % str(data)) root.see('end') root.after(250, lambda: consumer(root)) # 4 times per sec def makethreads( ): for i in range(4): thread.start_new_thread(producer, (i,)) # main Gui thread: spawn batch of worker threads on each mouse click import ScrolledText root = ScrolledText.ScrolledText( ) root.pack( ) root.bind('<Button-1>', lambda event: makethreads( )) consumer(root) # start queue check loop in main thread root.mainloop( ) # pop-up window, enter tk event loop
When this script is run, the main GUI thread displays the data
it grabs off the queue in the ScrolledText
window captured in Figure 11-11. A new batch of
four producer threads is started each time you left-click in the
window, and threads issue “get” and “put” messages to the standard
output stream (which isn’t synchronized in this example—messages might
overlap occasionally). The producer threads issue sleep calls to
simulate long-running tasks such as downloading mail, fetching a query
result, or waiting for input to show up on a socket (more on sockets
later in this chapter).
Example 11-16 takes the model one small step further and migrates it to a class to allow for future customization and reuse. Its operation and output are the same as the prior non-object-oriented version, but the queue is checked more often, and there are no standard output prints.
Example 11-16. PP3EGuiToolsxd5 ueuetest-gui-class.py
import thread, Queue, time from ScrolledText import ScrolledText class ThreadGui(ScrolledText): threadsPerClick = 4 def _ _init_ _(self, parent=None): ScrolledText._ _init_ _(self, parent) self.pack( ) self.dataQueue = Queue.Queue( ) # infinite size self.bind('<Button-1>', self.makethreads) # on left mouse click self.consumer( ) # queue loop in main thread def producer(self, id): for i in range(5): time.sleep(0.1) self.dataQueue.put('producer %d:%d' % (id, i)) def consumer(self): try: data = self.dataQueue.get(block=False) except Queue.Empty: pass else: self.insert('end', 'consumer got: %s ' % str(data)) self.see('end') self.after(100, self.consumer) # 10 times per sec def makethreads(self, event): for i in range(self.threadsPerClick): thread.start_new_thread(self.producer, (i,)) root = ThreadGui( ) root.mainloop( ) # pop-up window, enter tk event loop
We’ll revisit this technique in a more realistic scenario later in this chapter, as a way to avoid blocking a GUI that must read an input stream—the output of another program.
Notice that in the prior section’s examples, the data placed on the queue is always a string. That’s sufficient for simple applications where there is just one type of producer. If you may have many different kinds of threads producing many different types of results running at once, though, this can become difficult to manage. You’ll probably have to insert and parse out some sort of type information in the string so that the GUI knows how to process it. Imagine an email client, for instance, where multiple sends and receives may overlap in time; if all threads share the same single queue, the information they place on it must somehow designate the sort of event it represents—a downloaded message to display, a successful send completion, and so on.
Luckily, queues support much more than just strings—any type of Python object can be placed on a queue. Perhaps the most general of these is a callable object: by placing a callback function on the queue, a producer thread can tell the GUI how to handle the message in a very direct way. The GUI simply calls the objects it pulls off the queue.
Because Python makes it easy to handle functions and their
argument lists in generic fashion, this turns out to be easier than
it might sound. Example
11-17, for instance, shows one way to throw callbacks
on a queue that we’ll be using in Chapter 15 for PyMailGUI. The
ThreadCounter
class in this
module can be used as a shared counter and Boolean flag. The real
meat here, though, is the queue interface functions.
This example is mostly just a variation on those of the prior section; we still run a counter loop here to pull items off the queue in the main thread. Here, though, we call the object pulled off the queue, and the producer threads have been generalized to place a success or failure callback on the objects in response to exceptions. Moreover, the actions that run in producer threads receive a progress status function that, when called, simply adds a progress indicator callback to the queue to be dispatched by the main thread. We can use this, for example, to show progress during network downloads.
Example 11-17. PP3EGuiTools hreadtools.py
############################################################################## # system-wide thread interface utilities for GUIs; # single thread queue and checker timer loop shared by all windows; # never blocks GUI - just spawns and verifies operations and quits; # worker threads can overlap with main thread, and other workers; # # using a queue of callback functions and arguments is more useful than a # simple data queue if there can be many kinds of threads running at the # same time - each kind may have different implied exit actions # # because GUI API is not completely thread-safe, instead of calling GUI # update callbacks directly after thread exit, place them on a shared queue, # to be run from a timer loop in the main thread, not a child thread; this # also makes GUI update points less random and unpredictable; # # assumes threaded action raises an exception on failure, and has a 'progress' # callback argument if it supports progress updates; also assumes that queue # will contain callback functions for use in a GUI app: requires a widget in # order to schedule and catch 'after' event loop callbacks; ############################################################################## # run even if no threads try: # raise ImportError to import thread # run with GUI blocking except ImportError: # if threads not available class fakeThread: def start_new_thread(self, func, args): func(*args) thread = fakeThread( ) import Queue, sys threadQueue = Queue.Queue(maxsize=0) # infinite size def threadChecker(widget, delayMsecs=100): # 10x per second """ in main thread: periodically check thread completions queue; do implied GUI actions on queue in this main GUI thread; one consumer (GUI), multiple producers (load,del,send); a simple list may suffice: list.append/pop are atomic; one action at a time here: a loop may block GUI temporarily; """ try: (callback, args) = threadQueue.get(block=False) except Queue.Empty: pass else: callback(*args) widget.after(delayMsecs, lambda: threadChecker(widget)) def threaded(action, args, context, onExit, onFail, onProgress): """ in a new thread: run action, manage thread queue puts; calls added to queue here are dispatched in main thread; run action with args now, later run on* calls with context; allows action to be ignorant of use as a thread here; passing callbacks into thread directly may update GUI in thread - passed func in shared memory but called in thread; progress callback just adds callback to queue with passed args; don't update counters here: not finished till taken off queue """ try: if not onProgress: # wait for action in this thread action(*args) # assume raises exception if fails else: progress = (lambda *any: threadQueue.put((onProgress, any+context))) action(progress=progress, *args) except: threadQueue.put((onFail, (sys.exc_info( ),)+context)) else: threadQueue.put((onExit, context)) def startThread(action, args, context, onExit, onFail, onProgress=None): thread.start_new_thread( threaded, (action, args, context, onExit, onFail, onProgress)) class ThreadCounter: """ a thread-safe counter or flag """ def _ _init_ _(self): self.count = 0 self.mutex = thread.allocate_lock( ) # or use Threading.semaphore def incr(self): self.mutex.acquire( ) self.count += 1 self.mutex.release( ) def decr(self): self.mutex.acquire( ) self.count -= 1 self.mutex.release( ) def _ _len_ _(self): return self.count # True/False if used as a flag if _ _name_ _ == '_ _main_ _': # self-test code when run import time, ScrolledText def threadaction(id, reps, progress): # what the thread does for i in range(reps): time.sleep(1) if progress: progress(i) # progress callback: queued if id % 2 == 1: raise Exception # odd numbered: fail def mainaction(i): # code that spawns thread myname = 'thread-%s' % i startThread( action = threadaction, args = (i, 3), context = (myname,), onExit = threadexit, onFail = threadfail, onProgress = threadprogress) # thread callbacks: dispatched off queue in main thread def threadexit(myname): root.insert('end', '%s exit ' % myname) root.see('end') def threadfail(exc_info, myname): root.insert('end', '%s fail %s ' % (myname, exc_info[0])) root.see('end') def threadprogress(count, myname): root.insert('end', '%s prog %s ' % (myname, count)) root.see('end') root.update( ) # works here: run in main thread # make enclosing GUI # spawn batch of worker threads on each mouse click: may overlap root = ScrolledText.ScrolledText( ) root.pack( ) threadChecker(root) # start thread loop in main thread root.bind('<Button-1>', lambda event: map(mainaction, range(6))) root.mainloop( ) # pop-up window, enter tk event loop
This module’s self-test code demonstrates how this interface
is used. On each button click in a ScrolledTest
, it starts up six threads,
all running the threadaction
function. As this threaded function runs, calls to the passed-in
progress function place a callback on the queue, which invokes
threadprogress
in the main
thread. When the threaded function exits, the interface layer will
place a callback on the queue that will invoke either threadexit
or threadfail
in the main thread, depending
upon whether the threaded function raised an exception. Because all
the callbacks placed on the queue are pulled off and run in the main
thread’s timer loop, this guarantees that GUI updates occur in the
main thread only.
Figure 11-12 shows part of the output generated after clicking the example’s window once. Its exit, failure, and progress messages are produced by callbacks added to the queue by spawned threads and invoked from the timer loop running in the main thread.
To use this module, you will essentially break a modal
operation into thread and post-thread steps, with an optional
progress call. Study this code for more details and try to trace
through the self-test code. This is a bit complex, and you may have
to make more than one pass over this code. Once you get the hang of
this paradigm, though, it provides a general scheme for handling
heterogeneous overlapping threads in a uniform way. PyMailGUI, for
example, will do very much the same as mainaction
in the self-test code here,
whenever it needs to start a mail transfer.
3.12.163.180