Hour 22. Grand Central Dispatch: Using Queues and Threading
What You’ll Learn in This Hour
Exploring the Grand Central Dispatch mechanism with blocks
Using a queue
Using a concurrent queue
Getting Started with Concurrency
Concurrency refers to the ability of a computer to run more than one task at the same time. Because all processor operations are carried out by the computer’s processor, the capabilities of the processor come into play with concurrency.
Note: Running Multiple Tasks at the Same Time—What It Doesn’t Mean
When talking about concurrency, people are referring to processor tasks. A program can send a request to a printer to print a page, and the computer can continue processing other tasks while the printer chugs away. The computer can further send a request to a disk drive to read or write some data, and that operation can proceed without concurrency; printers and disk drives frequently chug away at the same time. The processor is normally only involved with sending an instruction to print or an instruction to read or write. The peripheral devices receive the instruction and then carry on their own processing. Computers actually have a number of separate processors, including processors to control communications. This discussion is in regard to the central processor.
Looking at Processors Inside Computers
Many people refer to the computer and its processor synonymously. One computer has one processor, and that is the end of the story—at least until the advent of multicore chips. Each core on a chip is its own processor, and the chip contains management software that often allows each core to access shared data. (Note that this is a high-level conceptual overview.)
For computers with a single processor (or core), concurrency is pretty much irrelevant. There are many of those devices in use today, but Apple’s current lineup of computers is multicore. The first iPad and iPhone models before the 4S were single core, but the newer versions use multicore chips.
As soon as you have multiple cores, the question naturally arises: How do you use the processing power they contain? Going back to the days of mainframe computers, engineers have devised various ways to use the power of multiple processors. Most of these techniques involve rewriting code because code written for a single processor does not naturally or automatically adapt to running on multiple processors.
Using Concurrency Without Rewriting User Apps
Fortunately, rewriting code to take advantage of multiple processors need not pose a significant burden on developers. This is because, on most computers, the biggest single user of the processor is the operating system itself. The high-level programming instructions that developers write are converted during the compile/build process to the actual processor instructions (and there may be several layers of abstraction involved in the process).
As Apple started to explore multicore chips for their hardware, it also started exploring the changes to the operating systems that would enable the OSs to take advantage of the multicore chips. Apple shipped its first multiprocessor computer with the Power Mac Gigabit Ethernet model in the summer of 2000. One of the considerations involved in moving to NeXTSTEP was the hope that it would make the transition to multiprocessing Macs easier and faster than other alternatives.
In addition to starting to implement concurrency in rewrites of the operating system, Apple used Xcode to explore ways in which concurrency could be used to speed up application programs. Over time, the concurrency code in the operating system has become much more sophisticated, and the number of multicore processors has proliferated, as has the number of cores on multiprocessor cores themselves.
Using Threads for Concurrency
Threads are sequences of tasks running inside a single process. In a single-processor environment, one thread can run at a time. However, the operating system can swiftly switch from one thread to another, which can give the impression that multiple threads are running at the same time.
With multicore processors, multiple threads can, indeed, run at the same time. In documentation, you frequently find references to the concept of thread-safe code. Because multiple threads in the same app might be accessing shared memory at the same time, thread-safe code contains safeguards so that two threads do not corrupt the shared store.
Programming with threads can be a bit tricky because it is up to the implementer to manage the threads. Furthermore, in today’s world of multicore processors in various configurations, using threads can involve significant amounts of code that are aware of the configuration of cores on which they are running. Thus, threads can be a challenge to implement in general as well as being challenging with regard to the environment in which they are running. In some ways, this aspect of threads is a throwback to the 1940s and 1950s in which, in many cases, programs were written for specific computers. The situation still exists today in some areas of software development.
Note: Processors, Chips, Cores, and CPUs
Each of these terms has a different, specific meaning. However, in Apple’s diagrams and in this book, CPU, the abbreviation for central processing unit, is used in the context of Grand Central Dispatch (GCD) to describe what is usually a core within a chip. This avoids repeating the fact that what is called a CPU in this context might be a chip or a core within a chip.
Introducing Grand Central Dispatch (GCD)
GCD is the technology that Apple uses to manage concurrency on iOS and OS X. It was introduced with iOS 4 and OS X 10.6 (Snow Leopard) and has continued to evolve.
Looking at the Overall GCD Design
GCD is built on the concept of queues, which is described in the following section. Queues are designed to be easily accessed by developers in an abstract environment. The specific processing that you often have to specify with regard to threads is now implemented deep in the operating system (specifically, in the Darwin kernel).
GCD is what helps make your multicore Mac or iOS device perky. The enormous amount of processing power required to make the user interface responsive at the same time processing power is required to compile and build your latest app can be managed with GCD.
Using GCD
Between the iOS and OS X operating systems and Xcode on OS X, if you are using Apple hardware that is fairly recent, you are using GCD and concurrency (and so, most likely, are your users). GCD is automatic in many cases, but you can also write explicitly to use the features of GCD.
Apps that benefit most from GCD are those that do intensive and often repetitive calculations; computing the square root of a single number might benefit from concurrency in the implementation of the square root function that you and other apps call, but that single process is unlikely to benefit from concurrency.
Calculation-heavy processes that might benefit from concurrency and multicore processors have typically been found in image- and video-processing apps as well as apps for long-term weather and climate forecasts and calculation-intensive apps such as brute-force decryption of data as well as artificial intelligence applications, including language and image recognition. In addition to calculation-intensive apps, real-time apps that must run at a given speed also traditionally have used multiprocessing and concurrency to stay on top of their changing data.
Introducing Queues
Queues are at the heart of GCD and of many other computer structures. A queue is a line of objects, whether those objects are people at a bus stop or tasks to be run in a computer. The queues used in GCD are first in, first out (FIFO) just like the queue at a bus stop. If you are the first person in line, you are the first person on the bus.
Just as with the bus queue, there are exceptions to the FIFO rule. Passengers with disabilities may jump the queue at the suggestion of a dispatcher or bus driver. In some societies, passengers may also jump the queue by brute force; others may jump the queue by bribing their way onto the bus ahead of other people.
At this point, the real-life instance of bus queues is left behind with the note that jumping the FIFO queue in GCD is possible at the behest of the developer. The chaos of brute force or bribery have no parallels in GCD.
GCD has several types of queues, which are outlined in this section. In thinking about these types of queues, remember that their characteristics are designed not only to allow developers to use the appropriate features for the concurrent processes, but also to create queues that, by their characteristics, are amenable to being managed by the kernel. You do not worry about scheduling your queues; you just set them up and the operating system does the scheduling.
Here are the types of GCD queues:
Dispatch queues—These are based on C, but they can contain Objective-C objects.
Dispatch sources—These are for system events that are handled asynchronously. They are also based on C.
Operation queues—These use Cocoa for their management. They are the queues you generally focus on in your code.
Dispatch Queues
Dispatch queues are FIFO queues that can contain either function pointers or block objects. The queue objects are generically referred to as tasks.
GO TO Hour 20, “Working with Blocks,” p. 259, for more about blocks and function pointers.
A dispatch queue can be either serial or concurrent. Both types dequeue each task in the queue in turn. Serial queues execute the dequeued task and, when it is complete, execute the next task. Concurrent queues dequeue the tasks in turn, but, instead of waiting for completion, if there is a CPU available, the next task can be dequeued and execution can begin without waiting for the completion of the first one.
Tip: Synchronizing Using Locks
In concurrent processing that you have to manage for yourself (such as using threads rather than queues), you can lock resources so that only one thread can access it at a time. With serial queues, you do not have to worry about locking. The nature of the serial queue is that only one of your tasks runs at a time.
It is GCD that handles this dequeuing and starting of processes from queues as well as managing the availability of CPU.
Dispatch Sources
Like dispatch queues, dispatch sources are C-based; they process system events asynchronously. They handle events such as timers, signal handlers, Mach port events, and custom events. They are beyond the scope of this book.
Operation Queues
These are the queues that you most frequently worry about with GCD. They are manipulated through Cocoa (and Cocoa Touch) using the NSOperationQueue class, and the tasks they contain are instances of NSOperation. You construct your NSOperation instances and enqueue them. Go to the NSOperation class reference in Xcode or on developer.apple.com for more information.
Note: True Stories from Developers
The release of Xcode 4 and a new generation of Macs, including the MacBook Pro, provided an interesting example of the power of multicore processors interacting with the tuned operating system and Xcode. More than one developer reported performance improvements that subjectively appeared to be much greater than the revised specs for the new hardware would have suggested. Much of the performance improvement appears to have been the result of better use of concurrency.
Using Dispatch Queues
Dispatch queues are the simplest type of queues you can use to achieve concurrency.
GO TO Hour 20, “Working with Blocks,” p. 259, for the simplest way to work with queues using blocks.
Tip: Using Block Pseudocode
For simplicity, blocks used in the pseudocode in this section are shown as
^ { ... }
Replace the { ... } with your own code.
Using Global Concurrent Dispatch Queues
GCD does its best to use your queues and the tasks within them as efficiently as possible with the available CPUs. As you design your app (and particularly the concurrent sections of it), bear in mind that you can make life easier for GCD and your users by using concurrent rather than serial queues. With serial queues, a task might wait for a previous one to complete, and during that time, a CPU might be idle. For the same basic reason, asynchronous tasks, rather than synchronous tasks, allow GCD to improve performance on multicore processors.
Note
Keeping these points in mind is important. You might want to revise the structure of some very computation-intensive sections of your app so that you can take advantage of concurrent and asynchronous processes.
There are three global concurrent dispatch queues for each app. You do not have to create them because they are ready for your use. You access a global concurrent dispatch queue with the following code:
dispatch_queue_t myQueue =
dispatch_get_global_queue (DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_queue_t is a typedef that is declared in the GCD framework code.
The zero as the last argument is a placeholder for future expansion. You can get the other three global dispatch queues using these terms:
DISPATCH_QUEUE_PRIORITY_LOW
DISPATCH_QUEUE_PRIORITY_HIGH
DISPATCH_QUEUE_PRIORITY_BACKGROUND
These provide lower and higher priorities than the default queue. Using the different queues can improve performance because you can send tasks to one or the other depending on considerations such as the task itself and the data that it is using. Tasks that are going to be updating the user interface usually should run at a higher priority than a task designed to index data that is not currently displayed.
Adding a Task to a Global Concurrent Queue
As noted previously, asynchronous tasks generally provide greater concurrency throughput. You distinguish between synchronous and asynchronous tasks when you add them to a queue. Note that synchronicity is a property of each task and not of the queue as a whole.
Designing with Queues
You can create your own queues to manage specific tasks or types of tasks. In general, the smaller and more focused your tasks are, the more they benefit from GCD and queues. The best way to get familiar with queues and GCD is with the sample code on developer.apple.com.
Remember that you can use __block specifiers for variables that you want to access within a block, and that you can have read-only access to variables declared in the scope where your block is defined. Together, these tools let you communicate into and out of blocks in queues.
Tip: More on Grand Central Dispatch
There is a good deal of information available for you on blocks and GCD. Look at the sample code on developer.apple.com, videos from the Worldwide Developers Conference, and discussion lists on developer.apple.com. It can help to appreciate an overview of GCD such as this and then to explore it when you are actually confronted with an issue that lends itself to concurrency. Most people find it much easier to learn about concurrency in the context of a real-life problem they are trying to deal with.
Summary
This hour has provided an introduction to queues and Grand Central Dispatch, the generalized replacement for threading in Cocoa and Cocoa Touch. Because of the complexity of working with threads (not to mention the fact that multicore processors were not widely available), many developers stayed away from all of the issues of concurrency. With Grand Central Dispatch, concurrency and highly efficient (automated) use of multicore processors is readily available.
Q&A
Q. What is the biggest difference between Grand Central Dispatch and thread-based concurrency?
A. With GCD, the actual scheduling of the queue tasks is handled by the Unix level kernel, deep in the operating system. From a developer’s point of view, the biggest difference is that Grand Central Dispatch is much easier to use than threads.
Q. Why are asynchronous tasks and concurrent queues particularly advantageous when using GCD?
A. CPUs are not idle while tasks are waiting for completion of another task.
Workshop
Quiz
1. What are the four default global concurrent queues?
2. What is a good replacement for locking when using queues?
Quiz Answers
1. The four default global concurrent queues are default, low, high, and background priority.
2. Use serial queues.
Activities
On developer.apple.com, search for sample code using Grand Central Dispatch (GCD). Also on developer.apple.com, search the WWDC videos for sessions demonstrating GCD. Furthermore, look at Apple Developer Forums for discussions. GCD can be intimidating at first, but as you get into it and communicate with other developers, you will become more comfortable with it.
Because concurrency is important to different types of apps and in different ways, you will probably feel more at home if you look at examples and discussions of concurrency for the type of processing that you are doing (processing images or video, handling real-time data streams, and so forth).