In modern programs, it’s common to need to run code in small chunks to keep user interfaces responsive and to run operations in parallel to reduce latency and take advantage of multicore CPUs. C# and the .NET Framework provide the following support for asynchronous and parallel programming:

• The Task Parallel Library
• C# language support for asynchronous execution (async and await keywords)
• Parallel Linq (PLINQ)

This chapter explores how to use these facilities.

History

In earlier version of the .NET Framework, there were two ways of doing an asynchronous operation. The first one was used for I/O operations. If you were writing code to fetch data from a web site, the simplest code was synchronous.

public void GetWebResponse(string url)
{
    WebRequest request = WebRequest.CreateHttp(url);
    WebResponse response = request.GetResponse();
    // read the data and process it...
}

This is simple code to write, and you could even use the WebClient.DownloadString() method to do it in a single line of code. When you call that method, the following things happen:

1. A request is created in the proper format.
2. The request is sent over the network to the specified URL.
3. The request arrives at a server.
4. The server thinks about the request, composes an appropriate response, and sends it back.
5. The data is received back on your system.
6. The data is transferred back to your code.

This process can take a fair amount of time, during which time your program isn’t doing anything else, and your program’s UI is frozen. That is bad.

Windows implements a feature known as I/O completion ports, in which an I/O operation can be started, and then your program can go off and do something else until the operation completes, at which time the operating system will interrupt your program and present the data. This approach is supported in the .NET Framework, where a single operation is broken into two; the first one, named BeginXXX(), takes a callback function and returns an IAsyncResult, and the second one, named EndXXX(), takes an IAsyncResult and returns the result of calling the method.

You can modify the previous example as follows:

public void GetWebResponseStart(string url)
{
    WebRequest request = WebRequest.CreateHttp(url);
    request.BeginGetResponse(new AsyncCallback(GetWebResponseCallback), request);
}
public void GetWebResponseCallback(IAsyncResult result)
{
    WebRequest request = (WebRequest)result.AsyncState;
    WebResponse response = request.EndGetResponse(result);
    // read the data, and process it...
}

To start the process, you call the BeginGetResponse() method, passing in the delegate you want to be called when the operation is completed and a state object (in this case, the request) that will be passed through to your callback. After that call, your program goes on its merry way, until the operation completes. At that time, the runtime will call the callback routine you specified in the BeginGetResponse() call. In that routine, you pull the request object out of the result, extract the request, and then call EndGetResponse() to get the result of the call. Instead of the “call/wait/return” behavior that you had in the first example, you now get “call/return/do something else/pick up the data when it’s ready” behavior.

This is a very good thing; not only can the program keep the UI up-to-date and responsive, but you can make multiple I/O requests at once, which gives you the chance to overlap operations. It is not, however, without its disadvantages.

• It’s complicated to write. In this example, the LoadImageResponse() method starts an asynchronous read on data that is coming back on the stream, which requires yet another method and callback. It takes about 50 lines of code in my “simple” example, instead of the one line in the synchronous example.
• It has complicated exception behavior. Exceptions can be thrown from the GetWebResponseStart() method or any of the callback methods. If an exception occurs in the callback, it needs to somehow be communicated back to the calling method.
• It works only for components that have I/O completion ports underneath.

A more-general construct is needed.

Thread Pools

Chapter 34 covered the creation and use of threads, and in fact, threads are a useful general-purpose construct that can help. Assume you have an operation that takes a long time to complete.

class SlowOperation
{
    public static int Process()
    {
            // expensive calculation here
       return sum;
    }
}

If you called Process(), your program would just appear to hang. To prevent that, you can use a thread to perform the calculation.

Thread thread = new Thread(PerformSlowOperation, 0);
thread.Start();

You will need to figure out a way to get the value back when the thread is done, but assuming you can do that, there is another unfortunate issue; creating a thread is an expensive operation, and threads consume system resources. It may make sense to use a thread to perform a single expensive operation, but it doesn’t make sense to use a thread to perform a simpler operation, and you certainly wouldn’t want to use it to call WebClient.DownloadString().

To reduce the number of thread creations and the number of threads taking up resources, the frameworks provide you with a thread pool, which is just a set of threads that are managed for the general use of the process. You can use the thread pool to perform the slow operation.

ThreadPool.QueueUserWorkItem(PerformSlowOperationThreadPool);

This is a more efficient use of resources, but if there are other users of the thread pool in the program, it’s possible that my routine may have to wait a while before it executes. The ThreadPool class does provide some control over the number of threads it creates.

Introducing Tasks

The Task Parallel Library sits on top of the thread pool and provides a more abstract API. You can use the following to run the slow operation:

var task = new Task(() => Console.WriteLine(SlowOperation.Process()));
task.Start();

Or, if you want to start the task immediately, you can do it in a single line.

Task.Factory.StartNew(() => Console.WriteLine(SlowOperation.Process()));

Both of these queue a work item to the thread pool to execute the operation so that it ends up executing on a background thread.

var task = Task<int>.Factory.StartNew(() => SlowOperation.Process());
int i = task.Result;
Console.WriteLine(i);

The async and await Keywords

The situation you are in is very similar to the situation you had when you were writing an iterator; you want to be able to write the code that looks like it executes sequentially, but in actuality, you need it to return in the middle of the “sequential” section. Here’s a simple example:

static async void AsyncTaskWithResult()
{
    int result = await Task<int>.Factory.StartNew(() => SlowOperation.Process());
    Console.WriteLine(result);
}

This functions very much like the BeginGetResponse()/EndGetResponse() example, except this time the compiler does the heavy lifting of creating a state machine that tracks the states of each request and performs the appropriate operations; it splits the code in the method into two sections (one to start the task and one to process the result) and then arranges it so that when the first section of code completes, the second section will be invoked, and then it returns, so the rest of the program can continue executing. The method could be expressed in the following pseudocode¹:

1. If you don’t have any async result information, start operation and return the task associated with it.
2. If you do have result information, get the result out of it, and write it out to the console.

If the method has more than one await statement in it, the state management gets more complicated, because the method will have to know which one has completed when it is called.

This method is a void method, but if you want, you can make an async method that returns a value. Perhaps you want to perform two operations and return the sum of those two operations.

static async Task<int> AsyncTaskWithReturnValue()
{
    int result1 = await Task<int>.Factory.StartNew(() => SlowOperation.Process1());
    int result2 = await Task<int>.Factory.StartNew(() => SlowOperation.Process2());
    return result1 + result2;
}

This will return the sum of the two results without blocking on the slow operations. It will be called three times.

• To start the task to call Process1()
• To get the result of the first task and to start the task to call Process2()
• To get the result of the second task, add it to the first task, and then return it

Tasks and Completion Ports

Tasks can also be used to unify the completion port approach and the task approach,² and some of the .NET Framework classes have already been modified to take advantage of this. I will return to my old friend WebClient.DownloadString(), which has a new method to help.

async void WebMethodDownload(string url)
{
    string contents = await new WebClient().DownloadStringTaskAsync(url);
    c_label.Content = contents;
}

This fetches the contents of a specified URL and puts it into a label control. The second line may raise some eyebrows; in typical asynchronous code, you would expect to see a call to Dispatcher.Invoke() to get from the background thread to the UI thread, but the C# async support ensures that the whole routine will execute using the same thread context that the routine was originally called with, which is quite slick.

Tasks and Parallel Operations

All of the task examples have performed only one operation at once. It is now time to explore performing operations in parallel. The following code will be your starting point. It uses async and await.

async void LoadImagesAsync()
{
    List<string> urls = m_smugMugFeed.Fetch();
    int i = 0;
    foreach (string url in urls)
    {
       ImageItem imageItem = new ImageItem();
       await imageItem.LoadImageAsync(url);
       AddImageToUI(imageItem);
    }
}

This code talks to the popular SmugMug photography site and asks for lists of popular photos.³ You get a list of URLs of the popular photos and then loop through them using await so that you don’t block the UI.

The code works nicely; it doesn’t block the UI, but it downloads one photo at a time, which takes a long time.⁴ What you need is code that launches multiple operations at once.

void LoadImage(string url)
{
    ImageItem imageItem = new ImageItem();
    imageItem.LoadImage (url);
    Dispatcher.BeginInvoke(new Action<ImageItem>(AddImageToUI), imageItem);
}
void LoadImagesAsyncNewInParallel()
{
    List<string> urls = m_smugMugFeed.Fetch();
    foreach (string url in urls)
    {
       Task.Run(() => LoadImage(url));
    }
}

You have pulled the inner-loop code out of the main method and put it into the LoadImage() method. This method synchronously loads the image and then adds it to the UI. Because you are running in the thread pool, you need to invoke back to the UI thread. This version takes about 50 seconds to run, considerably slower than the last version. This isn’t surprising; when LoadImage() is called on a thread, it blocks until the entire operation completes. My first thought is to throw more threads at it by modifying the behavior of the thread pool.

ThreadPool.SetMinThreads(25, 25);

That takes about 30 seconds, which is better but still not exciting, and the use of more threads is not free. If only there were a way to for the LoadImage() method to not block on the thread pool while it is waiting for a response so that other threads could run.

async void LoadImage(string url)
{
    ImageItem imageItem = new ImageItem();
    await imageItem.LoadImageAsync(url);
    await Dispatcher.BeginInvoke(new Action<ImageItem>(AddImageToUI), imageItem);
}

That is exactly the point of async and await: so that you don’t block in time-consuming operations. Blocking on a thread pool thread may not be as obvious as blocking on a UI thread, but it’s still a bad idea.

This version is fast enough that it’s a bit hard to time exactly, so I’ll say, “something under four seconds.” The task library and the C# async support have provided a nice way to speed up the code, and the code is very similar to the synchronous code you would have written.

Data Parallelism

The previous examples illustrated how to improve the speed of operations by not waiting when there was other work to do and by performing operations in parallel across multiple threads. The majority of the time savings came not because you could perform more than one operation at a time but because you weren’t waiting for slow external operations to finish.

But what if your operations are all local to the machine? It would certainly be nice to be able to make those run faster as well. Consider the following methods:

static List<int> m_numbersToSum;
static long SumNumbersLessThan(int limit)
{
    long sum = 0;
    for (int i = 0; i < limit; i++)
    {
       sum += 1;
    }
    return sum;
}
static void SumOfSums()
{
    foreach (int number in m_numbersToSum)
    {
       SumNumbersLessThan(number);
    }
}

You have a list of the integers from 0 to 50,000 in m_numbersToSum, and you call SumNumbersLessThan() with each of those numbers. That takes about 1.37 seconds on my computer. But my system has four CPU cores on it, and it would sure be nice if I could make use of them. Enter the Parallel class.

static void SumOfSumsParallel()
{
    Parallel.ForEach(m_numbersToSum, number =>
       {
            SumNumbersLessThan(number);
       });
}

You pass in the set of numbers to sum and a lambda that contains the code you want to run. The Parallel class (part of the Parallel Task Library) farms out the calls to different tasks running on different cores, and on my machine, the method finishes in 0.43 seconds, a three-times speedup.⁵ That’s a nice improvement for a very small change—well, except that the code doesn’t actually return the result that was calculated. That is quite simple to do in the nonparallel version, but the parallel version will take a bit more work.

static long m_sumOfSums;
static long SumOfSumsParallelWithResult()
{
    m_sumOfSums = 0;
    Parallel.ForEach(m_numbersToSum, number =>
    {
       long sum = SumNumbersLessThan(number);
       Interlocked.Add(ref m_sumOfSums, sum);
    });
    return m_sumOfSums;
}

In this case, since I am only adding up numbers, I could use the Interlocked.Add() method⁶ to add each sum to a global sum. More typically, I would need to accumulate the results somewhere else and then process them at the end.

static ConcurrentBag<long> m_results = new ConcurrentBag<long>();
static long SumOfSumsParallelWithConcurrentBag()
{
    Parallel.ForEach(m_numbersToSum, number =>
    {
       long sum = SumNumbersLessThan(number);
       m_results.Add(sum);
    });
    return m_results.Sum();
}

This uses the ConcurrentBag class, which stores an unordered collection of items. Since you don’t care about this ordering of the results, this works well.⁷

PLinq

There is another option for this example, a parallel version of Linq known as PLinq. The previous example is very easily expressed in PLinq.

static long SumOfSumsPLinq()
{
    return
       m_numbersToSum
            .AsParallel()
            .Select(number => SumNumbersLessThan(number))
            .Sum();
}

The AsParallel() method puts you into the parallel world, and then PLinq is free to parallelize as it sees fit. This approach doesn’t yield the same improvement as the previous one (only about a two-times speedup), because PLinq has to pull the sequence apart, parallelize the calls to SumNumbersLessThan(), and then put the results back together so that they can be summed up. Those operations consume some of the savings.

Design Guidelines

The task approach is much easier to use than the previous schemes, and the fact that it unifies the thread pool and completion port approaches is another point in its favor. async and await are very useful for what they are designed for. I suggest using them rather than the previous approaches.

Parallel.ForEach() and PLinq present some interesting trade-offs; the first is certainly faster when the results don’t relate to one another, while PLinq may be faster if they do relate to one another, and it will likely be much better if you care about the ordering of the sequence. For more information about when to choose one over the other, see the excellent paper “When Should I Use Parallel.ForEach? When Should I Use PLINQ?” available online from Microsoft.

¹ This is a simplification; if you want more details, run ildasm against the generated code.

² The unification is not automatic; you need to modify your classes so that the methods return task instances, but there are helpers to make this straightforward. See the MSDN topic “TPL and Traditional .NET Asynchronous Programming” for more information.

³ For the full code that illustrates both the old and new ways of performing async operations, see the downloadable content for this book.

⁴ Remember, await and async are about writing the code in one method, not about making it run fast. This approach takes about 18 seconds to fetch 100 photos.

⁵ Your mileage may vary. Actually mileage is likely to be lower.

⁶ See Chapter 34 for more information about the interlocked methods.

⁷ There is no ConcurrentList<T>, which seems a bit strange, though it would not be terribly useful with the Parallel class, because there is no guarantee that the operations complete in the order in which you started them, and maintaining order therefore takes some extra work.