I know how hard it is to watch it go.
And all the effort that it took to get there in the first place.
And all the effort not to let the effort show.
—Everything but the Girl, Band
Ordering Pizza
In asynchronous programming, we embrace the fact that certain operations are best represented by running them separately from the main flow of logic. Instead of stopping everything while we wait for a result, we expect those separate computations to notify us when they have completed, at which point we’ll deal with their results. It’s a bit like one of those restaurants where you order, say, a pizza, and they give you a pager that flashes when your order is ready. You’re free to grab a table and chat with your friends. When the pager goes off, you collect your pizza and carry on with the main business of your visit – eating!
F# offers an elegant model for asynchronous computation, but it has to be said that working asynchronously inevitably complicates the business logic of your code. The trick is to keep the impacts to the minimum. Fortunately, by adopting a small set of coding patterns, you can keep your code elegant and readable, while still getting the benefits of asynchronous working.
A World Without Async
Because the benefits and impacts of asynchronous working tend to manifest across the whole structure of your program, I’m going to break with the practice of most of this book and offer a complete, potentially useful program as the example for this whole chapter. The example is a “bulk file downloader,” a console program that can find all the file download links in a web page and download all the files. It’ll also be able to filter what it downloads. For example, you could download just the files whose names end in “.gz”. As a starting point, I’ll offer a synchronous version of the program. Then I’ll go through all the steps necessary to make it work asynchronously. This reflects my normal coding practice: I tend to write a synchronous version initially to get all the business logic clear, then I translate relevant portions into an asynchronous world.
To avoid including a huge listing, I’ve broken up the program into parts that I’ll discuss separately. If you want to follow along, create an F# console program and simply add the code from each successive listing into Program.fs.
We’ll start with a module that can print colored messages to the console, which will be useful to show when downloads start, complete, fail, and so forth (Listing 10-1). The report function also shows the managed thread ID for the current thread, which will help us explore the behavior of our program as we transition it to an asynchronous approach.
Printing colored console messages
A Discriminated Union to model success and failure
The isOk function in more verbose form
Now let’s write some functions that get the file’s download links from the target web page (Listing 10-4). The absoluteUri function deals with the fact that some web pages provide download links relative to their own addresses (e.g., downloads/myfile.txt) while others provide absolute addresses (e.g., https://mysite.org/downloads/myfile.txt ). The code here is pretty simplistic and may not work in all cases, but I wanted to keep it simple, as URL processing is not the main topic of this chapter.
Functions for getting download links from a web page
The tryDownload function
The GetFiles function
Finally, in Listing 10-7, we have a main function for the console program. It calls Download.GetFiles to do its work. We also use a System.Diagnostics.Stopwatch to time the whole operation, and we list out all the failed files at the end of processing.
Note
You’ll need to create a directory called “c: empdownloads,” or amend the code to use a directory that exists.
The console program’s main function
Some Download URLs and Name Patterns
Url | Pattern | Comments |
|---|---|---|
neam.*.json.gz$ | Minor planets | |
.zip$ | Computational Linguistics | |
http://storage.googleapis.com/books/ngrams/books/datasetsv2.html | eng-1M-2gram.*.zip$ | Google n-grams Very large. Don’t download this over a metered connection! |
Running the Synchronous Downloader
Behavior of the synchronous downloader
The files are downloaded one at a time, everything happens on the same thread (ID: 1), and the whole process takes about a minute.
WiFi usage during a run of the synchronous mass downloader
While the WiFi connection is kept fairly busy, it certainly isn’t maxed out. But the main concern with the behavior of this synchronous version is the fact that it hogs an entire thread throughout the time it is running. It does this even though much of the time is spent waiting for server responses as blocks of data are sent over the network. In .NET a thread is considered quite an expensive resource, one which – on a busy machine – could be doing other work during these waits. This other work could be for other unrelated programs on the same machine, or simply downloading other files for this same bulk download run.
Converting Code to Asynchronous
The return type of HtmlDocument.AsyncLoad
The code following the let html = binding won’t compile now, because html is no longer an HtmlDocument instance, it’s an Async<HtmlDocument>. Instead of giving you a pizza, the person at the counter has given you a pager: effectively the promise of a pizza and a means of knowing when it’s ready. So, just like when you enter a restaurant that uses a pager system, you need to adjust your expectations and behave a little differently: that is, don’t eat the pager!
Placing a function body into an async computation expression
The let! and return keywords are only valid in the context of computation expressions such as async {}. Here, let! effectively means “Please get me a pizza and page me when it’s ready. I will come back to this exact point when you page me. In the meantime, I’ll feel free to talk to my friends.” Using return is analogous to linking a particular pizza order with a pager, and handing over the pager instead of the pizza.
Again this is an I/O operation that is going to take time, in this case an eternity in CPU terms because we might be downloading large files. There are two asynchronous methods in the WebClient API to choose from: DownloadFileAsync and DownloadFileTaskAsync . The one we want is DownloadFileTaskAsync. The other one requires us to provide an event handler to notify us of completion, almost as if we had to give the pizza restaurant our own pager. This seems a bit too much trouble to be worth it, even for pizza.
Using Async.AwaitTask and do! to perform an async imperative operation
Place the body in an async {}.
Identify any time-consuming external operations where the API you are using offers an Async version.
Use let! or do! to bind or imperatively execute them. Where necessary, use Async.AwaitTask to translate from C# Task to F# Async.
Return (the promise of) results using the return keyword.
Incidentally, from F# 4.5 there is also a match! keyword, which you can use to call async functions, and pattern match on the results, in a single operation.
Starting to make GetFiles asynchronous
Since GetFiles is effectively the public API of the Download module, I’ve also renamed it AsyncGetFiles to cue callers that this is an asynchronous function.
Synchronous download code
An anti-pattern for multiple, similar async computations
This is like ordering multiple pizzas, taking the pager for each pizza, but then standing at the counter in everyone else’s way until each pager flashes.
Using Async.Parallel
Calling AsyncGetFiles
Locking Shared Resources
Make a function thread safe using a lock expression
The new version of report is a nice example of the technique we introduced in the previous chapter: using a binding that creates some state but keeps it private, then returns a function that uses that state. In this case the state in question is simply an arbitrary object that is used by the lock expression to ensure exclusive access.
Needless to say, locking is a very complex subject. But in this context, Listing 10-17 shows a simple and effective way to achieve exclusive access for an operation that won’t take long to run.
Testing Asynchronous Downloads
Log messages from an asynchronous run
The downloads run on several threads, and the thread that logs the completion of a download is usually different from the thread that started it. This is the magic of let! and do!.
All the downloads are started before any of them complete. Compare that with the way started/completed messages simply alternate in the synchronous version.
Most importantly of all, the whole operation takes 14 seconds instead of a minute.
WiFi throughput downloading files asynchronously
In Figure 10-1, throughput on the interface was very spiky and peaked at about 11Mbps. In the asynchronous version, we get up to over 40Mbps, and the throughput is pretty consistent over the brief time the run lasts. Given this is going over WiFi, through my home’s highly questionable, main wiring, then through England’s almost equally, questionable fiber infrastructure, this is really pretty good.
Batching
One thing I’ve learned in several decades of coding is never to trust one’s first successful run! Let’s try the same code against some of the Google n-grams dataset. (You’ll find the URL and regular expression pattern for this in Table 10-1).
Note
This is a large dataset. Don’t do this on a metered connection, or while your household is trying to watch Netflix!
Downloading a large number of files
WiFi throughput while downloading a large number of files
Although Google’s servers will probably be just fine, some other services might throttle if you ask for too much at once. Database servers might even run out of connection resources if not configured to service a tsunami of requests like this.1
We might want to start work on some downloaded files as soon as possible. For instance, we might want to start uncompressing them or getting data out of them as soon as they are downloaded. In the pizza analogy, we don’t want all the cooks to spend their time kneading dough and chopping toppings for a large order, when they could be spending at least some time putting batches of assembled pizzas into ovens.
Using Seq.chunkBySize to create computation batches
Behavior of explicitly batched download
WiFi throughput during explicitly batched download
On the plus side, we do start seeing downloads complete much earlier in the process, meaning that we could get started with further processing of those files. But notice the pattern of the log messages. The first file of the second batch doesn’t start downloading until the last file of the first batch has finished downloading. Hence the four-deep bands of “started” and “completed” messages in the log. This is reflected in the network throughput: it dips toward the end of each batch as the last part of the last file dribbles through.
What we need is throttling: the ability to start a limited number of computations simultaneously, and to start a new one each time a previous one completes.
Throttling
It would certainly be possible to write one’s own throttled, parallel computation logic. But this is such a common pattern, it’s worth looking to see if someone has solved it in a general way. As it turns out, the F# extensions library FSharpx contains exactly the functionality we need. (At the time of writing, this functionality had not been moved into the core F# libraries, but we live in hope that this will eventually happen.)
Asynchronous, parallel, throttled downloads
Behavior of a parallel, throttled download
WiFi throughput during parallel, throttled download
Initially a batch of 4 downloads is started, then as soon as one completes, another one is started on whatever thread happens to be available. This keeps the network connection nice and busy but without having a great number of downloads all fighting for limited bandwidth.
C# Task versus F# Async
Now that you’ve seen the benefits of asynchronous programming, it’s time to revisit something we glossed over earlier: the difference between an F# Async and a C# Task. They each represent their language’s conception of an asynchronous computation that will return some type when completed. However, there is an important difference. C# uses a “hot task” model: when something creates a Task instance, the underlying computation is already running. F# uses a “cold task” model: the caller is responsible for starting the computation. This has some advantages in terms of composability. For example, in Listing 10-20 we had the opportunity to group tasks into batches of a fixed size using Seq.chunkBySize before finally launching them using Async.Parallel and Async.RunSynchronously.
Although we happened to be using do! in this case, the same would have applied for a let! binding.
As we’ve already said, when an API returns a C# Task, you’ll have to convert it into an F# Async using Async.AwaitTask if you want to use it in an async computation expression with let! and do!.
If you’re writing a general-purpose API that exposes asynchronous functions, you should by default return a C# Task rather than an F# Async. This follows the general guidance that APIs for use in languages other than F# should not expose F#-specific types. You can work in terms of F# Async internally, and at the last moment convert into C# Task using Async.StartAsTask.
API functions that return a C# Task should be named with a suffix of Async – for example, WebClient.DownloadFileTaskAsync.
It’s OK, though, for APIs aimed primarily at F# consumers, such as FSharpx.HtmlDocument, to expose asynchronous functions that return F# Async instances.
F#-centric APIs that return an F# Async should be named with a prefix of Async – for example, HtmlDocument.AsyncLoad.
Overloads for exposing C# Task functions
(The code in Listing 10-24 is courtesy of Tomas Petricek via stackoverflow.com.)
Finally, I should point out that, at the time of writing, there is an F# language proposal to offer a task {} computation expression, that is, one that works in terms of C# Task rather than F# Async. This may simplify some coding in this area, but perhaps at the cost of some of the composability that F# Async offers.
Recommendations
Get your business logic working correctly in a synchronous way.
Identify places where you are “ordering pizza,” in other words. making a request, usually via an API, which will take some time and doesn’t require the involvement of the current thread. Any good API should offer an asynchronous implementation of the call you are making.
Assuming the function from where you are ordering pizza is reasonably well factored, simply enclose its body with async {}. Change the ordering-pizza calls from let to let!, or if they are imperative, use do! If you are using F# 4.5 or later you can also use match!.
If the function from where you are ordering pizza is not well factored, you may need to break it down to make it easier to enclose the appropriate code in async {} blocks.
If an asynchronous API call returns a C# Task rather than an F# Async, you’ll also have to convert the Task to an Async using Async.StartAsTask.
Return (the promise of) data from the async{} expression using the return keyword.
Do the same thing to any higher-level functions that call the functions you just changed. Keep on going until you reach the top of the hierarchy, where you actually use the data or expose an API. I’ve heard this process referred to as implementing async “all the way down.”
If exposing an API for use by other languages, translate the F# Async to a C# Task using Async.StartAsTask. This avoids exposing F# types where they may not be natively understood.
To actually get results, use Async.RunSynchronously. But do this as little as possible – generally at the top of your “async-all-the-way-down” chain. You may not have to do it at all if you want external code that calls your functions to decide when to wait for results.
To run similar, independent requests in parallel, use Async.Parallel, or Async.ParallelWithThrottle from the FSharpx.Async package. (Eventually this may move into core F# libraries.)
Finally, all this is moot if your computation is limited by the local CPU power available (“CPU bound”). In those cases, you might as well use Array.Parallel.map or one of its siblings from the Array.Parallel module. We’ll revisit this topic in Chapter 12.
Summary
In this chapter you learned how to deal with situations where your application is “ordering pizza” – in other words, setting off a computation that will take some time, and for which it isn’t necessary for the current thread to stay involved. You found out how to deal with these cases by enclosing them in an async {} block and using let!, match! and do! to set off the time-consuming computation, and to have control to return to the same point (but likely on a different thread) once a result is obtained.
Asynchronous and parallel programming is a huge topic. In a wide-ranging book like this. we can really only scratch the surface. If you want to learn more, I would suggest studying a superb blog post on medium.com, entitled “F# Async Guide,” by Lev Gorodinski (of Jet.com). Then you may also wish to look at the Nuget package “Hopac,” which is a concurrent programming library for F# in the style of Concurrent ML.2 You will find implementations there of most of the concurrent programming patterns you are likely to need in mainstream programming. Having said that, the techniques described in this chapter should serve you well in most situations.
In the next chapter, we’ll look at railway oriented programming, a coding philosophy that encourages you to think about errors as hard as you think about successes, so that both the “happy” and “sad” paths in your code are equally well expressed.
Exercises
This section contains exercises to help you get used to translating code into an asynchronous world.
Exercise 10-1 – Making Some Code Asynchronous
printfn "That took %ims" sw.ElapsedMilliseconds
If you run the demo() function, you’ll notice that this operation takes over 5 seconds to get 10 results.
Change the Consumer.GetData() function so that it is asynchronous, and so that it runs all its calls to Server.AsyncGetString() in parallel.
You don’t need to throttle the parallel computation. Consumer.GetData() should be an F# style async function, that is, it should return Async<String[]>.
Hint: You’ll also need to change the demo() function so that the result of Consumer.GetData is passed into Async.RunSynchronously.
Exercise 10-2 – Returning Tasks
How would your solution to Exercise 10-1 change if Consumer.GetData() needed to return a C# style Task?
Exercise Solutions
Exercise 10-1 – Making Some Code Asynchronous
Run demo() to verify that the computation takes roughly half a second.