The first step towards reactive applications is to break out of synchronous processing. In general, applications waste a lot of time waiting for things to happen. Maybe we are waiting on an expensive computation—say, calculating the 1000th Fibonacci number. Perhaps we are waiting for some information to be written to the database. We could also be waiting for a network call to return, bringing us the latest recommendations from our favorite online store.
Regardless of what we're waiting for, we should never block clients of our application. This is crucial to achieve the responsiveness we desire when building reactive systems.
In an age where processing cores are abundant—my MacBook Pro has eight processor cores—blocking APIs severely underutilizes the resources we have at our disposal.
As we approach the end of this book, it is appropriate to step back a little and appreciate that not all classes of problems that deal with concurrent, asynchronous computations require the machinery of frameworks such as RxJava or core.async.
In this chapter, we will look at another abstraction that helps us develop concurrent, asynchronous applications: futures. We will learn about:
- The problems and limitations with Clojure's implementation of futures
- An alternative to Clojure's futures that provides asynchronous, composable semantics
- How to optimize concurrency in the face of blocking IO
The first step toward fixing this issue—that is, to prevent a potentially long-running task from blocking our application—is to create new threads, which do the work and wait for it to complete. This way, we keep the application's main thread free to serve more clients.
Working directly with threads, however, is tedious and error-prone, so Clojure's core library includes futures, which are extremely simple to use:
(def f (clojure.core/future
(println "doing some expensive work...")
(Thread/sleep 5000)
(println "done")
10))
(println "You'll see me before the future finishes")
;; doing some expensive work...
;; You'll see me before the future finishes
;; doneIn the preceding snippet, we invoke the clojure.core/future macro with a body simulating an expensive computation. In this example, it simply sleeps for 5 seconds before returning the value 10. As the output demonstrates, this does not block the main thread, which is free to serve more clients, pick work items from a queue, or what have you.
Of course, the most interesting computations, such as the expensive one, return results we care about. This is where the first limitation of Clojure futures becomes apparent. If we attempt to retrieve the result of a future—by derefing it—before it has completed, the calling thread will block until the future returns a value. Try running the following slightly modified version of the previous snippet:
(def f (clojure.core/future
(println "doing some expensive work...")
(Thread/sleep 5000)
(println "done")
10))
(println "You'll see me before the future finishes")
@f
(println "I could be doing something else. Instead I had to wait")
;; doing some expensive work...
;; You'll see me before the future finishes
;; 5 SECONDS LATER
;; done
;; I could be doing something else. Instead, I had to waitThe only difference now is that we immediately try to deref the future after we create it. Since the future isn't done, we sit there waiting for 5 seconds until it returns its value. Only then is our program allowed to continue.
In general, this poses a problem when building modular systems. Often, a long-running operation like the one described earlier would be initiated within a specific module or function, and handed over to the next logical step for further processing.
Clojure futures don't allow us to schedule a function to be executed when the future finishes in order to perform such further processing. This is an important feature in building reactive systems.