Introducing safe side effects into the STM

The STM isn't safe as far as side effects are concerned. Since a dosync block may get retried, possibly more than once, any side effects will be executed again and again, whether they should be or not. Values may get written to the screen or logfile multiple times. Worse, values may be written to the database more than once.

However, all programs must produce side effects. The trick is adding them while getting a handle on complexity. The easiest way to do that is to keep side effects out of transactions.

For this recipe, to illustrate what can happen, we'll simulate thread starvation. That sounds serious, but it just means that one thread isn't able to access the resources it needs, so it can't do its job. We'll also use an atom (a reference that isn't controlled by the STM) to keep track of the number of times the STM retries a call to an agent. That way, we can see what happens that creates the problem, and what we need to do to fix it.

Getting ready

To prepare, we'll need access to java.lang.Thread in our REPL:

(import [java.lang Thread])

How to do it…

For this recipe, we'll walk through a couple of experiments to simulate thread starvation:

  1. For these experiments, we'll use one reference and two agents. The agents will try to read and increment the counter reference simultaneously, but they will wait for different amounts of time in the transaction, so one thread will starve the other:
    (def counter (ref 0))
(def a1 (agent :a1))
(def a2 (agent :a2))
  2. Now, we'll define a utility to start both agents on the same message function, with different sleep periods:
    (defn start-agents [msg a1-sleep a2-sleep]
  (send a1 msg a1-sleep)
  (send a2 msg a2-sleep))
  3. For the first experiment, we'll use a debug function for the side effect. It just prints the message and flushes the output stream:
    (defn debug [msg]
  (print (str msg 
ewline))
  (flush))
  4. The first message function will starve-out anything:
    (defn starve-out [tag sleep-for]
  (let [retries (atom 0)]
    (dosync
      (let [c @counter]
        (when-not (zero? @retries)
          (debug (str ":starve-out " tag
                      ", :try " @retries
                      ", :counter " c)))
        (swap! retries inc)
        (Thread/sleep sleep-for)
        (ref-set counter (inc c))
        (send *agent* starve-out sleep-for)
        tag))))
  5. If we send starve-out to both agents with very different sleep periods and look at the output, we'll see that :a2 is consistently getting starvedout. (You can stop the agents by calling shutdown-agents):
    user=> (start-agents starve-out 50 1000)
:starve-out :a2, :try 1, :counter 19
:starve-out :a2, :try 2, :counter 39
:starve-out :a2, :try 3, :counter 59
:starve-out :a2, :try 4, :counter 78
  6. In order to make this safe, we have to move all of the side effects out of the dosync block. This means that we'll move the debug call out of the STM. While we're at it, we'll move the send call since it's theoretically a side effect, even though it should be safe enough here. To be safer, we'll use a new output function, one that uses io! (highlighted). The io! block will throw an exception if it is executed inside a transaction:
    (defn debug! [msg] (io! (debug msg)))

(defn starve-safe [tag sleep-for]
  (let [retries (atom 0)]
    (dosync
      (let [c @counter]
        (swap! retries inc)
        (Thread/sleep sleep-for)
        (ref-set counter (inc c))))
    (when-not (zero? @retries)
      (debug! (str ":safe-starve " tag
                   ", :try " @retries
                   ", " @counter)))
    (send *agent* starve-safe sleep-for)
    tag))

This version safely handles the I/O in the STM. Moreover, if we forget and refactor the call to debug! back inside the transaction, our code will stop working.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.220.88.62