Implementing our equation

So far, our moving-average function only gives us a partitioned list of prices (or ticks). Now, we only need to visit a sliced list at each increment and calculate each increment's price average. The reduce function provides a way of accumulating (or folding over) each item in a list and producing a result based on those values. We'll take our partitioned list and have reduce apply an accumulator function on each sublist. The form looks roughly like this:

(reduce (fn [accumulated-result-so-far each-item-list] 
          ;; ** single line comments come after semi-colons
          ;; do stuff, return accumulated result 
          ;; for each item iteration 
         )
    initial-list-of-accumulated-values
    initial-list-of-input-values)

Our accumulator function only has to maintain a few temporary values that our equation needs. In this case, it's the sum of each-item-list (or lazy sequence). For this, we want to contain or bind intermediate values in a temporary context. The let macro gives us a scope or temporary place to set and use some vars.

To begin with, let's get a visualization of how our list will be traversed. We want to look at 20 price points or ticks at a time. Since this is a running average, our function will need to take a look at the first 20 ticks before getting the first average. By doubling the amount of ticks given to partition, it will give you a list of the first 20. It will then increment this number by 1 until the 40th tick (39) is reached. Also, since partition doesn't want to give you a list that contains less than 20 ticks, it will discard any list above this number. Thus, by constraint, some lists only have 19 items at the most, that is, a partitioned list at the 20th point will have the remaining ticks up to 39:

(partition 20 1 '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39))
'((0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19)
(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20)
(2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21)
(3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
(4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23)
(5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24)
(6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25)
(7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26)
(8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27)
(9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28)
(10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29)
(11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30)
(12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31)
(13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32)
(14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33)
(15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34)
(16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35)
(17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36)
(18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37)
(19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38)
(20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39))

This is how partition will behave in the following function. We need to double the given tick window and remember to take this amount as the input in an infinite sequence. When reducing over each partitioned list, we calculate the sum, and then the average. With the average for our given list in place, we'll add it to the existing last-trade-price and last-trade-time (merge and zipmap). We'll then add these to the tail position of our list (using the function lazy-cat). Let's write our simple-moving-average function now. In your leiningen project, create a new file called analytics.clj under the directory structure, src/edgar. The final file path should be src/edgar/analytics.clj. At the top of this file, declare the namespace with the (ns edgar.analytics) form. In here, we'll also want to use the code from our edgar.core namespace. Here's how we can do this:

(ns edgar.analytics
   (:require [edgaru.core :as core]))

Now we can start to write our function as follows:

(defn simple-moving-average
"This is an optional documentation string that can be passed to your functions definition. Here, you can put any relevant information about the function and its definition. For example, we can note down the meaning of the 'option' argument."

   Options are:
   :input - input key function will look for (defaults to :last-trade-price)
   :output - output key function will emit (defaults to :last-trade-price-average)
   :etal - other keys to emit in each result map
   ** This function assumes the latest tick is on the left**"
  [options tick-window tick-list]

  (let [start-index tick-window

        {input-key :input
        output-key :output
        etal-keys :etal
        :or {input-key :last-trade-price
              output-key :last-trade-price-average
              etal-keys [:last-trade-price :last-trade-time]}} options]

    (reduce (fn [rslt ech]

              (let [tsum (reduce (fn [rr ee]
                                  (let [ltprice (:last-trade-price ee)]
                                    (+ ltprice rr))) 0 ech)

                    taverage (/ tsum (count ech))]

                (lazy-cat rslt
                          [(merge
                            (zipmap etal-keys
                                    (map #(% (last ech)) etal-keys))
                            {output-key taverage
                              :population ech})])))
            '()
            (partition tick-window
                      1
                      (take (* 2 tick-window) tick-list)))))

Let's inspect this slowly for good measure:

  • All the code takes place inside a let binding (http://clojuredocs.org/clojure.core/let). In Clojure, let is a special function that lets us assign values to symbols that we can use inside the let block. So, start-index is assigned the value of tick-window.
  • The next expression is a trick called destructuring (https://gist.github.com/john2x/e1dca953548bfdfb9844). This is simply a technique to extract values from a collection or map and assign them to symbols. So, in our preceding example, the values from the options map are extracted with the :input, :output, and :etal keys. These values are assigned to the symbols, input-key, output-key, and etal-keys, respectively. If any of the keys, such as :input, :output, or :etal, cannot be found, then default values of :last-trade-price, :last-trade-price-average, or [:last-trade-price :last-trade-time]}} are assigned.
  • Our inner map (within lazy-cat) uses an anonymous #() function, where the % sign is an implicit parameter or each item is mapped over.
  • Recall that our input data looks like the following list. Here, ech, an argument of the reducing function, represents each list that the partition function generated:
      ({:last-trade-price {:last 11.08212939452855}, 
   :last-trade-time #<DateTime 2015-04-19T17:56:57.596Z>}
    {:last-trade-price {:last 15.423964294614002}, 
    :last-trade-time #<DateTime 2015-04-19T17:56:58.596Z>}
    {:last-trade-price {:last 15.737626638301053}, 
    :last-trade-time #<DateTime 2015-04-19T17:57:00.596Z>}
    {:last-trade-price {:last 17.94475850354465}, 
    :last-trade-time #<DateTime 2015-04-19T17:57:01.596Z>}
    {:last-trade-price {:last 16.66187358467036}, 
    :last-trade-time #<DateTime 2015-04-19T17:57:02.596Z>}
    {:last-trade-price {:last 17.521292102081574}, 
    :last-trade-time #<DateTime 2015-04-19T17:57:03.596Z>}
    {:last-trade-price {:last 19.310727826293594}, 
    :last-trade-time #<DateTime 2015-04-19T17:57:06.596Z>}
    {:last-trade-price {:last 16.444691958028105}, 
    :last-trade-time #<DateTime 2015-04-19T17:57:09.596Z>}
    ...)
  • etal-keys is a [:last-trade-price :last-trade-time] vector that is created in the let block's binding form. So, the (map #(% (last ech)) etal-keys) expression maps an anonymous function over this vector. Recall that keywords also act as functions on maps. This means that we're calling :last-trade-price and :last-trade-time on a map, that looks like { :last-trade-price {:last 11.08212939452855}, :last-trade-time #inst "2015-09-24T04:13:13.868-00:00" }.
  • The (map #(% (first ech)) etal-keys) expression pulls out a list of values that look like ({:last 5.466160487301605} #inst "2015-09-24T04:13:13.868-00:00"). This is done for each element that is reduced over.
  • We'll then rejoin the aforementioned values to the extraction keys that we just used. The zipmap function does this by interleaving values in two or more lists to produce a map:
    (def etal-keys [:last-trade-price :last-trade-time])

(map #(% {:last-trade-price {:last 5.466160487301605}, :last-trade-time #inst "2015-09-24T04:13:13.868-00:00"})
     etal-keys)
;; ({:last 5.466160487301605}#inst "2015-09-24T04:13:13.868-00:00")

(zipmap [:a :b] [1 2])
;; {:b 2, :a 1}

(zipmap etal-keys
        (map #(% {:last-trade-price {:last 5.466160487301605}, :last-trade-time #inst "2015-09-24T04:13:13.868-00:00"})
             etal-keys))
;; {:last-trade-time #inst "2015-09-24T04:13:13.868-00:00", :last-trade-price {:last 5.466160487301605}}

There is an actual interleave function that produces a result list (though this is not what we want). We want to produce a result map, which is what the zipmap function provides:

(interleave '(1 2 3 4) '(:a :b :c :d))
;; (1 :a 2 :b 3 :c 4 :d)

We'll then take the map result and merge in our new average value. The merge function is simple enough, simply joining together the values of two maps. We can then employ this function to merge together the result of the zipmap call to the new zipmap entry that we want to add:

(merge {:a 1 :b 2} {:c 3})
;; {:c 3, :b 2, :a 1}

(merge
  (zipmap etal-keys
          (map #(% {:last-trade-price 6.111604269033163, :last-trade-time trade-time})
               etal-keys))
{:last-trade-price-average 6.0972436})

;; {:last-trade-price-average 6.0972436,
;;  :last-trade-time #inst "2015-06-28T18:31:51.810-00:00",
;;  :last-trade-price 6.111604269033163}

Adding the result of this to our running (reduced) list is a bit trickier. There's a small semantic inefficiency that we have to manage. Our running time series is infinite and the newest values on the right. The reduce function is also known as a left fold (see https://en.wikipedia.org/wiki/Fold_(higher-order_function)), which means that it collects values from left to right. So far, we've been able to blithely ignore the ordering of our lists. However, if we cons or conj an item into the result list, it will always go to the head of the list, which is on the left. Recall that the list implementation decides how to append items based on its own knowledge of what's efficient. In this case, the left placement happens to be the most efficient way to append an item. If each rslt collection were a vector, then conj would add the new item to the end (or right) of the vector, instead of adding it to the head (or left) of it. This is just an alternative path to keep in mind:

(conj '(1 2 3) 4)
;; (4 1 2 3)

(cons 4 '(1 2 3))
;; (4 1 2 3)

Since we are dealing with a time series, however, the ordering of the list is crucially important. Thus, we need to place the result to the right-hand side of the list. This is tricky because we'll need to switch the kind of collection in which we place the result so that it goes to the collection's end. The concat function joins the contents of two lists and puts the addition on the right-hand side. The lazy-cat macro does the same thing but respects the semantics of lazy sequences, only realizing values when invoked. The key thing here is that both parameters need to be lists (or vectors). If we pass only a map to lazy-cat, it will do the next best thing it knows, which is to get the lists to join together with the sequenced out the entries in the map:

(concat [1 2] [3])
;; (1 2 3)

(lazy-cat [1 2] [3])
;; (1 2 3)

;; Not what we want
(lazy-cat '(1)
          (merge
            (zipmap etal-keys
                    (map #(% {:last-trade-price 6.111604269033163, :last-trade-time trade-time})
                         etal-keys))
            {:last-trade-price-exponential 6.0972436}))
;; (1
;;  [:last-trade-price-exponential 6.0972436]
;;  [:last-trade-time #inst "2015-06-28T18:31:51.810-00:00"]
;;  [:last-trade-price 6.111604269033163])

(seq {:a 1 :b 2})
;; ([:b 2] [:a 1])

In the subsequent code, we are forced to put the result in its own list or vector in this case. This will add the result to the end (or right-hand side) of the running result list.

;; What we want
(lazy-cat '(1)
          [(merge
              (zipmap etal-keys
                      (map #(% {:last-trade-price 6.111604269033163, :last-trade-time trade-time})
                           etal-keys))
              {:last-trade-price-average 6.0972436})])
;; (1
;;  {:last-trade-price-average 6.0972436,
;;   :last-trade-time #inst "2015-06-28T18:31:51.810-00:00",
;;   :last-trade-price 6.111604269033163})

We now properly append our average results to the right-hand side of the list, which corresponds to how the input was originally partitioned. To call simple-moving-average, we'll operate on our timeseries sequence, which was defined in the Reasoning about the equation needed to achieve our output section. Remember that the first average represents the first 20 price points, thus corresponding to the 20th item in our price list. The following algorithms will continue along this line:

(simple-moving-average {} 20 timeseries)
({:last-trade-price {:last 5.466160487301605},
  :last-trade-time #inst "2015-09-24T04:13:13.868-00:00",
  :last-trade-price-average 7.194490217405031,
  :population
  ({:last-trade-time #inst "2015-09-24T04:13:13.868-00:00",
    :last-trade-price {:last 5.466160487301605}}
   {:last-trade-time #inst "2015-09-24T04:13:15.868-00:00",
    :last-trade-price {:last 6.540895364039775}}
   ...)}
 {:last-trade-price {:last 6.540895364039775},
  :last-trade-time #inst "2015-09-24T04:13:15.868-00:00",
  :last-trade-price-average 7.180900526986235,
  :population
  ({:last-trade-time #inst "2015-09-24T04:13:15.868-00:00",
    :last-trade-price {:last 6.540895364039775}}
   {:last-trade-time #inst "2015-09-24T04:13:16.868-00:00",
    :last-trade-price {:last 5.53301182972796}}
   ...)}
...)
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.217.199.122