Using array/numeric libraries for efficiency

You may have noticed in the previous sections, when working with numerics, performance depends a lot on whether the data is based on arrays and primitives. It may take a lot of meticulousness on the programmer's part to correctly coerce data into primitives and arrays at all stages of the computation in order to achieve optimum efficiency. Fortunately, the high-performance enthusiasts from the Clojure community realized this issue early on and created some dedicated open source libraries to mitigate the problem.

HipHip

HipHip is a Clojure library used to work with arrays of primitive types. It provides a safety net, that is, it strictly accepts only primitive array arguments to work with. As a result, passing silently boxed primitive arrays as arguments always results in an exception. HipHip macros and functions rarely need the programmer to type hint anything during the operations. It supports arrays of primitive types such as int, long, float, and double.

The HipHip project is available at https://github.com/Prismatic/hiphip.

As of writing, HipHip's most recent version is 0.2.0 that supports Clojure 1.5.x or above, and is tagged as an Alpha release. There is a standard set of operations provided by HipHip for arrays of all of the four primitive types: integer array operations are in the namespace hiphip.int; double precision array operations in hiphip.double; and so on. The operations are all type hinted for the respective types. All of the operations for int, long, float, and double in respective namespaces are essentially the same except for the array type:

Category

Function/macro

Description

Core functions

aclone

Like clojure.core/aclone, for primitives

 

alength

Like clojure.core/alength, for primitives

 

aget

Like clojure.core/aget, for primitives

 

aset

Like clojure.core/aset, for primitives

 

ainc

Increment array element by specified value

Equiv hiphip.array operations

amake

Make a new array and fill values computed by expression

 

areduce

Like clojure.core/areduce, with HipHip array bindings

 

doarr

Like clojure.core/doseq, with HipHip array bindings

 

amap

Like clojure.core/for, creates new array

 

afill!

Like preceding amap, but overwrites array argument

Mathy operations

asum

Compute sum of array elements using expression

 

aproduct

Compute product of array elements using expression

 

amean

Compute mean over array elements

 

dot-product

Compute dot product of two arrays

Finding minimum/maximum, Sorting

amax-index

Find maximum value in array and return the index

 

amax

Find maximum value in array and return it

 

amin-index

Find minimum value in array and return the index

 

amin

Find minimum value in array and return it

 

apartition!

Three-way partition of array: less, equal, greater than pivot

 

aselect!

Gather smallest k elements at the beginning of array

 

asort!

Sort array in-place using Java's built-in implementation

 

asort-max!

Partial in-place sort gathering top k elements to the end

 

asort-min!

Partial in-place sort gathering min k elements to the top

 

apartition-indices!

Like apartition! but mutates index-array instead of values

 

aselect-indices!

Like aselect! but mutates index-array instead of values

 

asort-indices!

Like asort! but mutates index-array instead of values

 

amax-indices

Get index-array; last k indices pointing to max k values

 

amin-indices

Get index-array; first k indices pointing to min k values

To include HipHip as a dependency in your Leiningen project, specify it in project.clj:

:dependencies [;; other dependencies
               [prismatic/hiphip "0.2.0"]]

As an example of how to use HipHip, let's see how to compute the normalized values of an array:

(require '[hiphip.double :as hd])

(def xs (double-array [12.3 23.4 34.5 45.6 56.7 67.8]))

(let [s (hd/asum xs)] (hd/amap [x xs] (/ x s)))

Unless we make sure that xs is an array of primitive doubles, HipHip will throw ClassCastException when the type is incorrect, and IllegalArgumentException in other cases. I recommend exploring the HipHip project to gain more insight into using it effectively.

primitive-math

We can set *warn-on-reflection* to true to let Clojure warn us when the reflection is used at invocation boundaries. However, when Clojure has to implicitly use reflection to perform math, the only resort is to either use a profiler or compile the Clojure source down to bytecode, and analyze boxing and reflection with a decompiler. This is where the primitive-math library helps, by producing extra warnings and throwing exceptions.

The primitive-math library is available at https://github.com/ztellman/primitive-math.

As of writing, primitive-math is at version 0.1.4; you can include it as a dependency in your Leiningen project by editing project.clj as follows:

:dependencies [;; other dependencies
               [primitive-math "0.1.4"]]

The following code is how it can be used (recall the example from the Decompiling the .class files into Java source section):

;; must enable reflection warnings for extra warnings from primitive-math
(set! *warn-on-reflection* true)
(require '[primitive-math :as pm])
(defn mul [x y] (pm/* x y))  ; primitive-math produces reflection warning
(mul 10.3 2)                        ; throws exception
(defn mul [^long x ^long y] (pm/* x y))  ; no warning after type hinting
(mul 10.3 2)  ; returns 20

While primitive-math is a useful library, the problem it solves is mostly taken care of by the boxing detection feature in Clojure 1.7 (see next section Detecting boxed math). However, this library is still useful if you are unable to use Clojure 1.7 or higher.

Detecting boxed math

Boxed math is hard to detect and is a source of performance issues. Clojure 1.7 introduces a way to warn the user when boxed math happens. This can be configured in the following way:

(set! *unchecked-math* :warn-on-boxed)

(defn sum-till [n] (/ (* n (inc n)) 2))  ; causes warning
Boxed math warning, /private/var/folders/cv/myzdv_vd675g4l7y92jx9bm5lflvxq/T/form-init3701519533014890866.clj:1:28 - call: public static java.lang.Number clojure.lang.Numbers.unchecked_inc(java.lang.Object).
Boxed math warning, /private/var/folders/cv/myzdv_vd675g4l7y92jx9bm5lflvxq/T/form-init3701519533014890866.clj:1:23 - call: public static java.lang.Number clojure.lang.Numbers.unchecked_multiply(java.lang.Object,java.lang.Object).
Boxed math warning, /private/var/folders/cv/myzdv_vd675g4l7y92jx9bm5lflvxq/T/form-init3701519533014890866.clj:1:20 - call: public static java.lang.Number clojure.lang.Numbers.divide(java.lang.Object,long).

;; now we define again with type hint
(defn sum-till [^long n] (/ (* n (inc n)) 2))

When working with Leiningen, you can enable boxed math warnings by putting the following entry in the project.clj file:

:global-vars {*unchecked-math* :warn-on-boxed}

The math operations in primitive-math (like HipHip) are implemented via macros. Therefore, they cannot be used as higher order functions and, as a consequence, may not compose well with other code. I recommend exploring the project to see what suits your program use case. Adopting Clojure 1.7 obviates the boxing discovery issues by means of a boxed-warning feature.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.216.183.63