You want to test individual units of Clojure code.

Clojure includes a unit-testing framework in its clojure.test namespace. It provides ways to name and group tests, make assertions, report results, and orchestrate test suites.

For demonstration, imagine you had a capitalize-entries function that capitalized values in a map. To test this function, define a test using clojure.test/deftest:

;; A function in namespace com.example.core
(defn capitalize-entries
  "Returns a new map with values for keys 'ks' in the map 'm' capitalized."
  [m & ks]
  (reduce (fn [m k] (update-in m [k] clojure.string/capitalize)) m ks))

;; The corresponding test in namespace com.example.core-test
(require '[clojure.test :refer :all])

;; In a real test namespace, you would also :refer all of the target namespace
;; (require '[com.example.core :refer :all])

(deftest test-capitalize-entries
  (let [employee {:last-name "smith"
                  :job-title "engineer"
                  :level 5
                  :office "seattle"}]
    ;; Passes
    (is (= (capitalize-entries employee :job-title :last-name)
           {:job-title "Engineer"
            :last-name "Smith"
            :office "seattle"
            :level 5}))
    ;; Fails
    (is (= (capitalize-entries employee :office)
           {}))))

Run the test with the clojure.test/run-tests function:

(run-tests)
;; -> {:type :summary, :pass 1, :test 1, :error 0, :fail 1}
;; *out*
;; Testing user
;;
;; FAIL in (test-capitalize-entries) (NO_SOURCE_FILE:13)
;; expected: (= (capitalize-entries employee :office) {})
;;   actual: (not (= {:last-name "smith", :office "Seattle",
;;                    :level 5, :job-title "engineer"} {}))
;;
;; Ran 1 tests containing 2 assertions.
;; 1 failures, 0 errors.

The preceding example only scratches the surface of what clojure.test provides for unit testing. Let’s take a bottom-up look at its other features.

First, you can improve reporting when an assertion fails by providing a second argument that explains what the assertion is intended to test. When you run this test, you will see an extended description of how the code was expected to behave:

(is (= (capitalize-entries {:office "space"} :office) {})
    "The employee's office entry should be capitalized.")
;; -> false
;; * out*
;; FAIL in clojure.lang.PersistentList$EmptyList@1 (NO_SOURCE_FILE:1)
;; The employee's office entry should be capitalized.
;; expected: (= (capitalize-entries {:office "space"} :office) {})
;;   actual: (not (= {:office "Space"} {}))

For testing a function like capitalize-entries thoroughly, several use cases need to be considered. To more concisely test numerous similar cases, use the clojure.test/are macro:

(deftest test-capitalize-entries
  (let [employee {:last-name "smith"
                  :job-title "engineer"
                  :level 5
                  :office "seattle"}]
    (are [ks m] (= (apply capitalize-entries employee ks) m)
         [] employee
         [:not-a-key] employee
         [:job-title] {:job-title "Engineer"
                       :last-name "smith"
                       :level 5
                       :office "seattle"}
         [:last-name :office] {:last-name "Smith"
                               :office "Seattle"
                               :level 5
                               :job-title "engineer"})))

The first two parameters to are set up a testing pattern: given a sequence of keys ks and a map m, call capitalize-entries for those keys on the original employee map and assert that the return value equals m.

Writing out multiple use cases in a declarative syntax makes it easier to catch errors and untreated edge cases, such as the NullPointerException that will be thrown for the [:not-a-key] employee assertion pair in the preceding test.

Unlike testing frameworks for other popular dynamic languages, Clojure’s built-in assertions are minimal and simple. The is and are macros check test expressions for “truthiness” (i.e., that those expressions return neither false nor nil, in which case they pass). Beyond this, you can also check for thrown? or thrown-with-msg? to test that a certain java.lang.Throwable (error or exception) is expected:

(is (thrown? IndexOutOfBoundsException (nth [] 1)))

Above the level of individual assertions, clojure.test also provides facilities for calling functions before or after tests run. In the test-capitalize-entries test, we defined an ad hoc employee map for testing, but you could also read in external data to be shared across multiple tests by registering a data-loading function as a “fixture.” The clojure.test/use-fixtures multimethod allows registering Clojure functions to be called either before or after each test, or before or after an entire namespace’s test suite. The following example defines and registers three fixture functions:

(require '[clojure.edn :as edn])

(def test-data (atom nil))

;; Assuming you have a test-data.edn file...
(defn load-data "Read a Clojure map from test data in a file."
  [test-fn]
  (reset! test-data (edn/read-string (slurp "test-data.edn")))
  (test-fn))

(defn add-test-id "Add a unique id to the data before each test."
  [test-fn]
  (swap! test-data assoc :id (java.util.UUID/randomUUID))
  (test-fn))

(defn inc-count "Increment a counter in the data after each test runs."
  [test-fn]
  (test-fn)
  (swap! test-data update-in [:count] (fnil inc 0)))

(use-fixtures :once load-data)
(use-fixtures :each add-test-id inc-count)

;; Tests...

You can think about fixture functions as forming a pipeline through which each test is passed as a parameter, which we called test-fn in the preceding example. Take inc-count, for example. It is the job of this fixture to invoke the test-fn function, continuing the pipeline, and afterward, to increment a count (i.e., “do some work”). Each fixture decides whether to invoke test-fn before or after its own work (compare the add-test-id function with the inc-count function), while the clojure.test/use-fixtures multimethod controls whether each registered fixture function is run only once for all tests in a namespace or once for each test.

Finally, with a firm understanding of how to develop individual Clojure test suites, it is important to consider how you organize and run those suites as part of your project’s build. Although Clojure allows defining tests for functions anywhere in your code base, you should keep your testing code in a separate directory that is only added to the JVM classpath when needed (e.g., during development and testing). It is conventional to name your test namespaces after the namespaces they test, so that a file located at <project-root>/src/com/example/core.clj with namespace com.example.core has a corresponding test file at <project-root>/test/com/example/core_test.clj with namespace com.example.core-test. To control the location of your source and test directories and their inclusion on the JVM classpath, you should use a build tool like Leiningen or Maven to organize your project.

In Leiningen, the default directory for your tests is a top-level <project-root>/test folder, and you can run your project’s tests with lein test at the command line. Without any additional arguments, the lein test command will execute all of the tests in a project:

$ lein test

lein test com.example.core-test
lein test com.example.util-test

Ran 10 tests containing 20 assertions.
0 failures, 0 errors.

To limit the scope of tests Leiningen runs, use the :only option, followed by a fully qualified namespace or function name:

# To run an entire namespace
$ lein test :only com.example.core-test

lein test com.example.core-test

Ran 5 tests containing 10 assertions.
0 failures, 0 errors.

# To run one specific test
$ lein test :only com.example.core-test/test-capitalize-entries

lein test com.example.core-test

Ran 1 tests containing 2 assertions.
0 failures, 0 errors.

You want to unit-test a function that integrates with external dependencies such as HTTP services or databases.

Use Midje, a testing framework that provides ways to mock functions and return fakes.

To follow along with this recipe, start a REPL using lein-try:

$ lein try midje clj-http

Here is an example function that makes an HTTP request:

;; A function in namespace com.example.core
(require '[clj-http.client :as http])

(defn github-profile [username]
  (let [response (http/get (str "https://api.github.com/users/" username))]
    (when (= (:status response) 200)
      (:body response))))

(github-profile "clojure-cookbook")
;; -> "{"login":"clojure-cookbook","id":4176246, ...}"

To test the github-profile function, define a test using midje.sweet/facts and midje.sweet/fact in the corresponding test namespace:

;; In the com.example.core-test namespace...
(require '[midje.sweet :refer :all])

(facts "about successful requests"
  (fact "returns the response body"
    (github-profile "clojure-cookbook") => ..body..
    (provided
      (http/get #"/users/clojure-cookbook") =>
        {:status 200 :body ..body..})))

In Midje, facts associates a description with a group of tests, while fact maps to your test. Assertions in your fact take the form of:

;; actual => expected

10 => 10 ; This will pass
10 => 11 ; This will fail

Assertions behave a little differently than most testing frameworks. Within a fact body, every single assertion is checked, irrespective of whether a previous one failed.

Midje only provides mocks, not stubs. All functions specified in the provided body have to be called for the test to pass. Mocks use the same syntax as assertions, but with a slightly different meaning:

;; <function call & arguments to match> => <return value of function>

(provided (+ 10 10) => 0)

It is important to note you are not calling the (+ 10 10) function here—you are setting up a pattern. Every function call occurring in the test is checked to see if it matches this pattern. If it does match, Midje will not call the function, but will instead return 0. When defining mocks with provided, there is a lot of flexibility in terms of how to match mock functions against real calls. In the preceding solution, for example, regular expressions are used. This expression instructs Midje to mock calls to http/get whose URLs end in /users/clojure-cookbook:

;; The expectation
(http/get #"/users/clojure-cookbook$")

;; Would match
(http/get "http://localhost:4001/users/clojure-cookbook")
;; or
(http/get "https://api.github.com/users/clojure-cookbook")

Midje provides a lot of match-shaping functions that you can use to match against the arguments of a mock:

;; Match an argument list that contains 1
(provided
  (http/get (contains [1])) => :result)

;; Match against a custom fn that must return true
(provided
  (http/get (as-checker (fn [x] (x == 10)))) => :result)

;; Match against a single argument of any value
(provided
  (http/get anything) => :result)

From within a REPL, you can investigate all of Midje’s checkers:

(require 'midje.repl)
(doc midje-checkers)
;; *out*
;; -------------------------
;; midje.sweet/midje-checkers
;;
;;  (facts "about checkers"
;;    (f) => truthy
;;    (f) => falsey
;;    (f) => irrelevant ; or `anything`
;;    (f) => (exactly odd?) ; when you expect a particular function
;;    (f) => (roughly 10 0.1)
;;    (f) => (throws SomeException #"with message")
;;    (f) => (contains [1 2 3]) ; works with strings, maps, etc.
;;    (f) => (contains [1 2 3] :in-any-order :gaps-ok)
;;    (f) => (just [1 2 3])
;;    (f) => (has every? odd?)
;;    (f) => (nine-of odd?) ; must be exactly 9 odd values.
;;    (f) => (every-checker odd? (roughly 9)) ; both must be true
;;    (f) => (some-checker odd? (roughly 9))) ; one must be true

You may have noticed in the solution that we used ..body.. instead of an actual response. This is something Midje refers to as a metaconstant.

A metaconstant is any name that starts and ends with two dots. It has no properties other than identity. Think of it as a fake or placeholder, where we do not care about the actual value or might be referencing something that does not exist yet. In our example, we don’t really care what ..body.. is; we just care that it is the thing returned.

To add Midje to an existing project, add [midje "1.5.1"] to your development dependencies and [lein-midje "3.1.2"] to your development plug-ins. Your project.clj should look something like this:

(defproject example "1.0.0-SNAPSHOT"
  :profiles {:dev {:dependencies [[midje "1.5.1"]]
                   :plugins [[lein-midje "3.1.2"]}})

Midje provides two ways to run tests: through a REPL, as you may have been doing, or through Leiningen. Midje actually encourages you to run all your tests through the REPL, as you develop them. One very useful way to run your tests is with the midje.repl/autotest function. This continuously polls the filesystem looking for changes in your project. When it detects these changes, it will automatically rerun the relevant tests:

(require '[midje.repl :as midje])

(midje/autotest) ; Start auto-testing

;; Other options are...
(midje/autotest :pause)
(midje/autotest :resume)
(midje/autotest :stop)

There are many more things you can do from the REPL with Midje. To find out more, read the docstring of midje-repl by running (doc midje-repl) in a REPL.

You can also run Midje tests through the Leiningen plug-in lein-midje (add as noted in project.clj). lein-midje allows you run tests at a number of granularities—all of your tests, all the tests in a group, or all the tests in a single namespace:

# Run all your tests
$ lein midje

# Run a group of namespaces
$ lein midje com.example.*

# Run a specific namespace
$ lein midje com.example.t-core

You want to test a function using randomly generated inputs to ensure that it works in all possible scenarios.

Use the test.generative library to specify a function’s inputs, and test it across randomly generated values.

To follow along with this recipe, start a REPL using lein-try:

$ lein try org.clojure/test.generative "0.5.0"

Say you are trying to test the following function, which calculates the arithmetic mean of all the numbers in a sequence:

(defn mean
  "Calculate the mean of the numbers in a sequence"
  [s]
  (/ (reduce + s) (count s)))

The following test.generative code defines a specification for the mean function:

(require '[clojure.test.generative :as t]
         '[clojure.test.generative.runner :as r]
         '[clojure.data.generators :as gen])

(defn number
  "Return a random number, of a random type"
  []
  (gen/one-of gen/byte
              gen/short
              gen/int
              gen/long
              gen/float
              gen/double))

(defn seq-of-numbers
  "Return a list, seq, or set of numbers"
  []
  (gen/one-of (gen/list number)
              (gen/set number)
              (gen/vec number)))

(t/defspec mean-spec
  mean
  [^example.generative-tests/seq-of-numbers arg]
  (assert (number? %)))

To run the mean-spec specification, invoke the run function in the clojure.test.generative.runner namespace, passing in the number of threads upon which to run the simulation, the number of milliseconds to run, and the var referring to a spec.

Here’s what happens when we run the previous example at the REPL:

(r/run 2 5000 #'example.generative-tests/mean-spec)
;; -> clojure.lang.ExceptionInfo: Generative test failed

This shows the behavior when the generative test fails. The exact details of the failure are returned as the data of a Clojure information-bearing exception; you must retrieve an instance of the exception itself and call ex-data on it to return the data map.

In the REPL, if you didn’t explicitly catch the exception, you can use the special *e symbol to retrieve the most recent exception. Calling ex-data on it returns information on the test case that provoked the error:

(ex-data *e)
;; -> {:exception #<ArithmeticException java.lang.ArithmeticException:
;;     Divide by zero>, :iter 7, :seed -875080314,
;;     :example.generative-tests/mean-spec, :input [#{}]}

This states that after only seven iterations, using the random number seed –875080314, the function under test was passed #{} as input and threw a divide by zero error.

Once highlighted in this way, the problem is easy to see; the mean function will divide by zero if (count s) is zero. Fix the bug by rewriting the mean function to handle that case:

(defn mean
  [s]
  (if (zero? (count s))
    0
    (/ (reduce + 1.0 s) (count s))))

Rerunning now shows a passing test:

(r/run 2 5000 #'example.generative-tests/mean-spec)
;; -> {:iter 3931, :seed -1495229764, :test testgen-test.core/mean-spec}
;;    {:iter 3909, :seed -1154113663, :test testgen-test.core/mean-spec}

This output indicates that over the allotted 5 seconds, two threads ran about 3,900 iterations of the test each and did not encounter any errors or assertion failures.

There are two key parts to the preceding test definition: the defspec form itself, which defines the generative test, and the functions used to generate random data. In this case, the data generator functions are built from primitive data generation functions found in the clojure.data.generators namespace.

Generator functions take no arguments and return random values. Different functions produce different types of data. The clojure.data.generators namespace contains generator functions for all of Clojure’s primitive types, as well as collections. It also contains functions for randomly choosing from a set of options; the one-of function used previously, for example, takes a number of generator functions and chooses a value from one at random.

The defspec macro takes three types of forms: a function to put under test, an argument specification, and a body containing one or more assertion forms.

The function under test is simply the function to call. Over the course of the generative test, it will be called many times, each time with different values.

The argument specification is a vector of argument names and should match the signature of the function under test. Each argument should have metadata attached. Specifically, it should have a :tag metadata key, mapped to the fully qualified name of a generator function. Each time the test driver calls the function, it will use a random value for each argument pulled from its corresponding generator function.

The body of a defspec simply contains expressions that may throw an exception if some condition is not met. It is executed on each iteration of the test, with the instantiated arguments available, and with the return value of the function under test bound to %. This example merely has a single assertion that the result is a number, for brevity, but you can have any number of assertions executing arbitrary checks.

An interesting difference between test.generative and traditional unit tests is that rather than specifying what tests to run and having them take as long as they do, in test.generative you specify how long to run, and the system will run as many random permutations of the test as it can fit into that time. This has the property of keeping test runtimes deterministic, while allowing you to trade off speed and comprehensiveness depending on the situation. For example, you might have tests run for five seconds in development, but thoroughly hammer the system for an hour every night on the continuous integration server, allowing you to find that (literally) one-in-a-million bug.

$ lein run -m clojure.test.generative.runner tests/generative

:jvm-opts ["-Dclojure.test.generative.threads=32"
           "-Dclojure.test.generative.msec=10000"]

You want to specify properties of a function that should hold true for all inputs, and find input values that violate those properties.

Use simple-check. This is a property-specification library for Clojure that is capable of “shrinking” the input case to find the minimal failing input.^[35]

To follow along with this recipe, add [reiddraper/simple-check "0.5.3"] to your project’s dependencies, or start a REPL using lein-try:

$ lein try reiddraper/simple-check

Then, find a function to test. This example uses a contrived function that calculates the sum of the reciprocals of a sequence of numbers:

(defn reciprocal-sum [s]
  (reduce + (map (partial / 1) s)))

Here’s the test code itself:

(require '[simple-check.core :as sc]
         '[simple-check.generators :as gen]
         '[simple-check.properties :as prop])

(def seq-of-numbers (gen/one-of [(gen/vector gen/int)
                                 (gen/list gen/int)]))

(def reciprocal-sum-check
  (prop/for-all [s seq-of-numbers]
    (number? (reciprocal-sum s))))

seq-of-numbers is a data generator composed of primitive generators found in the simple-check.generators namespace.

The actual test is defined using the simple-check.properties/for-all macro, which emits a property definition. It takes a binding form (similar to let or for) that specifies the possible values to bind to one or more symbols, and a body. The body is what actually specifies the properties that must hold, and must return true if and only if the test passes for a particular set of values.

To run the test, invoke the simple-check.core/quick-check function, passing it the defined property:

(sc/quick-check 100 reciprocal-sum-check)

quick-check takes the number of samples to execute, and the property definition to execute. The body of the property definition will be sampled repeatedly, with randomized values bound to the symbols specified in the binding form.

As you may have already observed, the reciprocal-sum function has a problem: it will throw a “divide by zero” error if a zero is present in the input sequence. The quick-check function returns a data structure showcasing the problem:

{:result
 #<ArithmeticException java.lang.ArithmeticException: Divide by zero>,
 :failing-size 8,
 :num-tests 9,
 :fail [(5 0 0 -8 1 -2)],
 :shrunk
 {:total-nodes-visited 10,
  :depth 5,
  :result
  #<ArithmeticException java.lang.ArithmeticException: Divide by zero>,
  :smallest [(0)]}}

Fix the function by eliminating zero values:

(defn reciprocal-sum [s]
  (reduce + (map (partial / 1)
                 (filter (complement zero?) s))))

Rerunning the test now indicates success:

(sc/quick-check 100 reciprocal-sum-check)
;; -> {:result true, :num-tests 100, :seed 1384622907885}

simple-check has the very useful property of not only returning a failing sample input to a test, but returning the minimal failing sample. In the preceding example program, for instance, any time a zero occurs in the input sequence, it causes an error. However, merely from looking at the sequence (5 0 0 -8 1 -2), it might not be apparent that zeros are the problem. Not knowing anything else about the function under test, the problem might be, for example, the negative numbers, or the value 5. simple-check returns not just any arbitrary failing input, but the specific input that will consistently cause the program to fail. As useful as it is to know that there is an input that will provoke failure, it’s even more useful to know the specific problematic value. And, the larger and more complex the inputs to the function are, the more useful it is to be able to reduce the failing case.

You want to run browser-based tests.

Use Selenium WebDriver via the clj-webdriver library. This will allow you to use clojure.test to test your application’s behavior in actual browser environments.

To follow along with this recipe, create a new Leiningen project:

$ lein new browser-testing
Generating a project called browser-testing based on the 'default' template.

Modify the new project’s project.clj file to match the following:

(defproject browser-testing "0.1.0-SNAPSHOT"
  :profiles {:dev {:dependencies [[clj-webdriver "0.6.0"]]}}
  :test-selectors {:default (complement :browser)
                   :browser :browser})

Next, add a simple Selenium test to test/browser_testing/core_test.clj, overwriting its content:

(ns browser-testing.core-test
  (:require [clojure.test :refer :all]
            [clj-webdriver.taxi :as t]))

;; A simple fixture that sets up a test driver
(defn selenium-fixture
  [& browsers]
  (fn [test]
    (doseq [browser browsers]
      (println (str "
[ Testing " browser " ]"))
      (t/set-driver! {:browser browser})
      (test)
      (t/quit))))

(use-fixtures :once (selenium-fixture :firefox))

(deftest ^:browser test-clojure
  (t/to "http://clojure.org")

  (is (= (t/title) "Clojure - home"))
  (is (= (t/current-url) "http://example.com/")))

(deftest ^:browser test-clojure-download
  (t/to "http://clojure.org")
  (t/click {:xpath "//div[@class='menu']/*/a[text()='Download']"})

  (is (= (t/title) "Clojure - downloads"))
  (is (= (t/current-url) "http://clojure.org/downloads"))
  (is (re-find #"Rich Hickey" (t/text {:id "foot"}))))

$ git clone https://github.com/clojure-cookbook/browser-testing
$ cd browser-testing

Run the tests on the command line:

$ lein test :browser

lein test browser-testing.core-test

[ Testing :firefox ]

lein test :only browser-testing.core-test/test-clojure

FAIL in (test-clojure) (core_test.clj:20)
expected: (= (t/current-url) "http://example.com/")
  actual: (not (= "http://clojure.org/" "http://example.com/"))

Ran 2 tests containing 5 assertions.
1 failures, 0 errors.
Tests failed.

Browser tests verify that your application behaves as expected in your targeted browsers. They test the appearance and behavior of your application as rendered in the browser itself.

Manually testing applications in a browser is a tedious and repetitive task. The amount of time and effort required for a complete test run can be unmanageable for even a moderately sized project. Automating browser tests ensures they are run consistently and relatively quickly, resulting in reproducible errors and more frequent test runs. However, automated tests lack the visual inspection by a human inherent to manual tests. For example, a manual test could easily catch a positioning error that an automated test would likely miss if it were not explicitly tested for.

To write browser tests in Clojure, use the clj-webdriver library with your preferred test framework, such as clojure.test. clj-webdriver provides a clean Clojure interface to Selenium WebDriver, a tool used to control and automate browser actions.

Some additional configuration may be required to use Selenium WebDriver or clj-webdriver with your browsers of choice. See the Selenium WebDriver documentation and the clj-webdriver wiki.

Before you dive into testing, you can experiment with clj-webdriver at a REPL. Start up a REPL with clj-webdriver using lein-try:

$ lein try clj-webdriver "0.6.0"

Use the clj-webdriver.taxi/set-driver! function, selecting the Firefox WebDriver implementation (other options include :chrome or :ie, but these may require more setup):

(require '[clj-webdriver.taxi :as t])

(t/set-driver! {:browser :firefox})
;; -> #clj_webdriver.driver.Driver{:webdriver ...}

This will open the browser you picked, ready to receive commands. Try a few functions from the clj-webdriver.taxi namespace:

(t/to "http://clojure.org/")

(t/current-url)
;; -> "http://clojure.org/"

(t/title)
;; -> "Clojure - home"

(t/click {:xpath "//div[@class='menu']/*/a[text()='Download']"})
(t/current-url)
;; -> "http://clojure.org/downloads"

(t/text {:id "foot"})
;; -> "Copyright 2008-2012 Rich Hickey"

When you’re finished, close the browser from the REPL:

(t/quit)

Your tests will use these functions to start up and run against the browser. To save yourself some work, you should set up the browser startup and teardown using a clojure.test fixture.

clojure.test/use-fixtures allows you to run functions around each individual test, or once around the namespace’s test run as a whole. Use the latter, as restarting the browser for each test will be far too slow.

The selenium-fixture function uses clj-webdriver’s set-driver! and quit functions to start up a browser for each of the keywords it’s provided and run the namespace’s tests inside that browser:

(defn selenium-fixture
  [& browsers]
  (fn [test]
    (doseq [browser browsers]
      (t/set-driver! {:browser browser})
      (test)
      (t/quit))))

(use-fixtures :once (selenium-fixture :firefox))

It’s important to note that using a :once fixture means the state of the browser will persist between tests. Depending on your particular application’s behavior, you may need to guard against this when you write your tests by beginning from a common browser state for each test. For example, you might delete all cookies or return to a certain top-level page. If this is necessary, you may find it useful to write this common reset behavior as an :each fixture.

To begin writing tests, modify your project’s project.clj file to include the clj-webdriver dependency in the :dev profile and :test-selectors for :default and browser convenience:

(defproject my-project "1.0.0-SNAPSHOT"
  ;; ...
  :profiles {:dev {:dependencies [[clj-webdriver "0.6.0"]]}}
  :test-selectors {:default (complement :browser)
                   :browser :browser})

Test selectors let you run groups of tests independently. This prevents slower browser tests from impacting the faster, more frequently run unit and lower-level integration tests.

In this case, you’ve added a new selector and modified the default. The new :browser selector will only match tests that have been annotated with a :browser metadata key. The default selector will now exclude any tests with this annotation.

With the fixture and test selectors in place, you can begin writing your tests. Start with something simple:

(deftest ^:browser test-clojure
  (t/to "http://clojure.org/")

  (is (= (t/title) "Clojure - home"))
  (is (= (t/current-url) "http://example.com/")))

Note the ^:browser metadata attached to the test. This test is annotated as a browser test, and will only run when that test selector is chosen.

In this test, as in the REPL experiment, you navigate to a URL and check its title and URL. Run this test at the command line, passing the additional test selector argument to lein test:

$ lein test :browser

lein test browser-testing.core-test

[ Testing :firefox ]

lein test :only browser-testing.core-test/test-clojure

FAIL in (test-clojure) (core_test.clj:20)
expected: (= (t/current-url) "http://example.com/")
  actual: (not (= "http://clojure.org/" "http://example.com/"))

Ran 2 tests containing 5 assertions.
1 failures, 0 errors.
Tests failed.

Clearly, this test was bound to fail—replace http://example.com/ with http://clojure.org/ and it will pass.

This test is very basic. In most real tests, you’ll load a URL, interact with the page, and verify that the application behaved as expected. Write another test that interacts with the page:

(deftest ^:browser test-clojure-download
  (t/to "http://clojure.org")
  (t/click {:xpath "//div[@class='menu']/*/a[text()='Download']"})

  (is (= (t/title) "Clojure - downloads"))
  (is (= (t/current-url) "http://clojure.org/downloads"))
  (is (re-find #"Rich Hickey" (t/text {:id "foot"}))))

In this test, after loading the URL, the browser is directed to click on an anchor located with an XPath selector. To verify that the expected page has loaded, the test compares the title and URL, as in the first test. Lastly, it finds the text content of the #foot element containing the copyright and verifies that the text includes the expected name.

clj-webdriver provides many other capabilities for interacting with your application. For more information, see the clj-webdriver wiki.

You want to trace the execution of your code, in order to see what it is doing.

Use the tools.trace library’s bevy of “trace” functions and macros to examine your code as it runs.

Before starting, add [org.clojure/tools.trace "0.7.6"] to your project’s dependencies under the :development profile (in the vector at the [:profiles :dev :dependencies] path instead of the [:dependencies] path). Alternatively, start a REPL using lein-try:

$ lein try org.clojure/tools.trace

To examine a single value at execution, wrap that value in an invocation of clojure.tools.trace/trace:

(require '[clojure.tools.trace :as t])

(map #(inc (t/trace %))
     (range 3))
;; -> (1 2 3)
;; *out*
;; TRACE: 0
;; TRACE: 1
;; TRACE: 2

To examine multiple values without losing context of which trace is which, supply a descriptive name string as the first argument to trace:

(defn divide
  [n d]
  (/ (t/trace "numerator" n)
     (t/trace "denominator" d)))

(divide 4 6)
;; -> 2/3
;; *out*
;; TRACE numerator: 4
;; TRACE denominator: 6

At its core, the tools.trace library is all about introspecting upon the execution of a body of code. The trace function is the simplest and most low-level tracing operation. Wrapping a value in an invocation of trace does two things: it logs a tracer message to STDOUT and, most importantly, returns the original value unadulterated. tools.trace provides a number of other granularities for tracing execution.

Stepping up a level from simple values, you can define functions with clojure.tools.trace/deftrace instead of defn to trace the input to and output from the function you define:

(t/deftrace pow [x n]
  (Math/pow x n))

(pow 2 3)
;; -> 8.0
;; *out*
;; TRACE t815: (pow 2 3)
;; TRACE t815: => 8.0

If you’re trying to diagnose a difficult-to-understand exception, use the clojure.tools.trace/trace-forms macro to wrap an expression and pinpoint the origin of the exception. When no exception occurs, trace-forms prints no output and returns normally:

(t/trace-forms (* (pow 2 3)
                  (divide 1 (- 1 1))))
;; *out*
;; ...
;; ArithmeticException Divide by zero
;;   Form failed: (divide 1 (- 1 1))
;;   Form failed: (* (pow 2 3) (divide 1 (- 1 1)))
;;   clojure.lang.Numbers.divide (Numbers.java:156)

Apart from explicitly tracing values or functions, tools.trace also allows you to dynamically trace vars or entire namespaces. To add a trace function to a var, use clojure.tools.trace/trace-vars. To remove such a trace, use clojure.tools.trace/untrace-vars:

(defn add [x y] (+ x y))

(t/trace-vars add)
(add 2 2)
;; -> 4
;; *out*
;; TRACE t1309: (user/add 2 2)
;; TRACE t1309: => 4

(t/untrace-vars add)
(add 2 2)
;; -> 4

To trace or untrace an entire namespace, use clojure.tools.trace/trace-ns and clojure.tools.trace/untrace-ns, respectively. This will dynamically add tracing to or remove it from all functions and vars in a namespace. Even things defined after trace-ns is invoked will be traced:

(def my-inc inc)
(defn my-dec [n] (dec n))

(t/trace-ns 'user)

(my-inc (my-dec 0))
;; -> 0
;; TRACE t1217: (user/my-dec 0)
;; TRACE t1218: | (user/my-dec 0)
;; TRACE t1218: | => -1
;; TRACE t1217: => -1
;; TRACE t1219: (user/my-inc -1)
;; TRACE t1220: | (user/my-inc -1)
;; TRACE t1220: | => 0
;; TRACE t1219: => 0

(t/untrace-ns 'user)

(my-inc (my-dec 0))
;; -> 0

You want to verify that your code handles nil correctly, eliminating potential null-pointer exceptions.

Use core.typed, an optional type system for Clojure, to annotate and check a namespace for misuses of nil.

To follow along with this recipe, create a file core_typed_samples.clj and start a REPL using lein-try:

$ touch core_typed_samples.clj
$ lein try org.clojure/core.typed

Consider, for example, that you are writing a function handle-number to process numbers. To verify that handle-number handles nil correctly, annotate it with clojure.core.typed/ann to accept the union (U) of the nil and Number types, returning a Number:

(ns core-typed-samples
  (:require [clojure.core.typed :refer [ann] :as t]))

(ann handle-number [(U nil Number) -> Number])
(defn handle-number [a]
  (+ a 20))

Verify the function’s correctness at the REPL using clojure.core.typed/check-ns:

user=> (require '[clojure.core.typed :as t])
user=> (t/check-ns 'core-typed-samples)
# ...
Type Error (core-typed-samples:6:3) Static method clojure.lang.Numbers/add
could not be applied to arguments:

Domains:
        t/AnyInteger t/AnyInteger
        java.lang.Number java.lang.Number

Arguments:
        (U nil java.lang.Number) (Value 20)

Ranges:
        t/AnyInteger
        java.lang.Number

with expected type:
        java.lang.Number

in: (clojure.lang.Numbers/add a 20)
in: (clojure.lang.Numbers/add a 20)

ExceptionInfo Type Checker: Found 1 error  clojure.core/ex-info (core.clj:4327)

The current definition is unsafe. check-ns recognizes that + can only handle numbers, while the handle-number function accepts numbers or nil.

Protect the call to + by wrapping it in an if statement, returning 0 in the absence of a:

(ns core-typed-samples
  (:require [clojure.core.typed :refer [ann] :as t]))

(ann handle-number [(U nil Number) -> Number])
(defn handle-number [a]
  (if a
    (+ a 20)
    0))

Check the namespace with check-ns again:

user=> (t/check-ns 'core-typed-samples)
# ...
:ok

Now that there is no way nil could accidentally be passed to + by this code, a null-pointer exception is impossible.

core.typed is designed to avoid all misuses of nil or null in typed code. To achieve this, the concepts of the null pointer and reference types are separated. This is unlike in Java, where a type like java.lang.Number implies a “nullable” type.

In core.typed, reference types are implicitly non-nullable. To express a nullable type (such as in the preceding example), construct a union type of the desired type and nil. For example, a java.lang.Number in core.typed syntax is non-nullable; the union type (U nil java.lang.Number) expresses the equivalent to a nullable java.lang.Number (the latter is closest to what java.lang.Number implies in Java type syntax).

This separation of concepts allows core.typed to throw a type error on any potential misuse of nil. The preceding solution threw a type error when type checking the equivalent expression: (+ nil 20).

To better understand core.typed type errors, it is useful to note that some functions have inline definitions. core.typed fully expands all code before type checking, so it is common to see calls to the Java method clojure.lang.Numbers/add in type errors when user code invokes clojure.core/+.

It is also common to see ordered intersection function types in type errors. Our first type error claims that the arguments (U Number nil) and (Value 20) are not under either of the ordered intersection function domains, listed under “Domains.” Notice two “Ranges” are provided, which correspond to the listed domains.

The full type of clojure.lang.Numbers/add is:

(Fn [t/AnyInteger t/AnyInteger -> t/AnyInteger]
    [Number Number -> Number])

Briefly, the function is “ordered” because it tries to match the argument types with each arity until one matches.

You want to verify that you are using Java libraries safely and unambiguously.

Java provides a vast ecosystem that is a major draw for Clojure developers; however, it can be often be complex to use large, cumbersome Java APIs from Clojure.

To type-check Java interop calls, use core.typed.

To follow along with this recipe, create a file core_typed_samples.clj and start a REPL using lein-try:

$ touch core_typed_samples.clj
$ lein try org.clojure/core.typed

To demonstrate, choose a standard Java API function such as the java.io.File constructor.

Using the dot constructor to create new files can be annoying—wrap it in a Clojure function that takes a string new-file:

(ns core-typed-samples
  (:require [clojure.core.typed :refer [ann] :as t])
  (:import (java.io File)))

(ann new-file [String -> File])
(defn new-file [s]
  (File. s))

Setting *warn-on-reflection* when compiling this namespace will tell us that there is a reflective call to the java.io.File constructor. Checking this namespace at the REPL with clojure.core.typed/check-ns will report the same information, albeit in the form of a type error:

user=> (require '[clojure.core.typed :as t])
user=> (t/check-ns 'core-typed-samples)
# ...
ExceptionInfo Internal Error (core-typed-samples:6)
  Unresolved constructor invocation java.io.File.

Hint: add type hints.

in: (new java.io.File s)  clojure.core/ex-info (core.clj:4327)

Add a type hint to call the public File(String pathname) constructor:

(ns core-typed-samples
  (:require [clojure.core.typed :refer [ann] :as t])
  (:import (java.io File)))

(ann new-file [String -> File])
(defn new-file [^String s]
  (File. s))

Checking again, core.type is satisfied:

user=> (t/check-ns 'core-typed-samples)
# ...
:ok

File has a second single-argument constructor: public File(URI uri). Enhance new-file to support URI or String filenames:

(ns core-typed-samples
  (:require [clojure.core.typed :refer [ann] :as t])
  (:import (java.io File)
           (java.net URI)))

(ann new-file [(U URI String) -> File])
(defn new-file [s]
  (if (string? s)
    (File. ^String s)
    (File. ^URI s)))

Even after relaxing the input type to (U URI String), core.typed is able to infer that each branch has the correct type by following the string? predicate.

While java.io.File is a relatively small API, careful inspection of Java types and documentation is needed to confidently use foreign Java code correctly.

Though the File constructor is fairly innocuous, consider writing file-parent, a thin wrapper over the getParent method:

(ns core-typed-samples
  (:require [clojure.core.typed :refer [ann] :as t])
  (:import (java.io File)))

(ann file-parent [File -> String])
(defn file-parent [^File f]
  (.getParent f))

The preceding implementation is free from reflective calls, so… all good? No. Checking this function with core.typed tells another story; Java’s return types are nullable and core.typed knows it. It is possible that getParent will return nil instead of a String:

user=> (t/check-ns 'core-typed-samples)
# ...
Type Error (core-typed-samples:7:3) Return type of instance method
java.io.File/getParent is (U java.lang.String nil), expected
java.lang.String.

Hint: Use `non-nil-return` and `nilable-param` to configure where
`nil` is allowed in a Java method call. `method-type` prints the
current type of a method.
in: (.getParent f)

Type Error (core-typed-samples:6) Type mismatch:

Expected:       java.lang.String

Actual:         (U String nil)
in: (.getParent f)

Type Error (core-typed-samples:6:1) Type mismatch:

Expected:       (Fn [java.io.File -> java.lang.String])

Actual:         (Fn [java.io.File -> (U String nil)])
in: (def file-parent (fn* ([f] (.getParent f))))

ExceptionInfo Type Checker: Found 3 errors clojure.core/ex-info ...

core.typed assumes all methods return nullable types, so it is a type error to annotate parent as [File -> String]. Each preceding type error reiterates that the annotation tried to claim a (U nil String) was a String, with the most specific (and useful) error being the first.

core.typed is designed to be pessimistic about Java code, while being accurate enough to avoid adding arbitrary code to “please” the type checker. For example, core.typed distrusts Java methods enough to assume all method parameters are non-nullable and the return type is nullable by default. On the other hand, core.typed knows Java constructors never return null.

If core.typed is too pessimistic for you with its nullable return types, you can override particular methods with clojure.core.typed/non-nil-return. Adding the following to the preceding code would result in a successful type check (check omitted for brevity):

(t/non-nil-return java.io.File/getName :all)

Sometimes the type checker might seem overly picky; in the solution, two type-hinted constructors were necessary. It might seem normal in a dynamically typed language to simply call (File. s) and allow reflection to resolve any ambiguity. By conforming to what core.typed expects, however, all ambiguity is eliminated from the constructors, and the type hints inserted enable the Clojure compiler to generate efficient bytecode.

It is valid to wonder why both type hints and core.typed annotations are needed to type-check ambiguous Java calls. A type hint is a directive to the compiler, while type annotations are merely for core.typed to consume during type checking. core.typed does not have influence over resolving reflection calls at compile time, so it chooses to assume all reflective calls are ambiguous instead of trying to guess what the reflection might resolve to at runtime. This simple rule usually results in faster, more explicit code, often desirable in larger code bases.

Clojure strongly encourages higher-order functions, but tools for verifying their use focus on runtime verification. You want earlier feedback, preferably at compile time.

Use core.typed to type-check higher-order functions.

To follow along with this recipe, create a file core_typed_samples.clj and start a REPL using lein-try:

$ touch core_typed_samples.clj
$ lein try org.clojure/core.typed

To demonstrate core.typed’s abilities, define a typed higher-order function hash-of?, which accepts two predicates and returns a new predicate.

Use clojure.core.typed/fn> to return an anonymous function with type annotations attached:

(ns core-typed-samples
  (:require [clojure.core.typed :refer [ann fn>] :as t]))

(ann hash-of? [[Any -> Any] [Any -> Any] -> [Any -> Any]])
(defn hash-of? [ks? vs?]
  (fn> [m :- Any]
    (when (map? m)
      (and (every? ks? (keys m))
           (every? ks? (vals m))))))

Each argument to hash-of? has type [Any -> Any]: a single argument function taking anything and returning anything.

Verifying hash-of? confirms that the preceding type annotations are correct:

user=> (require '[clojure.core.typed :as t])
user=> (t/check-ns 'core-typed-samples)
# ...
:ok

Using the clojure.core.typed/cf macro, you can type-check individual forms at the REPL (or under test). Invoking hash-of? with two predicates verifies as expected, outputting the resulting type:

user=> (require '[core-typed-samples :refer [hash-of?]])
user=> (t/cf (hash-of? number? number?))
(Fn [Any -> Any])

Passing + as a predicate, however, is a type error:

user=> (t/cf (hash-of? + number?))
Type Error (user:1:7) Type mismatch:

Expected:       (Fn [Any -> Any])

Actual:         (Fn [t/AnyInteger * -> t/AnyInteger]
                    [java.lang.Number * -> java.lang.Number])

ExceptionInfo Type Checker: Found 1 error  clojure.core/ex-info (core.clj:4327)

This is because hash-of? takes a function with an Any parameter and + takes at most a Number.

While Clojure’s built-in pre/post conditions are useful for defining anonymous functions that fail fast, these checks only provide feedback at runtime. Why not type-check our higher-order functions as well? core.typed’s type-checking abilities aren’t limited to only data types—it can also type-check functions as types themselves.

By writing returning anonymous functions created with the clojure.core.typed/fn> form instead of fn, it is possible to annotate function objects with core.typed’s rich type-checking system. When defining functions with fn>, annotate types to its arguments with the :- operator. For example, (t/fn> [m :- Map] ...) would indicate an anonymous function that accepted a Map as its sole argument.

Beyond definition, it can also be useful to check the types of forms at the REPL. The clojure.core.typed/cf macro is a versatile REPL-oriented tool for on-demand type checking. It proves useful not only for checking your code, but also for investigating built-in functions. Invoking cf on any of Clojure’s higher-order functions reveals their type signatures:

user=> (t/cf iterate)
(All [x]
  (Fn [(Fn [x -> x]) x -> (clojure.lang.LazySeq x)]))

The All around iterate’s type indicates that it is polymorphic in x. It reads, “for all types x, takes a function that accepts an x and returns an x, and takes an x, and returns a lazy sequence of x.”

The cf macro can also detect when the wrong number of arguments are being passed to a function returned by another function:

user=> (t/cf (fn [] ((hash-of? + number?))))
Type Error (user:1:15) Type mismatch:

Expected:       (Fn [Any -> Any])

Actual:         (Fn [t/AnyInteger * -> t/AnyInteger]
                    [java.lang.Number * -> java.lang.Number])
in: ((core-typed-samples/hash-of? clojure.core/+ clojure.core/number?))

Type Error (user:1:14) Wrong number of arguments, expected 1 fixed
parameters, and got 0 for function [Any -> Any] and arguments []
in: ((core-typed-samples/hash-of? clojure.core/+ clojure.core/number?))

ExceptionInfo Type Checker: Found 2 errors  clojure.core/ex-info (core.clj:4327)