Chapter 7. Web Applications

7.0. Introduction

Web application development is the breadwinner for many languages nowadays; Clojure is no exception. In the annual 2013 State of Clojure Survey, web development ranked first for the question, “In which domains are you applying Clojure and/or ClojureScript?”

Much of the Clojure web development community today centers around Ring, an HTTP server library very much akin to Ruby’s Rack. Starting with the introduction to Ring in Recipe 7.1, “Introduction to Ring”, you’ll find a full complement of recipes here that will get you up to speed in no time.

Following Ring, the chapter takes a tour of the other Clojure web development ecosystems available at the time of writing, covering a few templating and HTML-manipulation libraries and a number of alternative web frameworks.

7.1. Introduction to Ring

Problem

You need to write an HTTP service with Clojure.

Solution

Clojure has no built-in HTTP server, but the de facto standard for serving basic, synchronous HTTP requests is the Ring library.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
    [ring.adapter.jetty :as jetty]
    clojure.pprint))

;; Echo (with pretty-print) the request received
(defn handler [request]
  {:status 200
   :headers {"content-type" "text/clojure"}
   :body (with-out-str (clojure.pprint/pprint request))})

(defn -main []
  ;; Run the server on port 3000
  (jetty/run-jetty handler {:port 3000}))

Discussion

Ring is the basis for most web applications in Clojure. It provides a low-level and straightforward request/response API, where requests and responses are plain old Clojure maps.

Ring applications are architected around handlers: functions that accept requests and return responses. The preceding example defines a single handler that just echoes the response it receives.

A basic response map consists of three keys: :status, the status code of the response; :headers, an optional string-string map of the response headers you want; and :body, the string you want as your response body. Here, :status is 200 and :body is a pretty-printed string of the request. Therefore, the following sample response from hitting the URL http://localhost:3000/test/path/?qs=1 on the author’s machine demonstrates the structure of a request:

{:ssl-client-cert nil,
 :remote-addr "0:0:0:0:0:0:0:1",
 :scheme :http,
 :request-method :get,
 :query-string "qs=1",
 :content-type nil,
 :uri "/test/path/",
 :server-name "localhost",
 :headers
 {"accept-encoding" "gzip,deflate,sdch",
  "connection" "keep-alive",
  "user-agent"
  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36
  (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36",
  "accept-language" "en-US,en;q=0.8",
  "accept"
  "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
  "host" "localhost:3000",
  "cookie" ""},
 :content-length nil,
 :server-port 3000,
 :character-encoding nil,
 :body #<HttpInput org.eclipse.jetty.server.HttpInput@43efe432>}

You can see that this is comprehensive, but low level, with the salient features of the request parsed into Clojure data structures without additional abstraction. Typically, additional code or libraries are used to extract meaningful information from this data structure.

The jetty adapter is used to run an embedded Jetty server. Ring also comes with adapters to run as a servlet in any Java servlet container.

Note that the call to run-jetty is synchronous and will not return as long as the server is running. If you call it from the REPL, you should wrap it in a future (or use some other concurrency mechanism) so the server runs on another thread and your REPL does not become unresponsive.

See Also

7.2. Using Ring Middleware

Problem

You’d like to build a transformation to be automatically applied to Ring requests or responses. For example, Ring gives you query strings, but you’d really rather work with a parsed map.

Solution

Since Ring works with regular Clojure data and functions, you can easily define middlewares as functions returning functions. In this case, define a middleware that modifies the request by adding a parsed version of the query string to the request before passing it on to the handler.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
    [ring.adapter.jetty :as jetty]
    [clojure.string :as str]
    clojure.pprint))

(defn parse-query-string
  "Parse a query string to a hash-map"
  [qs]
  (if (> (count qs) 0) ; Don't operate on nils or empty strings
    (apply hash-map (str/split qs #"[&=]"))))

(defn wrap-query
  "Add a :query parameter to incoming requests that contains a parsed
  version of the query string as a hash-map"
  [handler]
  (fn [req]
    (let [parsed-qs (parse-query-string (:query-string req))
          new-req (assoc req :query parsed-qs)]
      (handler new-req))))

(defn handler [req]
  (let [name (get (:query req) "name")]
    {:status 200
     :body (str "Hello, " (or name "World"))}))

(defn -main []
  ;; Run the server on port 3000
  (jetty/run-jetty (wrap-query handler) {:port 3000}))

Discussion

Since Ring handlers operate on regular Clojure maps, it’s very easy to write a middleware that wraps a handler. Here, we write a middleware that modifies the request before passing it to the handler. If the original request looked like this:

{:query-string "x=1&y=2"
 ; ... and the rest
 }

then the request received by the handler becomes:

{:query-string "x=1&y=2"
 :query {"x" "1" "y" "2"}
 ; ... and the rest
 }

However, you don’t need to write your own middleware for this. Ring provides a number of middlewares to add to your apps, including one called wrap-params that does the same as the middleware we just wrote, but better. From the Ring documentation:

wrap-params

Middleware to parse urlencoded parameters from the query string and form body (if the request is a urlencoded form). Adds the following keys to the request map:

:query-params - a map of parameters from the query string

:form-params - a map of parameters from the body

:params - a merged map of all types of parameter

Takes an optional configuration map. Recognized keys are:

:encoding - encoding to use for url-decoding. If not specified, uses the request character encoding, or “UTF-8” if no request character encoding is set.

You are in no way limited to using one middleware. Usually, you’ll at least want to use the cookie, session, and parameter middleware. One concise way to wrap your handler with several middlewares is to use the -> macro:

(require '[ring.middleware.session :refer [wrap-session]])
(require '[ring.middleware.cookies :refer [wrap-cookies]])
(require '[ring.middleware.params :refer [wrap-params]])
(def wrapped-handler
  (-> handler
    wrap-cookies
    wrap-params
    wrap-session))

7.3. Serving Static Files with Ring

Problem

You want to serve static files through your Ring application.

Solution

Use ring.middleware.file/wrap-file:

(require '[ring.middleware.file :refer [wrap-file]])

;; Serve all files from your public directory
(def app
  (wrap-file handler "/var/webapps/public"))

Discussion

wrap-file wraps another web request handler so that you will serve a static file if one exists in the specified directory, and call the handler if the requested file does not exist.

wrap-file is only one way to serve static files. If you only want to serve up a particular file, ring.util.response/file-response will return a handler that serves up that file:

(require '[ring.util.response :refer [file-response]])

;; Serve README.html
(file-response "README.html")

;; Serve README.html from the public/ directory
(file-response "README.html" {:root "public"})

;; Serve README.html through a symlink
(file-response "README.html" {:allow-symlinks? true})

Often, you will want to bundle your static files with your application. In that case, it makes more sense to serve files from your classpath rather than from a specific directory. For this, use ring.middleware.resource/wrap-resource:

(require '[ring.middleware.resource :refer [wrap-resource]])

(def app
  (wrap-resource handler "static"))

This will serve all files under a directory called static in your classpath. You can put the static directory under resources/ in a Leiningen project and have your static files packaged with JAR files you generate from that project.

You may want to wrap any file responses with ring.middleware.file-info/wrap-file-info. This Ring middleware checks the modification date and the type of the file, setting the Content-Type and Last-Modified headers. wrap-file-info needs to wrap around wrap-file or wrap-resource.

7.4. Handling Form Data with Ring

Problem

You want your app to accept user input using an HTML form.

Solution

Use ring.middleware.params/wrap-params to add incoming HTTP form parameters to incoming request maps.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
    [ring.adapter.jetty :as jetty]
    [ring.middleware.params :refer [wrap-params]]))

(def greeting-form
  (str
    "<html>"
    "  <form action='' method='post'>"
    "    Enter your name: <input type='text' name='name'><br/>"
    "    <input type='submit' value='Say Hello'>"
    "  </form>"
    "</html>"))

(defn show-form []
  {:body greeting-form
   :status 200 })

(defn show-name
  "A response showing that we know the user's name"
  [name]
  {:body (str "Hello, " name)
   :status 200})

(defn handler
  "Show a form requesting the user's name, or greet them if they
  submitted the form"
  [req]
  (let [name (get-in req [:params "name"])]
    (if name
      (show-name name)
      (show-form))))

(defn -main []
  ;; Run the server on port 3000
  (jetty/run-jetty (wrap-params handler) {:port 3000}))

Discussion

wrap-params is a Ring middleware that handles the retrieval of query string and form parameters from raw requests. It adds three keys to the request:

:query-params
Contains a map of the parsed query string parameters
:form-params
Contains a map of form body parameters
:params
Contains the contents of both :query-params and :form-params merged together

In the preceding example we used :form-params, so our handler will only respond with a greeting on POST requests containing form-encoded parameters. If we had used :params, we would have had the option of also passing a URL query string with a "name" parameter. :params works with any kind of parameter (form- or URL-encoded). :form-params only works for form parameters.

Note that the form keys are passed in as strings, not keywords.

7.5. Handling Cookies with Ring

Problem

Your web application needs to read or set cookies on the user’s browser (for example, to remember a user’s name).

Solution

Use the ring.middleware.cookies/wrap-cookies middleware to add cookies to your requests.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
    [ring.adapter.jetty :as jetty]
    [ring.middleware.cookies :refer [wrap-cookies]]
    [ring.middleware.params :refer [wrap-params]]))

(defn set-name-form
  "A response showing a form for the user to enter their name."
  []
  {:body "<html>
            <form action=''>
              Name: <input type='text' name='name'>
              <input type='submit'>
            </form>
          </html>"
   :status 200
   :content-type "text/html"})

(defn show-name
  "A response showing that we know the user's name"
  [name]
  {:body (str "Hello, " name)
   :cookies {"name" {:value name}} ; Preserve the cookies
   :status 200 })

(defn handler
  "If we know the user's name, show it; else, show a form to get it."
  [req]
  (let [name (or
              (get-in req [:cookies "name" :value])
              (get-in req [:params "name"]))]
    (if name
      (show-name name)
      (set-name-form))))

(def wrapped-handler
  (-> handler
      wrap-cookies
      wrap-params))

(defn -main []
  ;; Run the server on port 3000
  (jetty/run-jetty wrapped-handler {:port 3000}))

Discussion

This example uses the wrap-cookies and wrap-params middlewares included with ring-core. The first time users visit a page, it shows them a form to enter their names. Once entered, it stores the user’s name in a cookie and displays it instead, until the cookie is removed.

The example uses wrap-cookies to retrieve the user’s stored name from the cookie map, or, if it’s not there, wrap-params to retrieve the user’s name from the request parameters.

Ring’s cookie middleware simply adds an extra parameter, :cookies, onto the incoming request map, and sets any cookies you pass out as the :cookies parameter on the response. The :cookies parameter is a map that looks something like this:

{"name" {:value "Some Guy"}}

You can add other optional parameters to each cookie, along with :value. From the Ring cookie documentation:

As well as setting the value of the cookie, you can also set additional attributes:

  • :domain—restrict the cookie to a specific domain
  • :path—restrict the cookie to a specific path
  • :secure—restrict the cookie to HTTPS URLs if true
  • :http-only—restrict the cookie to HTTP if true (not accessible via e.g. JavaScript)
  • :max-age—the number of seconds until the cookie expires
  • :expires—a specific date and time the cookie expires

7.6. Storing Sessions with Ring

Problem

You need to store secure data about a logged-in user as state on the server.

Solution

Use ring.middleware.session/wrap-session to add sessions to your Ring application.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
    [ring.adapter.jetty :as jetty]
    [ring.middleware.session :refer [wrap-session]]
    [ring.middleware.params :refer [wrap-params]]))

(def login-form
  (str
    "<html>"
    "  <form action='' method='post'>"
    "    Username: <input type='text' name='username'><br/>"
    "    Password: <input type='text' name='password'><br/>"
    "    <input type='submit' value='Log In'>"
    "  </form>"
    "</html>"))

(defn show-form []
  {:body login-form
   :status 200 })

(defn show-name
  "A response showing that we know the user's name"
  [name session]
  {:body (str "Hello, " name)
   :status 200
   :session session })

(defn do-login
  "Check the submitted form data and update the session if necessary"
  [params session]
  (if (and (= (params "username") "jim")
           (= (params "password") "password"))
    (assoc session :user "jim")
    session))


(defn handler
  "Log a user in, or not"
  [{session :session params :form-params :as req}]
  (let [session (do-login params session)
        username (:user session)]

    (if username
      (show-name username session)
      (show-form))))

(def wrapped-handler
  (-> handler
      wrap-session
      wrap-params))

(defn -main []
  ;; Run the server on port 3000
  (jetty/run-jetty wrapped-handler {:port 3000}))

Discussion

Ring’s session middleware has an API similar to the cookies API. You get session data from the :session request map, and set it by including a :session key in the response map. Whatever you pass to :session is up to you, but usually you’ll want to use a map to store keys and values.

Behind the scenes, Ring sets a cookie called ring-session, which contains a unique ID identifying the session. When a request comes in, the session middleware gets the session ID from the request, then reads the value of the session from some session store.

Which session store the middleware uses is configurable. The default is to use an in-memory session store, which is useful for development but has the side effect of losing sessions whenever you restart the app. Ring includes an encrypted cookie store as well, which is persistent, and you can get third-party libraries for many popular storages, including Memcached and Redis. You can write your own, too, to store your sessions in any database.

You can set your store by passing an options map with a :store parameter to wrap-session:

(wrap-session handler {:store (my-store)}))

To set the value of :session, just pass it along with your response. If you don’t need the session changed, leave :session out of your response. If you want to actually clear the session, pass nil as the value of the :session key.

7.7. Reading and Writing Request and Response Headers in Ring

Problem

You need to read or write HTTP request or response headers.

Solution

Read from the :headers key in a Ring request map, or assoc values onto the response map before returning from a Ring handler function.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
    [ring.adapter.jetty :as jetty]))

(defn user-agent-as-json
  "A handler that returns the User-Agent header as a JSON
   response with an appropriate Content-Type"
  [req]
  {:body (str "{"user-agent": "" (get-in req [:headers "user-agent"]) ""}")
   :headers {"Content-Type" "application/json"}
   :status 200})

(defn -main []
  ;; Run the server on port 3000
  (jetty/run-jetty user-agent-as-json {:port 3000}))

Discussion

This example defines a Ring handler that returns the incoming User-Agent header as a JSON response. It gets the User-Agent from the request header map, and uses a Content-Type header in the response to indicate to the client that it should be parsed as JSON.

Ring passes request headers as a :headers parameter in the request map, and accepts a :headers parameter in response maps as well. The keys and values of the headers map should both be strings. Clojure keywords are not supported.

You can use Ring to set any header that is valid in HTTP.

According to RFC-2616, header names are not case sensitive. To make it easier to consistently get values from the request map, no matter what their case, Ring passes in all header values as lowercase, regardless of what the client sent. You may wish to send headers using the actual capitalization used in the specification, though, just in case the client you’re communicating with is not compliant (following the classic robustness principle: “Be conservative in what you send; be liberal in what you accept”).

See Also

7.8. Routing Requests with Compojure

Problem

You want an easy way to route URLs to specific Ring handler functions.

Solution

Use the Compojure library to add routing to your app.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
    [compojure.core :refer [defroutes GET]]
    [ring.adapter.jetty :as jetty]))

;; View functions
(defn view [x]
  (str "<h1>" x "</h1>"))

(defn index []
  (view "Hello"))

(defn index-fr []
  (view "Bonjour"))

;; Routing
(defroutes main-routes
  (GET "/" [] (index))
  (GET "/en/" [] (index))
  (GET "/fr/" [] (index-fr))
  (GET "/:greeting/" [greeting] (view greeting)))

;; Server
(defn -main []
 (jetty/run-jetty main-routes {:port 3000}))

Discussion

Compojure is a routing library that lets you define routes for your app. It does this via the defroutes macro, which produces a Ring handler function.

Here, we define four routes:

/
Displays “Hello”
/en/
Also displays “Hello”
/fr/
Displays “Bonjour”
/:greeting/
Echoes the greeting passed by the user

The last view is an example of Compojure’s URL parameter syntax. The section of the URL identified by :greeting is passed along to the view, which displays it for the user. So, visiting http://localhost:3000/Buenos%20Dias/ displays “Buenos Dias” in response.

One caveat to be aware of is that Compojure routes are sensitive to a trailing slash: a route defined as /users/:user/blog/ will not match the URL http://mysite.com/users/fred/blog, but it will match http://mysite.com/users/fred/blog/.

The [] in each route is actually syntactic sugar for intercepting these parameters. You can also use req or any other symbol to get the whole request:

(defroutes main-routes-2
  (GET "/" req (some-view req)))

You can even use Clojure’s destructuring syntax[23] to extract parts of the request. For example, if you’re using the wrap-params middleware, you can grab the parameters and pass them to a function:

(defroutes main-routes-2
  (GET "/" {params :params} (some-other-view params)))

It is important to realize that Compojure works on top of Ring. Your views should still return Ring response maps (although Compojure will wrap strings you return with a basic 200 response).

You can also define routes for other types of HTTP requests by specifying the route using the relevant Compojure directive: compojure.core/POST and compojure.core/PUT are most commonly used, in addition to compojure.core/GET.

Compojure provides a few other helpful tools, such as compojure.route, which provides helpers for serving resources, files, and 404 responses; and compojure.handler, which bundles a number of Ring middlewares into one convenient wrapper.

See Also

7.9. Performing HTTP Redirects with Ring

Problem

In a Ring application, you need to return an HTTP response code that will redirect the browser to another URL.

Solution

To redirect a Ring request, use the redirect function in the ring.util.response namespace.

To follow along with this recipe, clone the https://github.com/clojure-cookbook/ringtest repository and overwrite src/ringtest.clj:

(ns ringtest
  (:require
   [ring.adapter.jetty :as jetty]
   [ring.util.response :as response]))

(defn redirect-to-github
  "A handler that redirects all requests"
  [req]
  (response/redirect "http://github.com/"))

(defn -main []
  ;; Run the server on port 3000
  (jetty/run-jetty redirect-to-github {:port 3000}))

Discussion

The ring.util.response namespace contains a function for redirecting to a URL. This URL can be generated dynamically from the request map (using parameters from wrap-params, headers, etc.). Underneath, this function simply creates a response map with a 302 :status value and a location header containing the URL to redirect to.

According to the HTTP specification, if the response method is a POST, PUT, or DELETE, it should be assumed that the server received the request, and the client should issue a GET request to the URL in the location header. This is an important caveat when writing REST services. Fortunately, the specification provides a 307 status code, which should signal to clients that the request should be redirected to the new location using the original method and body. To do this, simply return a response map in the handler function like so:

(defn redirect-to-github
  [req]
  {:status 307
   :headers {"Location" "http://github.com"}
   :body ""})

See Also

7.10. Building a RESTful Application with Liberator

Problem

You want to build a RESTful (RFC 2616–compliant) web application on top of Ring and Compojure at a higher level of abstraction, by defining resources.

Solution

Use Liberator to create HTTP-compliant, RESTful web apps.

To follow along with this recipe, create a new project using the command lein new liberator-test.

Inside your project.clj, add the following dependencies to your :dependencies key:

[compojure "1.0.2"]
[ring/ring-jetty-adapter "1.1.0"]
[liberator "0.9.0"]

Then, modify src/liberator_test/core.clj to match the following contents:

(ns liberator-test.core
  (:require [compojure.core :refer [defroutes ANY]]
            [ring.adapter.jetty :as jetty]
            [liberator.core :refer [defresource]]))

;; Resources

(defresource root
   :available-media-types #{"text/plain"})

;; Routing
(defroutes main-routes
  (ANY "/" [] root))

;; Server
(defn -main []
 (jetty/run-jetty main-routes {:port 3000}))

Discussion

Liberator is a library for developing HTTP-compliant web servers. It handles content negotiation, status codes, and standard request methods on RESTful resources. It decides what status to respond with using a decision tree, which follows the HTTP spec.

Liberator does not handle routing, so another library needs to be used. In this recipe, Compojure was used. Since Liberator does a better job of handling the request method (GET, PUT, POST, etc.), you should use ANY in your Compojure routes. You could also use a different routing library, such as Clout, Moustache, or the playnice router.

The defresource form defines a web resource, which is modeled as a Ring handler. You can therefore pass the resource as the last argument to Compojure routes.

Liberator resources are set up with sensible defaults. The default for the available media types is the empty set, so it needs to be set to something; otherwise, Liberator will return a 406 “Not Acceptable” response. In this recipe it is set to respond with text/plain as the MIME type. The default response is “OK,” which you will see if you run the recipe and point a browser at http://localhost:3000.

See Also

7.11. Templating HTML with Enlive

Problem

You want to create HTML dynamically based on a template, without using traditional mixed code or DSL-style templating.

Solution

Use Enlive, a Clojure library that takes a selector-based approach to templating HTML. Unlike other template frameworks like PHP, ERB, and JSP, it doesn’t mix code and text. And unlike systems like Haml or Hiccup, it doesn’t use specialized DSLs. Instead, templates are plain old HTML files, and Enlive uses Clojure code to target specific areas for replacement or duplication based on incoming data.

To follow along with this recipe, start a REPL using lein-try:

$ lein try enlive

To begin, create a file post.html to serve as an Enlive template:

<html>
  <head><title>Page Title</title></head>
  <body>
    <h1>Page Title</h1>
    <h3>By <span class="author">Mickey Mouse</span></h3>
    <div class="post-body">
      Lorem ipsum etc...
    </div>
  </body>
</html>

Note

Place this file in the resources/ directory, if you’re using Enlive in the context of a project.

The following Clojure code defines an Enlive template based on the contents of post.html:

(require '[net.cgrand.enlive-html :as html])

;; Define the template
(html/deftemplate post-page "post.html"
  [post]
  [:title] (html/content (:title post))
  [:h1] (html/content (:title post))
  [:span.author] (html/content (:author post))
  [:div.post-body] (html/content (:body post)))

;; Some sample data
(def sample-post {:author "Luke VanderHart"
                  :title "Why Clojure Rocks"
                  :body "Functional programming!"})

To apply the template to the data, invoke the function defined by deftemplate. Since it returns a sequence of strings, in most applications you’ll probably want to concatenate the results into a single string:

(reduce str (post-page sample-post))

Here’s the formatted output: 

<html>
  <head><title>Why Clojure Rocks</title></head>
  <body>
    <h1>Why Clojure Rocks</h1>
    <h3>By <span class="author">Luke VanderHart</span></h3>
    </h3><div class="post-body">Functional programming!</div>
  </body>
</html>

See the following discussion section for a detailed explanation of the deftemplate macro and what is actually happening in this code.

Repeating elements

The preceding code simply replaces the values of certain nodes in the emitted HTML. In real scenarios, another common task is to repeat certain items from input HTML, one repetition for each item in the input data. For this task, Enlive provides snippets, which are selections from an input HTML that can then be repeated as many times as desired in the output of another template:

(def sample-post-list
  [{:author "Luke VanderHart"
    :title "Why Clojure Rocks"
    :body "Functional programming!"}
   {:author "Ryan Neufeld"
    :title "Clojure Community Management"
    :body "Programmers are like..."}
   {:author "Rich Hickey"
    :title "Programming"
    :body "You're doing it completely wrong."}])

(html/defsnippet post-snippet "post.html"
  {[:h1] [[:div.post-body (html/nth-of-type 1)]]}
  [post]
  [:h1] (html/content (:title post))
  [:span.author] (html/content (:author post))
  [:div.post-body] (html/content (:body post)))

(html/deftemplate all-posts-page "post.html"
  [post-list]
  [:title] (html/content "All Posts")
  [:body] (html/content (map post-snippet post-list)))

Invoking the defined all-posts-page function now returns an HTML page populated with all three sample posts:

(reduce str (all-posts-page sample-post-list))

Here’s the formatted output: 

<html>
  <head><title>All Posts</title></head>
  <body>
    <h1>Why Clojure Rocks</h1>
    <h3>By <span class="author">Luke VanderHart</span></h3>
    <div class="post-body">Functional programming!</div>
    <h1>Clojure Community Management</h1>
    <h3>By <span class="author">Ryan Neufeld</span></h3>
    <div class="post-body">Programmers are like...</div>
    <h1>Programming</h1>
    <h3>By <span class="author">Rich Hickey</span></h3>
    <div class="post-body">You're doing it completely wrong.</div>
  </body>
</html>

In this example, the defsnippet macro defines a snippet over a range of elements in the input HTML, from the <h1> element to the <div class="post-body">.

Then, the deftemplate for all-posts-page uses the result of mapping post-snippet over the content of the body element. Since there are three posts in the sample input data, the snippet is evaluated three times, and there are three posts output in the resulting HTML.

Discussion

Enlive can be slightly difficult to get the hang of, compared to some other libraries. There are several contributing factors to this:

  • It has a more novel conceptual approach than other templating systems (although it bears a lot of similarity to some other non-Clojure templating techniques, such as XSLT).
  • It utilizes functional programming techniques to the fullest, including liberal use of higher-order functions.
  • It’s a large library, capable of many things. The subset of features required to accomplish a particular task is not always evident.

In general, the best way to get past these issues and experience the power and flexibility that Enlive can provide is to understand all the different parts individually, and what they do. Then, composing them into useful templating systems becomes more manageable.

Enlive and the DOM

First of all, it is important to understand that Enlive does not operate on HTML text directly. Instead, it first parses the HTML into a Clojure data structure representing the DOM (Document Object Model). For example, the HTML fragment:

<div id="foo">
  <span class="bar">Hello!</span>
</div>

would be parsed into the Clojure data:

{:tag :html,
  :attrs nil,
  :content
  ({:tag :body,
    :attrs nil,
    :content
    ({:tag :div,
      :attrs {:id "foo"},
      :content
      ({:tag :span, :attrs {:class "bar"}, :content ("Hello!")})})})}

This is more verbose, but it is easier to manipulate from Clojure. You won’t necessarily have to deal with these data structures directly, but be aware that anywhere Enlive says it operates on an element or a node, it means the Clojure data structure for the element, not the HTML string.

Templates

The most important element of these examples is the deftemplate macro. deftemplate takes a symbol as a name, a classpath-relative path to an HTML file, an argument list, and a series of selector and transform function pairs. It emits a function, bound to the same name and of the specified arguments, which, when called, will return the resulting HTML as a sequence of strings.

An Enlive selector is a Clojure data structure that identifies a specific node in the input HTML file. They are similar to CSS selectors in operation, although somewhat more capable. In the example in the solution, [:title] selects each <title> element, [:span.author] each <span> with class="author", etc. More selector forms are described in the following subsection.

A template transform function takes an Enlive node and returns a modified node. Our example uses Enlive’s content utility function, which returns a function that swaps the contents of a node with the value given as its argument.

The return value is not itself a string, but a sequence of strings, each one a small fragment of HTML code. This allows the underlying data structure to be transformed to a string representation lazily. For simplicity, our example just reduces the string concatenation function str across the results, but this is actually not optimally performant. To build a string most efficiently, use the Java StringBuilder class, which uses mutable state to build up a String object with the best possible performance. Alternatively, forego the use of strings altogether and pipe the result seq of the template function directly into an output Writer, which most web application libraries (including Ring) can use as the body of an HTTP response (the most common destination for templated HTML).

Selectors

Enlive selectors are data structures that identify one or more HTML nodes. They describe a pattern of data—if the pattern matches any nodes in the HTML data structure, the selector will select those nodes. A selector may select one, many, or zero nodes from a given HTML document, depending on how many matches the pattern has.

The full reference for valid selector forms is quite complex, and beyond the scope of this recipe. See the formal selector specification for complete documentation.

The following selector patterns should be sufficient to get you started:

[:div]
Selects all <div> element nodes.
[:div.sidebar]
Selects all <div> element nodes with a CSS class of "sidebar".
[:div#summary]
Selects the <div> element with an HTML ID of "summary".
[:p :span]
Selects all <span> elements that are descendants of <p> elements.
[:div.menu :ul :li :span]
Selects only <span> elements inside an <li> element inside a <ul> element inside a <div> element with a CSS style of "menu".
[[:div (nth-child 2)]]
Selects all <div> elements that are the second children of their parent elements. The double square brackets are not a typo—the inner vector is used to denote a logical and condition. In this case, the matched element must be a <div>, and the nth-child predicate must hold true.

Other predicates besides nth-child are available, as well as the ability to define custom predicates. See the Enlive documentation for more details.

Finally, there is a special type of selector called a range selector that is not specified by a vector, but rather by a map literal (in curly braces). The range selector contains two other selectors and inclusively matches all the nodes between the two matched nodes, in document order. The starting node is in key position in the map literal and the ending node is in value position, so the selector {[:.foo] [:.bar]} will match all nodes between nodes with an ID of “foo” and an ID of “bar”.

The example in the solution uses a range selector in the defsnippet form to select all the nodes that are part of the same logical blog post, even though they aren’t wrapped in a common parent element.

Snippets

A snippet is similar to a template, in that it produces a function based on a base HTML file. However, snippets have two major differences from templates:

  1. Rather than always rendering the entire HTML file like a template does, snippets render only a portion of the input HTML. The portion to be rendered is specified by an Enlive selector passed as the third argument to the defsnippet macro, right after the name and the path to the HTML file.
  2. The return values of the emitted functions are Enlive data structures rather than HTML strings. This means that the results of rendering a snippet can be returned directly from the transform function of a template or another snippet. This is where Enlive starts to show its power; snippets can be recycled and reused extensively and in different combinations.

Other than these differences, the defsnippet form is identical to deftemplate, and after the selector, the rest of the arguments are the same—an argument vector and a series of selector and transform function pairs.

Using Enlive for scraping

Because of its emphasis on selectors and use of plain, unannotated HTML files, Enlive is extremely useful not just for templating and producing HTML, but also for parsing and scraping data from HTML from any source.

To use Enlive to extract data from HTML, you must first parse the HTML file into an Enlive data structure. To do this, invoke the net.cgrand.enlive-html/html-resource function on the HTML file. You may specify the file as a java.net.URL, a java.io.File, or a string indicating a classpath-relative path. The function will return the parsed Enlive data structure representing the HTML DOM.

Then, you can use the net.cgrand.enlive-html/select function to apply a selector to the DOM and extract specific data. Given a node and a selector, select will return only the matched nodes. You can then use the net.cgrand.enlive-html/text function to retrieve the text content of a node.

For example, the following function will return a sequence of the most recent n comic titles in the XKCD archives:

(defn comic-titles
  [n]
  (let [dom (html/html-resource
             (java.net.URL. "http://xkcd.com/archive"))
        title-nodes (html/select dom [:#middleContainer :a])
        titles (map html/text title-nodes)]
    (take n titles)))

(comic-titles 5)
;; -> ("Oort Cloud" "Git Commit" "New Study"
       "Telescope Names" "Job Interview")

When to use Enlive

As an HTML templating system, Enlive has two primary value propositions over its alternatives in the Clojure ecosystem.

First, the templates are pure HTML. This makes it much easier to work with HTML designers: they can hand their HTML mockups directly to a developer without having to deal with inline markup code, and developers can use them directly without manually slicing them (outside of code, that is). Furthermore, the templates themselves can be viewed in a browser statically, meaning they can serve as their own wireframes. This eliminates the burden of keeping a web project’s visual prototypes in sync with the code.

Secondly, because it uses real Clojure functions and data structures instead of a custom DSL, Enlive exposes the full power of the Clojure language. There are very few situations where you should feel limited by Enlive’s capabilities, since it is always possible to extend it using only standard Clojure functions and macros, operating on familiar persistent, immutable data structures.

See Also

7.12. Templating with Selmer

Problem

You want to create server-side page templates using syntax similar to that of Django and Jinja. You want to be able to insert dynamic content and use template inheritance to structure the templates.

Solution

Use the Selmer library to create your template and call it with a context map containing the dynamic content.

To follow along with this recipe, start a REPL using lein-try:

$ lein try selmer

A Selmer template is an HTML file with special tags that is populated with dynamic content at runtime. A simple template (base.html) might look like the following:

<!DOCTYPE html>
<html lang="en">
  <body>
    <header>

      <h1>{{header}}</h1>

      <ul id="navigation">

        {% for item in nav-items %}
        <li>
            <a href="{{item.link}}">{{item.name}}</a>
        </li>
        {% endfor %}

      </ul>
    </header>
  </body>
</html>

The template can then be rendered by calling the selmer.parser/render-file function:

(require '[selmer.parser :refer [render-file]])

(println
  (render-file "base.html"
               {:header "Hello Selmer"
                :nav-items [{:name "Home" :link "/"}
                            {:name "About" :link "/about"}]}))

When render-file runs, it will populate the tags with the content provided. The value will be returned as a string, suitable for use as the response body in a Ring application (for example). Here the result is simply printed to standard output, for easy inspection.

We can apply filters to variables for additional post-processing at runtime. Here, we use the upper filter to convert our heading to uppercase:

<h1>{{header|upper}}</h1>

We can extract parts of the template into individual snippets using the include tag. For example, if we wanted to define the header in a separate file header.html:

<header>

  <h1>{{header}}</h1>

  <ul id="navigation">

    {% for item in nav-items %}
    <li>
        <a href="{{item.link}}">{{item.name}}</a>
    </li>
    {% endfor %}

  </ul>
</header>

we could then include it as follows:

<!DOCTYPE html>
<html lang="en">
  <body>

    {% include "header.html" %}

  </body>
</html>

The include tag will simply be replaced by the contents of the file it points to when the template is compiled.

We can also extend our base template when we create individual pages. To do that, we first define a block in our base template. This will serve as an anchor for the child template to override:

<!DOCTYPE html>
<html lang="en">
  <body>

    {% include "header.html" %}

    {% block content %}
    {% endblock %}

  </body>
</html>

The child template will reference the parent using the extends tag and define its own content for the content block:

{% extends "base.html" %}

{% block content %}

<h1>This is the home page of the site</h1>
<p>some exciting content follows</p>

{% endblock %}

Discussion

Selmer provides a powerful and familiar templating tool with many tags and filters for easily accomplishing many common tasks. It separates the view logic from presentation by design.

Selmer is also performant because it compiles the templates and ensures that only the dynamic content needs to be evaluated when serving a request.

Selmer concepts

Selmer includes two types of elements, variables and tags.

Variables are used to render values from the context map on the page. The {{ and }} are used to indicate the start and end of a variable.

In many cases, you may wish to post-process the value of a variable. For example, you might want to convert it to uppercase, pluralize it, or parse it as a date. Variable filters (described in the following subsection) are used for this purpose.

Tags are used to add various functionality to the template, such as looping and conditions. We already saw examples of the for, include, and extends tags. The tags use {% and %} to define their content.

The default tag characters might conflict with client-side frameworks such as AngularJS. In this case, we can specify custom tags by passing a map containing any of the following keys to the parser:

:tag-open
:tag-close
:filter-open
:filter-close
:tag-second
:custom-tags
:custom-filters

If we wanted to use [ and ] as our opening and closing tags, we could call the render function as follows:

(render (str "[% for ele in foo %] "
             "{{I'm not a tag, but the next one is}} [{ele}] [%endfor%]")
        {:foo [1 2 3]}
        {:tag-open [
         :tag-close ]})

The render function works just like render-file, except that it accepts the template content as a string.

Defining filters

Selmer provides a rich set of filters that allow decorating of the dynamic content. Some of the filters include capitalize, pluralize, hash, length, and sort.

However, if you need a custom filter that’s not part of the library, you can trivially add one yourself. For example, if we wanted to parse Markdown using the markdown-clj library and display it on the page, we could write the following filter:[24]

(require '[markdown.core :refer [md-to-html-string]]
         '[selmer.filters/add-filter!])

(add-filter! :markdown md-to-html-string)

We can now use this filter in our templates to render our Markdown content:

<h2>Blog Posts</h2>
<ul>
  {% for post in posts %}
    <li>{{post.title|markdown|safe}}</li>
{% endfor %}
</ul>

Note that we had to chain the markdown filter with the safe filter. This is due to the fact that Selmer escapes variable content by default. We can change our filter definition to indicate that its content does not need escaping as follows:

(add-filter! :markdown (fn [s] [:safe (md-to-html-string s)]))

Defining tags

Again, we can define custom tags in addition to those already present in the library. This is done by calling the selmer.parser/add-tag! function.

Let’s say we wish to add a tag that will capitalize its contents:

(require '[selmer.parser :refer [add-tag!]])

(add-tag! :uppercase
          (fn [args context-map content]
            (.toUpperCase (get-in content [:uppercase :content])))
          :enduppercase)

(render "{% uppercase %}foo {{bar}} baz{% enduppercase %}" {:bar "injected"})

Inheritance

We already saw some examples of template inheritance. Each template can extend a single template and include any number of templates in its content.

The templates can extend templates that themselves extend other templates. In this case, the blocks found in the outermost child will override any other blocks with the same name.

See Also

7.13. Templating with Hiccup

Problem

You want to create HTML dynamically based on a template, written in pure Clojure data.

Solution

Use Hiccup, a library for representing and rendering HTML templates made up of regular Clojure data structures.

To follow along with this recipe, start a REPL using lein-try:

$ lein try hiccup

Hiccup represents HTML nodes as vectors. The first entry of the vector is the element’s name; the second is an optional map of the element’s attributes; and any remaining entries are the element’s body:

;; <h1 class="header">My Page Title</h1> in Hiccup...
[:h1 {:class "header"} "My Page Title"]

;; <ul>
;;   <li>lions</li>
;;   <li>tigers</li>
;;   <li>bears</li>
;; </ul> in Hiccup...
[:ul
  [:li "lions"]
  [:li "tigers"]
  [:li "bears"]] ;; oh my!

Render any Hiccup data structure to HTML using the hiccup.core/html function:

(require '[hiccup.core :refer [html]])
(html [:h1 {:class "header"} "My Page Title"])
;; -> "<h1 class="header">My Page Title</h1>"

Since nodes are represented as regular Clojure data, you can leverage any of Clojure’s built-in functions or techniques to yield Hiccup-compliant vectors:

(def pi 3.14)
(html [:p (str "Pi is approximately: " pi)])
;; -> "<p>Pi is approximately: 3.14</p>"

(html [:ul
        (for [animal ["lions" "tigers" "bears"]]
          [:li animal])])
;; -> "<ul><li>lions</li><li>tigers</li><li>bears</li></ul>"

Using all of the preceding techniques, it’s possible to create a simple function to dynamically populate the contents of a minimal blog page using only Clojure functions and data:

(defn blog-index
  "Render a blog's index as Hiccup data"
  [title author posts]
  [:html
    [:head
      [:title title]]
    [:body
      [:h1 title]
      [:h2 (str "By " author)]
      (for [post posts]
        [:article
          [:h3 (:title post)]
          [:p (:content post)]])]])

(-> (blog-index "My First Blog"
                "Ryan"
                 [{:title "First post!" :content "I'm here!"}
                  {:title "Second post." :content "Yawn, bored."}])

    html)

Formatted output: 

<html>
  <head>
    <title>My First Blog</title>
  </head>
  <body>
    <h1>My First Blog</h1>
    <h2>By Ryan</h2>
    <article>
      <h3>First post!</h3>
      <p>I'm here!</p>
    </article>
    <article>
      <h3>Second post.</h3>
      <p>Yawn, bored.</p>
    </article>
  </body>
</html>"

Discussion

Hiccup is an easy, “no muss, no fuss” way of templating and rendering HTML from raw functions and data. This comes in particularly handy when you don’t have the time to learn a new DSL or you prefer to work exclusively with Clojure.

An HTML node is represented in Hiccup as a vector of a few elements:

  • The node’s name, represented as a keyword (e.g., :h1, :article, or :body)
  • An optional map of the node’s attributes, with attribute names represented as keywords (e.g., {:href "/posts/"} or {:id "post-1" :class "post"})
  • Any number of other nodes or string values constituting the node’s body

Invoke hiccup.core/html with a single node, snippet, or entire page to render its contents as HTML. For content with special characters that should be escaped, wrap values in a hiccup.core/h invocation:

(require '[hiccup.core :refer [h]])
(html [:a {:href (h "/post/my<crazy>url")}])
;; -> "<a href="/post/my&amp;lt;crazy&amp;gt;url"></a>"

Hiccup also has basic support for rendering forms. Use form-to and a bevy of other helpers in the hiccup.form namespace to simplify rendering form tags:

(require '[hiccup.form :as f])

(f/form-to [:post "/posts/new"]
  (f/hidden-field :user-id 42)
  (f/text-field :title)
  (f/text-field :content))
;; -> [:form {:method "POST", :action #<URI /posts/new>}
;;      [:input {:type "hidden"
;;               :name "user-id"
;;               :id "user-id"
;;               :value 42}]
;;      [:input {:type "text"
;;               :name "title"
;;               :id "title"
;;               :value nil}]
;;      [:input {:type "text"
;;               :name "content"
;;               :id "content"
;;               :value nil}]]

See Also

7.14. Rendering Markdown Documents

Problem

You need to render a Markdown document.

Solution

Use the markdown-clj library to render Markdown documents.

To follow along with this recipe, start a REPL using lein-try:

$ lein try markdown-clj

Use markdown.core/md-to-html to read a Markdown document and generate a string containing HTML:

(require '[markdown.core :as md])

(md/md-to-html "input.md" "output.html")

(md/md-to-html (input-stream "input.md") (output-stream "test.txt"))

Use markdown.core/md-to-html-string to convert a string with Markdown content to its HTML representation:

(md/md-to-html-string
  "# This is a test

some code follows:
```
(defn foo [])
```")
<h1> This is a test</h1><p>some code follows:</p><pre>
&#40;defn foo &#91;&#93;&#41;
</pre>

Discussion

Markdown is a popular lightweight markup language that’s easy to read and write and can be converted to structurally valid HTML.

Since Markdown leaves many aspects of rendering the HTML open to interpretation, it’s not guaranteed that different parsers will produce the same representation. This can be a problem if you render a Markdown preview on the client using one parser and then later generate HTML on a server using a different parser. By virtue of compiling to both Clojure and ClojureScript, markdown-clj avoids this problem. With it, you can use the same parser on both the server and the client and be guaranteed that the documents will be rendered consistently.

Let’s take a look at more examples of using the library. The code blocks can be annotated with language hints. In this case, the pre tags will be decorated with a class compatible with the SyntaxHighlighter:

(md/md-to-html-string (str "# This is a test

some code follows:
"
                           "```clojure
(defn foo [])
```"))
<h1> This is a test</h1><p>some code follows:</p><pre class="brush: clojure">
&#40;defn foo &#91;&#93;&#41;
</pre>

markdown-clj supports all the standard Markdown tags, with the exception of reference-style links (because the parser uses a single pass to generate the document).

The markdown.core/md-to-html processes the input line by line, and the entirety of the content does not need to be stored in memory when processing. On the other hand, both the md-to-html-string and md-to-html functions load the entire contents into memory.

The parser accepts additional formatting options. These include :heading-anchors, :code-style, :custom-transformers, and :replacement-transformers.

When the :heading-anchors keyis set to true, an anchor will be generated for each heading tag:

(md/md-to-html-string "###foo bar BAz" :heading-anchors true)
<h3>
  <a name="heading" class="anchor" href="#foo&#95;bar&#95;baz></a>
  foo bar BAz
</h3>

The :code-style key allows overriding the default style hint for code blocks:

(md/md-to-html-string "```clojure
(defn foo [])
```"
                      :code-style #(str "class="" % """))
<pre class="clojure">
&#40;defn foo &#91;&#93;&#41;
</pre>

We can specify transformers for custom tags by using the :custom-transformers key. The transformer function should accept the text parameter, which is the current line, and the state parameter, which contains the current state of the parser. The state can be used to store information such as what tags are active:

(defn capitalize [text state]
  [(.toUpperCase text) state])

(md/md-to-html-string "#foo" :custom-transformers [capitalize])
<H1>FOO</H1>

Finally, we can provide a custom set of transformers to replace the built-in ones using the :replacement-transformers key:

(markdown/md-to-html-string "#foo" :replacement-transformers [capitalize])

See Also

7.15. Building Applications with Luminus

Problem

You want to quickly create a typical Ring/Compojure web application structure to get a fast start on a new web development project.

Solution

Use the Luminus Leiningen template when creating a new project.

At the command line, type:

$ lein new luminus myapp

This will create a new Ring/Compojure application with a skeletal namespace and resource directory structure that is ready to be packaged as standalone Java Archive (JAR) or Web Archive (WAR) file that can be deployed on an application server.

You can start the application in development mode by running:

$ lein ring server

Discussion

Luminus doesn’t do anything you couldn’t do yourself, but provides a standardized set of libraries and boilerplate for creating common Ring/Compojure applications.

The template generates a standard directory structure within your project, defines a main handler for your application, adds lein-ring hooks for it in the project.clj file, provides a default logging configuration, and sets up the default routes.

When creating the application, you can add functionality by specifying profiles that extend the generated code to include the relevant stubs. The following are some examples of initializing the application with default configurations for different databases:

$ lein new luminus app1 +h2

# Or, with PostgreSQL:
$ lein new luminus app2 +postgres

# Or ClojureScript!
$ lein new luminus app3 +cljs

# You can also specify multiple profiles simultaneously:
$ lein new luminus app4 +cljs +postgres

The resulting application is structured using the following namespaces.

The <app-name>.handler namespace contains init and destroy functions. These will be called when the application is starting up and shutting down, respectively. It also contains the app handler function that’s used by Ring to initialize the route handlers.

The <app-name>.routes namespace is used to house the core logic of the application. Here is where you would define the application routes and their handlers. The <app-name>.routes.home namespace contains the routes for the default / and /about pages.

The layout for the site is generated by the render function in the <app-name>.views.layout namespace. The HTML templates for the pages can be found under src/<app-name>/views/templates/. Luminus uses Selmer, introduced in Recipe 7.12, “Templating with Selmer”, as its default templating engine.

Any miscellaneous helpers will be found under the <app_name>.views.util namespace.

When a database profile is selected, the <app_name>.models.db and <app_name>.models.schema namespaces will be created. The schema namespace is reserved for table definitions, while the db namespace houses the functions dealing with the application model.

The application can be packaged as a standalone JAR file using lein ring uberjar or as a WAR file using lein ring uberwar.

See Also



[23] If you’re not familiar with Clojure’s destructuring syntax, we suggest reading Jay Fields’s “Clojure: Destructuring” blog post. For a more extensive resource, pick up a copy of Clojure Programming (O’Reilly) by Chas Emerick, Brian Carper, and Christophe Grand, which covers destructuring in depth.

[24] You’ll need to restart a new REPL with lein-try including markdown-clj to try this.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.118.208.97