Our pointy-haired marketing wizard (me) has decided that our hipstr application requires the ability to add a new artist and album (with a release date). The form should be validated, ensuring that all of the data is appropriate, before sending it off to the database. If, for whatever reason, the inserts into the database fail, then nothing should be changed. Also, if there are any validation errors with the form, we want to show an appropriate message to the user. Finally, I want to be able to use this form anywhere in the site with minimal modifications and maximum code reuse. (Say whaaaaaaat?!)
Let's brainstorm what this magical and revolutionary piece of functionality should look like. Based on what currently exists, the following idea should suffice:
This seems pretty doable, and it satisfies all of our requirements. First things first though: Let's create the HTML for the form.
We'll assume that the expected data structure against which this form will be bound is the following:
{:form {:error "" ; For reporting any error that's not a
; validation error
:validation-errors { :artist_name #{}
:album_name #{}
:release_date #{}
:new {:artist_name "" :album_name "" :release_date ""}For now, we just want to extend our existing recently-added.html template. That's the quickest entry point. Open the resources/templates/albums/recently-added.html template, and add the additional, bolded HTML shown as follows:
{% extends "templates/base.html" %}
{% block content %}
<h1>Recently Added</h1>
<div class="row">
<div class="col-sm-6 col-md-4">
<ol class="albums">
{% for a in albums %}
<li>
<div class="artist"><a href="/albums/{{a.artist}}">{{a.artist}}</a></div>
<div class="album-name">{{a.album_name}}</div>
<div class="release-date">{{a.release_date}}</div>
</li>
{% endfor %}
</ol>
</div>
<div class="col-sm-6 col-md-4">
{% if form.error %} <!-- #1 -->
<p class="bg-danger">{{form.error}}</p>
{% endif %}
<form role="form" method="post" class="add-album">
<div class="form-group">
<label for="artist_name">Artist</label>
<ul class="errors">
{% for e in form.validation-errors.artist_name %}
<li>{{e}}</li>
{% endfor %}
</ul>
<input type="input" name="artist_name" class="form-control" id="artist_name" value="{{ form.new.artist_name }}">
</div>
<div class="form-group">
<label for="album_name">Album</label>
<ul class="errors">
{% for e in form.validation-errors.album_name %}
<li>{{e}}</li>
{% endfor %}
</ul>
<input type="input" name="album_name" class="form-control" id="album_name" value="{{ form.new.album_name }}">
</div>
<div class="form-group">
<label for="release_date">Release Date</label>
<ul class="errors">
{% for e in form.validation-errors.release_date %}
<li>{{e}}</li>
{% endfor %}
</ul>
<input type="date" name="release_date" class="form-control" id="release_date" value="{{ form.new.release_date }}">
<p class="help-block">yyyy-mm-dd</p>
</div>
<div class="form-group submit">
<button type="submit" class="btn btn-primary">Add</button>
</div>
</form>
</div>
{% endblock %}Everything in here looks pretty familiar. We simply extended our Selmer template to include the form. The only interesting part is at #1, where we make use of the {% if %} tag. Anything inside the {% if [condition] %}{% endif %} block is only rendered if the [condition] evaluates to something truthy (that is, not nil, or an actual true Boolean value). Other than that, this seems very similar to our User Sign Up form; we have a list of potential errors for each field, and we are binding the value for each of our inputs to the values in the form.new context map.
Save the file and navigate to http://localhost:3000/albums/recently-added, then pick your jaw up off the floor after you feast your eyes on the wondrous marvel that is the following:
This looks strikingly like the mock-up we came up with. Meaning: Success! However, there's a catch.
One of the requirements that got snuck in was the following: Finally, I want to be able to use this form anywhere in the site with minimal modifications and maximum code reuse.
Hmmm. If that's the case, then the HTML code we created isn't up to snuff. It's possible we'll want to have the add artist/album form on other pages, but we're unlikely to want to copy and paste the form code every time. That's gross. Thankfully, Selmer provides a couple of tags that we can use in conjunction with each other to abstract the form and load it wherever we want. Those tags are with and include. They are explained as follows:
With the preceding tags in mind, we can actually abstract our form into its own separate Selmer template. Let's do that now:
- Create a new Selmer template alongside our
recently-addedHTML template,resources/templates/albums/add-album.html - Move the DOM inside the second
<div class="col-sm-6 col-md-4">code, from therecently-added.htmltemplate, into theadd-album.htmltemplate so thatadd-album.htmllooks like the following:{% if form.error %} <p class="bg-danger error">{{form.error}}</p> {% endif %} <form role="form" method="post" class="add-album"> <!-- snipped for brevity --> </form> - Inside the now empty
divtag, use theincludetag to load ouradd-album.html, and wrap it inside awithblock:<div class="col-sm-6 col-md-4"> {% with form=form %} {% include "templates/albums/add-album.html" %} {% endwith %} </div>
The include function tells Selmer, "Hey, compile the templates/albums/add-album.html template and load its contents here." The with block passes the context to the included template; in this case, the expected :form data structure we outlined at the start of creating the form.
When we refresh the page we should see absolutely no visual difference.
Now that we have the form in place, we need to set up the route against which the form will POST. Since we don't currently have a target attribute on the form, the form will simply POST to whatever the current URL is. For the purpose of this exercise, we'll allow that. As such, we'll need to define a Compojure route for /albums/recently-added that will accept the form input, validate it, add the data to the database, and then re-render the page. Of course, we'll break all this up across multiple namespaces, because we're good developers like that, and because we fear the wrath of our cubicle mates.
Let's create the Compojure route. In the hipstr.routes.albums namespace, add the following to our album-routes:
(POST "/albums/recently-added" [& album-form] (recently-added-submit album-form))
As the route alludes, we next want to create a function for handling the actual POSTed data – hence the need to create a recently-added-submit function.
The recently-added-submit function will validate the information submitted by the user, and then render the recently-added.html template. It will also take into account the form's expected data structure, as well as the data required for generating the recently-added-albums list:
(defn recently-added-submit
"Handles the add-album form on the recently-added page.
In the case of validation errors or other unexpected errors,
the :new key in the context will be set to the album
information submitted by the user."
[album]
(let [errors (v/validate-new-album album) ;#1
form-ctx (if (not-empty errors)
{:validation-errors errors :new album} ;#2
(try
(album/add-album! album) ;#3
{:new {} :success true}
(catch Exception e
(timbre/error e)
{:new album
:error "Oh snap! We lost the album. Try it again?"}))) ;#3.1
ctx (merge {:form form-ctx}
{:albums (album/get-recently-added)})] ;#4
(layout/render "albums/recently-added.html" ctx)) ;#5We'll also need to adjust the required libraries in our namespace definition. Add the following to the :require in our namespace definition:
(:require ...
[hipstr.validators.album-validator :as v]
[taoensso.timbre :as timbre])In the preceding code we've taken the strategy of constructing the context that we'll pass to the template being rendered. The very last thing we do is actually render the template. The basic algorithm for the preceding code is as follows:
- Validate the input for errors.
- If the input fails to validate, set those validation errors on the context, along with the user submitted data (this way they don't have to re-type their data, and they can see what was actually invalid).
- Otherwise, attempt to add the album, and provide a success flag on the context when successful.
- If adding the album fails for any reason, include the submitted data on the context, as well as a simple error message
- Add the recently added albums to the context.
- Render the template.
There are a thousand ways we could have done this. Feel free to try your own ideas! But for now, this algorithm and implementation seem pretty sound. There's one thing we may want to consider however, and that's the fact that our hipstr.routes.albums namespace now has 2 calls to render the same template in 2 different locations. Kind of yucky. Let's abstract that out to its own function, render-recently-added-html, which accepts the context:
(defn render-recently-added-html
"Simply renders the recently added page with the given context."
[ctx]
(layout/render "albums/recently-added.html" ctx))
(defn recently-added-page
"Renders out the recently-added page."
[]
(render-recently-added-html {:albums (album/get-recently-added)}))
(defn recently-added-submit
;… snipped for brevity…
(render-recently-added-html ctx)))We could probably employ further patterns to reduce things like the call to album/get-recently-added, but for now this seems pretty good. What we absolutely must do in order for this to work, however, is the following:
- Write the
validate-new-albumvalidator (v/validate-new-album). - Include the album validator as a requirement.
- Write the function to add the actual album to the database (
album/add-album!).
In Chapter 5, Handling Form Input, we were introduced to Validateur, a simple library that allows us to validate user input, as well as provide specific error messages for input that is invalid. In this section, we'll use Validateur again to write some validations for new albums.
Create the new namespace, hipstr.validators.album, and add to that file the following code:
(ns hipstr.validators.album
(:require [validateur.validation :refer :all]
[noir.validation :as v]
[clj-time.core :as t]
[clj-time.format :as f]))Note
The clj-time library provides us a sane way of working with dates. At a high level, it wraps the Joda-Time library. The clj-time library is already on our classpath as it's a dependency of im.chit/cronj included with Luminus projects. You can read more about clj-time at https://github.com/clj-time/clj-time and Joda-Time at http://www.joda.org/joda-time/.
The artist and album fields on our form are mandatory, but will allow any character. However, our database schema does have a 255 character limit on both the artist and the album columns.
If you can't remember the schema definition for a table in PostgreSQL, you can run d [table-name] inside psql and it will show you a (reasonably) formatted description of the tables:
With that in mind, let's write the artist validation first, ensuring that the artist name is an acceptable specific length:
(def artist-name-validations
"Returns a validation set, ensuring an artist name is valid."
(validation-set
(length-of :artist_name :within (range 1 256)
:message-fn (fn [type m attributes & args]
(if (= type :blank)
"Artist name is required."
"Artist name must be less than 255
characters long.")))))The preceding code should look familiar as it's very similar to the username validation in the hipstr.validators.user-validator namespace. The artist-name-validations def returns a validation set that implicitly ensures that something other than blank values makes up the artist name (the default behavior for length-of), and that the name length is a maximum of 255 characters long. We'll add the same for the album name:
(def album-name-validations
(validation-set
(length-of :album_name :within (range 1 256)
:message-fn (fn [type m attributes & args]
(if (= type :blank)
"Album name is required."
"Album name must be less than 255
characters long.")))))The release date validator is slightly more complex. Not only must the release date meet a specific format (year-month-day), it must also be a real date (there's no 13th month or 32nd day in July, for example). Ensuring the format of the data is easy, since we can use the format-of validator and a simple regular expression:
(def release-date-format-message
"The release date's format is incorrect. Must be yyyy-mm-dd.")
(def release-date-invalid-message
"The release date is not a valid date.")
(def release-date-format-validator
"Returns a validator function which ensures the format of the
date-string is correct."
(format-of :release_date
:format #"^d{4}-d{2}-d{2}$"
:blank-message release-date-format-message
:message release-date-format-message)) Again, this should look somewhat familiar, as we also used the format-of validator to ensure that a submitted username is of the proper format.
Ensuring a date is valid, however, requires a bit more thought. Firstly, there's no point in validating the release date if the format of the date string is invalid. Secondly, we need to ensure the date is parseable and valid. To do this, we'll use some of Clojure's date/time libraries, as well as Validateurs's validate-when predicate.
The validate-when function returns a function that will run a validator if, and only if, a provided predicate returns true. You can think of validate-when as Validateur's "if predicate true, then do such and such". In our case, we only want to validate the release date when it's in the correct format. As such, we can leverage our release-date-format-validator to be the predicate for validate-when:
(def release-date-formatter
(f/formatter "yyyy-mm-dd"))
(defn parse-date
"Returns a date/time object if the provided date-string is
valid; otherwise nil."
[date]
(try
(f/parse release-date-formatter date)
(catch Exception e)))
(def release-date-validator
"Returns a validator function which ensures the provided
date-string is a valid date."
(validate-when #(valid? (validation-set
release-date-format-validator) %)
(validate-with-predicate :release_date #(v/not-nil? (parse-date (:release_date %)))
:message release-date-invalid-message)))In the preceding code, the validateur.validation.valid? is our test condition, and, only when its result is true, will our validate-with-predicate actually be executed. The validate-with-predicate validator ensures that parse-date returns something other than nil, which will only occur if parse-date fails to parse the date string (a result of the date string either not being in the expected format or not being a real date).
Finally, we can wrap these up into a simple, easy-to-use validation set:
(def release-date-validations
"Returns a validator which, when the format of the date-string
is correct, ensures the date itself is valid."
(validation-set release-date-format-validator
release-date-validator))The last thing to do, now that we have all this wonderfully modular code, is to create the validate-new-album validation set which will be used from the hipstr.routes.album/recently-added-submit function:
(def validate-new-album
"Returns a validator that knows how to validate all the fields
for a new album."
(compose-sets artist-name-validations album-name-validations
release-date-validations))Phew! Data validation…Always the most tedious thing. Users should just know, shouldn't they? Either way, we're done! The last thing we need to do is expand our hipstr.models.album-model to include the add-album! function.
Currently, our hipstr.models.album-model namespace defers everything to the YeSQL-generated queries from the hipstr/models/albums.sql file. This is fine, as we will leverage it to add a couple new functions. However, because our constraints are such that album.artist_id is a foreign key to artist.artist_id, we must have the means of inserting a new artist. As such, we'll create a new SQL file, hipstr/models/artists.sql.
The first function we'll define in the artists.sql file is the insert-artist<! function:
-- name: insert-artist<! -- Inserts a new artist into the database. -- Expects :name. INSERT INTO artists(name) VALUES (:artist_name);
The artist name is the only value we need to provide because everything else in the artists table is generated for us by PostgresSQL. The create_at and update_at field values are handled by our trigger, and artist_id is handled by Postgres because we defined it to be SERIAL (that is, an auto-incrementing integer). Great! However, you may be wondering (and possibly groaning), "Do we need to create a hipstr.modesl.artist-model now?" To which I reply, "No."
There is nothing prohibiting us from making multiple calls to YeSQL's defqueries function and passing it a different SQL file each time. In this spirit, we can generate multiple functions across multiple tables in the current namespace. As such, the only thing we need to do in order to make use of our shiny new insert-artist<! function is tell YeSQL to make use of it. Add the following to the hipstr.models.album-model namespace:
(defqueries "hipstr/models/albums.sql" {:connection db-spec})
(defqueries "hipstr/models/artists.sql" {:connection db-spec})
YeSQL will generate functions for both SQL files and load them into the current namespace, and by default run any SQL queries using the same database specification (db-spec). With that in mind, let's add the insert-album<! function to the hipstr/models/albums.sql file as well. Add the following:
-- name: insert-album<! -- Adds the album for the given artist to the database -- EXPECTS :artist_id, :album_name, and :release_date INSERT INTO albums (artist_id, name, release_date) VALUES (:artist_id, :album_name, date(:release_date));
Again, this is a pretty straightforward insert statement. Life is easy in these parts of the woods. Now that we have those queries available for generation, let's make use of them.
Back in the hipstr.models.album namespace, add a new (and dare I say, first!) function, add-album!, which will accept the same information posted from our Add artist/album form; that is {:artist_name :album_name :release_date}. The first iteration of this function will blindly add the artist and album to our database:
(defn add-album!
"Adds a new album to the database."
[album]
(let [artist-info (insert-artist<!
{:artist_name (:artist_name album)})]
(insert-album<! (assoc album :artist_id
(:artist_id artist-info)))))The preceding code is pretty simple. Add the artist, associate the new artist_id with the album, and then insert the album. This works like gravy on potatoes until you try to add a second album by the same artist (our UNIQUE INDEX constraint on the artists.name column will make sure of that). The same thing will happen if we try to add an album of the same name by the same artist, as we have a UNIQUE INDEX spanning the albums.artist_id and albums.name columns.
The requirements don't state to throw an exception if the artist or album already exists. And frankly, I don't see the point. Let's not be wasteful and just make use of the already existing artist/album, if that's the case. This implies, of course, that we have a way of getting the artist or album, by name, from the database. No sweat! Let's add a get-artists-by-name to our artists.sql file:
-- name: get-artists-by-name -- Retrieves an artist from the database by name. -- Expects :artist_name SELECT * FROM artists WHERE name=:artist_name;
Similarly, let's add the get-albums-by-name query to our albums.sql file:
-- name: get-albums-by-name -- Fetches the specific album from the database for a particular -- artist. -- Expects :artist_id and :album_name. SELECT * FROM albums WHERE artist_id = :artist_id and name = :album_name;
Again, not a whole lot of surprises here. We include an artist_id in the WHERE clause so we can limit the scope of the albums returned.
Lets get back to our hisptr.models.album-model/add-album! function. Now that we have a couple more queries, let's make use of them to avoid the pitfalls of duplicate artist/albums. Adjust the function so that it resembles the following:
(defn add-album!
"Adds a new album to the database."
[album]
(let [artist-info {:artist_name (:artist_name album)}
; fetch or insert the artist record
artist (or (first (get-artists-by-name artist-info)) ;#1
(insert-artist<! artist-info))
album-info (assoc album :artist_id (:artist_id artist))]
(or (first (get-album-by-name album-info)) ;#2
(insert-album<! album-info)))) In the preceding code, we're now checking that an artist or album exists and, if not, we add them. The only thing that might stump us is the call to first at #1 and #2. Despite the logic of the query and the schema guaranteeing only 1 or zero returned rows, we'll still need to make a call to first because query results are always returned as a sequence. Remember, YeSQL isn't intelligent; it's not examining the schema or query and making necessary adjustments, it's merely generating Clojure functions which map back to the SQL for us.
We now have a model function which makes use of an already existing artist/album and inserts only when required. However, there's one glaring hole: it's not in a transaction. The function could, in theory, successfully insert an artist, and then fail when inserting the album, which isn't what we would expect from such a form. Let's ensure that doesn't happen.
At the start of this chapter, we learned that wrapping YeSQL functions inside a transaction is trivial. Every YeSQL-generated function takes, as its second (optional) argument, an alternate connection, which can be a handle to an existing transaction. We can pass a transaction context, generated by clojure.java.jdbc/with-db-transaction, as the second parameter.
First, add a reference to the clojure.java.jdbc library at the top in our :require:
(:require [clojure.java.jdbc :as jdbc] …)
Secondly, wrap the body of add-album! inside the call to with-db-transaction:
(defn add-album! "Adds a new album to the database." [album] (jdbc/with-db-transaction [tx db-spec] ;#1 (let [artist-info {:artist_name (:artist_name album)} txn {:connection tx} ;#2 ; fetch or insert the artist record artist (or (first (get-artist-by-name artist-info txn)) (insert-artist<! artist-info txn)) album-info (assoc album :artist_id (:artist_id artist))] (or (first (get-albums-by-name album-info txn)) (insert-album<! album-info txn)))))
At #1, in the preceding code, we get a new transaction handle, tx, from the jdbc library; and then, at #2, we create a connection map to use the tx transaction handle. All of our subsequent calls to YeSQL then use the txn connection map. And that's it! The add-album! is now fully wrapped inside a transaction. Go ahead and fail, world! But there is one, single, tiny improvement we can make.
A downfall to our preceding transaction code is that it will never allow any transaction other than its own. We are good programming citizens, cognizant of the needs of others. And it's quite possible there will be cases wherein add-album! will be part of a larger transaction. We can support this by making use of arity overloading. Modify the add-album! function such that it accepts multiple arities: one that accepts only an album, and another that accepts an album and a connection:
(defn add-album!
"Adds a new album to the database."
([album]
(jdbc/with-db-transaction [tx db-spec]
(add-album! album tx)))
([album tx] ; #1
(let [artist-info {:artist_name (:artist_name album)}
txn {:connection tx}
; fetch or insert the artist record
artist (or (first (get-artist-by-name artist-info txn))
(insert-artist<! artist-info txn))
album-info (assoc album :artist_id (:artist_id artist))]
(or (first (get-album-by-name album-info txn))
(insert-album<! album-info txn)))))The preceding code extracts the meat of the add-album! function into its own arity (#1), which accepts the album to be inserted and a second, alternate connection. The downside of this pattern is that the connection parameter (tx) doesn't have to be a transaction. Thus, clients of this code now have the ability to add an album in a non-transaction state. But that's okay, sometimes you need to give somebody the power to shoot themselves in the foot in order for them to learn.