Syntax-Quoting and Unquoting

The big problem with this assert implementation is that it takes a pretty big structural leap to go from the macro implementation to the result of the macroexpansion. This is no problem for the compiler, and our human brains can work it out too, but there’s an easier way. The syntax-quote gives us a way to structure a macro’s code to look much more like its macroexpansion.

The syntax-quote lets us create lists similar to the way we create them with a normal quote, but it has the added benefit of letting us temporarily break out of the quoted list and interpolate values with an unquote. Think of it as a template, where we can punch holes and insert values wherever we like. For instance, if we had a list where we wanted to insert a value, our normal quote wouldn’t fly:

The fourth element in the first list we created is just the symbol a. If we want the value of a, we have to either use the more verbose list construction, or use a syntax-quote with an unquote:

If you don’t see the difference in these two quote characters at first, look a wee bit closer. The normal quote (’) looks like it’s on the straight and narrow, and the syntax-quote (‘) is a little cockeyed and apparently ready to party. You might also know the syntax-quote as the backtick. The unquote (~), well, it unquotes, letting us insert evaluations into the syntax-quoted expression.

If we take a look at how assert is actually implemented, we see that syntax-quote and unquote completely solve the verbosity problem we saw earlier:

Wow! That’s a lot less code than the nested-list version, and it looks a lot closer to the macroexpansion. As a result, it’s easier to understand and maintain. There’s one potentially tricky bit left, though: what does it mean when we say ’~x within a syntax-quoted expression?

The REPL is a great place to experiment whenever you see something you don’t understand. Why don’t we give it a try?

Aha! So this strange ’~ dance gives us a way to quote the result of evaluation and plug that into a slot in the syntax-quote expression.

Recall from our introduction to quoting that the normal quote is a reader macro expanding to (quote ...). Well, it turns out that the unquote ~ is another reader macro. So an expanded version of this would look like:

Internally, Clojure’s reader has some special code to walk through the syntax-quote form looking for clojure.core/unquote occurrences and unquoting those things. I wouldn’t try to use clojure.core/unquote outside the scope of a syntax-quote, though—it won’t work unless you’ve written a macro that makes it work. Leiningen^[12] (as of version 2.3.4) actually allows the unquote to be used in project.clj for evaluation, but it’s now discouraged in favor of read-eval. That var (clojure.core/unquote) is unbound by default, and it’s unclear whether it’s strictly needed by the language, since everything using it looks for a symbol, not a var.

Joe asks:

What’s Read-eval?

Read-eval is the name of the Clojure reader macro that allows you to evaluate code during a read. For instance, (read-string "#=(+ 1 2)") returns 3, not (+ 1 2) as you’d get without the #=. This has security implications for read and anything that depends on it, so you should always be careful not to read user input, preferring something like clojure.edn/read instead. Generally you should make it a goal to steer clear of read-eval whenever possible.

There are other strange ways we can mix and match these quotes and unquotes as well, but first let’s see a special kind of unquote called unquote-splicing. Let’s say we have a list in hand of an unknown length, and we want to insert all the elements from that list into another list. Using the unquote that we already know about won’t fly here, but we can use the usual concat to get the result we want. Always remember that when you hit a wall in writing macros, you can fall back on all your normal Clojure-writing experience, since when you write macros you’re manipulating normal data structures like lists.

This concat version is a little unsatisfying. It works fine for this use case, but it might not be so great if we needed to inject the values into a syntax-quoted expression. Luckily, the unquote-splicing reader macro, ~@, gives us a succinct and fully syntax-quote-compatible way of doing the same thing:

And there are hooks in Clojure’s syntax-quote implementation in the reader that look for clojure.core/unquote-splicing occurrences, just like with the normal unquote, to make this behavior possible.

Besides the ability to interpolate values when syntax-quoting, there’s an additional implication for symbols that occur within the syntax-quoted form. Take a look at the difference between this normal-quoted expression and its syntax-quoted sibling:

With the syntax-quoted version, the symbols all include the namespaces where the syntax-quote appears! We say that these symbols are namespace-qualified. At first this might seem strange, but there’s a good reason for it. Imagine you had this macro (which, as you’ll see in Chapter 3, ​Use Your Powers Wisely​, is a bad idea to begin with, but we’ll use it here for clarity):

Easy enough—we just map over the input collection, squaring each element.

What do you think would happen if we were to use this macro from a namespace where map meant something different?

So in a situation like this, since the verb map is an unqualified symbol, the squares macro call in the namespace foo will cause foo/map to be used as a function. The squaring function gets passed in as the value to look up, and the not-found default gets returned as the result in these cases. Well, this clearly isn’t what we wanted when we wrote that macro, and that’s where syntax-quote’s namespace qualification leaps to the rescue. If we instead define the squares macro using a syntax-quote (or if we’re into making things cumbersome, namespace-qualifying the map symbol ourselves), we don’t have this problem:

This is a great example of how Clojure’s macro system tries to help you avoid shooting yourself in the foot. Use and master the syntax-quote, and your macro-writing life will be much easier. But we do need to go just a bit further to get you all the syntax-quoting knowledge you need.

If we had chosen to write the squares macro in a slightly different way, using the fn special form instead of the shortcut syntax (#(* % %)), we would have seen an error:

Ack! The namespace expansion has bitten us in this case, rather than helping. We could roll back to the list approach without too much pain here, but we’d really like to take advantage of syntax-quote’s benefits. Given what you know about syntax-quoting and unquoting, you can extrapolate to a solution that puts a non-namespaced symbol in the macroexpansion:

This solution would work fine for our use case: we could slap ~’ in front of all the xs in the definition of squares and have a working implementation in no time:

And because the scope of x is limited to that anonymous squaring function, we don’t have the namespace-related problems we saw before. However, let’s expand our worldview to include macros other than the now-worn-out squares:

So make-adder expands into a function definition, and that function adds the macro’s argument to the function’s argument. This has a strange result when we try to use it:

Despite the fact that we defined y to be 100, it looks like the value being used for y is 5! Why do you think this is? By macroexpanding the make-adder call in question, we can see that we’re creating a function that takes one argument named y, which shadows the (def y 100) definition:

If you were trying to use this version of make-adder without digging into the macro definition, you’d think this was horribly broken, wouldn’t you? We’ve accidentally allowed what’s called symbol capture, where the macro has internally shadowed, or captured, some symbols that users of this macro might expect to be available when their expression is evaluated. There is a solution here, though, and it’s called the gensym.