Macroexpansion

Manually walking through a macroexpansion, as you saw in the previous section, is fairly tedious work. But as you might guess, you don’t always have to do it manually. Clojure exposes to users the very same macroexpansion tools that it uses internally. This way, you can have a look at the expression a macro will generate, without having to go through that whole process you just saw.

macroexpand-1 is the simplest of these tools, converting a macro expression to its resulting expression.

Note the similarity here to what we did by hand in the last section with our first when expression. macroexpand-1 is a big help when debugging macro issues, because it tells you precisely how a macro expression will be replaced during the macroexpansion step. Make sure you give macroexpand-1 a quoted expression or something that generates one, or you’ll be in for a surprise!

Omitting the quote here failed to do what we wanted, because macroexpand-1 is a regular function. This means it will evaluate its arguments before passing control to the code that composes macroexpand-1. So the when expression actually executes, returns nil, and since nil isn’t a macro expression, macroexpand-1’s job is done. Macroexpansion has no effect when a given expression is not a macro call.

It’s worth noting that macros must return something that it makes sense to evaluate! The when macro works because it returns a list whose first element is the quoted symbol if. Let’s see what would happen if we rewrote our own version of the when macro, but forgot that the result of the macro needed to be an eval-able form. We’ll delete the ’if quoted symbol to demonstrate the point—removing the docstring and other metadata just makes it easier to focus.

When I’m learning something new, I sometimes find myself practicing EDD (exception-driven development). I try to evaluate some code, get an exception or error message, and then Google the error message to figure out what the heck happened. We could do that here, and we’d probably find the right answer, but since we now know about macroexpand-1, let’s try that first.

Aha! The expression we generate here is a list, and its first element is another list, so in order to decide how to evaluate the top-level list, we’d need to evaluate the first element. (= 1 1) is true, so the first element of the top-level list will be true. But true is not a verb of any sort (function, macro, or special form)—it’s a Boolean! This explains the error message entirely: we expected to have an IFn (a function), but we got a Boolean, and that doesn’t make any sense. You’ve probably seen this kind of thing before in Clojure, when accidentally typing unquoted lists at the REPL. As long as we make sure the expressions we return from our macroexpansion are possible to evaluate, we’ll avoid these kinds of errors.

We can go a step further with macroexpand, which does what macroexpand-1 does, but it actually continues along in the same way until the returned expression is either a non-list or a list whose first element is no longer a macro. So, for example, if we have a macro that expands to another macro call, macroexpand will do that second expansion for us, but macroexpand-1 will not. This is in contrast to the ladder idea, where a macro might use another macro instead of expanding to another macro.

Both tools have their place, depending on whether you want to examine a single macroexpansion or look at the final product. Note that this applies only to macros at the start of the expression being expanded—expanding macros within the expression would need heavier machinery. It’s tricky to correctly macroexpand everything, and the built-in clojure.walk/macroexpand-all works for some expressions but is too naïve to cover all cases. clojure.tools.macro^[9] and Riddley^[10] are two more ambitious code-walking projects intended to allow expanding everything, with different strategies based on those projects’ goals.

A Confession, and Next Steps

So the macro ladder analogy is a bit of a stretch. It’s a decent way to figure out which macro code will execute, but not necessarily when. When we write a macro that uses a macro to do its expansion, Clojure will perform the necessary expansion when it’s compiling our macro. Whenever we go to use a macro, any macros it uses have already been expanded. In the case of cond, Clojure doesn’t need to climb any of the ladder rungs above when, as they have already been traversed by the time we call (cond). This means that there’s usually only one ladder rung that we have to worry about at any given time.

But the ladder concept can still be useful when tracing code execution in a new macro, because unlike the Clojure compiler, when we do macroexpansion in our heads, we don’t hold onto the expansions of every macro in memory! We humans usually have to look at each function or macro in isolation, and either remember know by heart what a given macro call does or find its definition and mentally expand it (or use tools like macroexpand-1 as we explore).

At this point we have most of the tools we need to write macros. Next we’ll look at some advanced techniques and syntax to avoid some of the verbosity that’s required with what you know so far, and even some tricks that make crazy new things possible.