Evaluating Your First Macro

In order to think about how macros work, it’s useful to picture a ladder like the one in the figure that follows this paragraph. When we take a step up this ladder, we’re shifting from thinking in code to thinking in data, and when we step back down the ladder, we’re shifting from data back to code. In this world, when we encounter a macro call in a piece of code, we’ll think of it just like a function, but one that operates one level up the macro ladder, on unevaluated code rather than on evaluated data. When we need to expand a macro, we’ll step up the ladder, and once we’re done expanding it, we’ll step back down (with the expanded expression in our pocket). This is a simplification, but it’s a good enough way to think about macros for now.

Without further ado, let’s look at the built-in when macro.

Since we’re thinking about macros as functions operating one step up the macro ladder, we can think of when as being a function that receives an argument test, and a variable number of arguments rolled up into a sequence as body. Now let’s look at an actual call of when to see how this plays out in practice.

Just like a normal function, when has values bound to test and body when it executes, but unlike a normal function, nothing is evaluated yet. test actually gets bound to the list (= 2 (+ 1 1)). And this list is just data. Sound familiar? No addition or comparison happens yet; this is that wedge between read and eval that we alluded to earlier. Moving on, body is bound to the list ((print "You got") (print " the touch!") (println)): a list of lists, in fact! The extra set of parens come from rolling up three arguments due to the & body varargs (variable number of arguments) syntax.

So if we keep on reading when as a function, substituting the bindings for the names in the code, and then executing the macro body (not the resulting code), the result is another list.

And indeed, this is the code that the when macro generates, but in our model of macros as functions acting on data, we’re still thinking of this return value as a list, one step up the macro ladder from the bottom. So now we just need to descend the macro ladder by one rung, where we can evaluate this code that we’ve constructed. And of course, the result will be that we print out a lovely song lyric from a wonderful film (well, I choose to remember it as wonderful, anyway).

This is pretty much how macros work. There is indeed an opportunity to wedge behavior in between reading and evaluating the code, and that wedge is named macroexpansion. Whenever Clojure encounters a macro in the verb position on an expression it’s trying to execute, it will iterate this process we’ve just gone through. Macroexpansion is stepping up the macro ladder one rung: we treat the code of the macro’s arguments as data. Those arguments are used in whatever way the macro says, and eventually we create a new expression by evaluating the code that makes up the body of the macro. We put that resulting expression into our pocket, descend the macro ladder one rung again, take the expression out of our pocket, and replace the original macro expression with the resulting expression for execution.

This may feel a bit like macros you’ve seen elsewhere, in C or C++, or even Scala. The difference is that with Lisp we write macros in terms of normal sequence manipulation rather than string or AST manipulation. We don’t need to write our own parser or know how to generate the compiler’s syntax tree nodes. In Lisp, metaprogramming feels more similar to regular programming.

Of course, things are often more complicated in practice. In a larger example, we might encounter another macro while in the process of expanding the first macro. Let’s consider another macro defined in terms of when: cond.

This one will take a little more effort to expand, because we need to continue climbing the macro ladder until the expression we have is no longer a macro call. When we try to macroexpand the smallest possible invocation of cond, passing no arguments, we’re already one macro-ladder rung up from the bottom, and so we might feel a little stuck when we find when (a macro) in the verb position.

As humans, we know that clauses will be nil, since no clauses were passed in, but we can’t yet substitute nil in place of clauses in this when expression, since we’re not at the bottom of the ladder. when is a macro, and macros act on code, evaluating if and when they choose. We need to climb the ladder another rung to see what lies in store for us, but we’ll keep in mind that clauses will be bound to nil when we get back down to the first macro ladder rung.

Now since list is a function, we get to evaluate the arguments and move back down a rung to get closer to our eventual return value.

Now our result starts with a special form, if, and we’re still one rung up from where we started, so we need to evaluate the form in the way the Clojure language defines if. Since if says we evaluate the test (the first argument) first to decide whether to evaluate the second or the third argument, we need to first evaluate clauses.

And because we’re at level 1, and level 1 is where we have clauses bound to nil, we have that binding available for this evaluation. Since nil is a false-like value, we’d ordinarily go down the else branch (the third argument) of the if, but there is no third argument! This leaves us with nil as the result, because Clojure defaults the else branch to be nil.

So at long last, we’ve arrived at our goal: the result of macroexpanding the expression (cond). And that result is just nil. So we put that in our pocket, descend the last ladder rung back to the bottom, and insert that nil into the original context as code. Well, nil as code just evaluates to itself, so the result of evaluating (cond) is nil as well.

Sigh—that was pretty anticlimactic, and it took a lot of tedious and error-prone work to sort it out by hand. Trust me: macro writing is not always going to feel like this. It will become easier with each macro you write, and the macros you use will fade away into your tacit knowledge of your systems. And luckily, we do have tools to manage these sorts of complexities and to check our work as we write macros. Let’s look at those, before we start feeling acrophobic with all this ladder talk.