Error Handling in Macros

Most macros, like our pattern matching example in the previous section, have very specific sets of input that they accept, and will throw strange-looking errors when those input expectations aren’t satisfied. For instance, if we didn’t realize that snore.match only handled vectors as input, we might try to match against a keyword, which would give us this error:

This stack trace gives us all the information we need to trace the code path from the call all the way to the place the exception was thrown, but it doesn’t tell the user of the macro that they’re just using the macro with unexpected values. And if you’re not used to reading compiler stack traces or remembering to try macroexpanding the expression, that long stack trace might be a bit overwhelming. There are a few ways to deal with this issue, some more common than others, and each with pros and cons:

The first two options, doing nothing and writing documentation, are quite easy to understand. Writing good documentation is difficult, but it’s something we’re used to seeing, and hopefully doing, as programmers. The other options, on the other hand, could use some discussion.

Making the macro smarter, to accept more kinds of input, seems like a great goal at first blush. It’s basically Postel’s Law^[36] applied to macros. This can be a great hack for simple and obvious errors. But the problems space of “errors a user might make” is vast, approaching infinite, and with many errors, it’s hard to even reason through why they were made. Unfortunately, being able to detect and fix any user error is not a realistic goal. So even if we’re able to give the user some help and we don’t mind a bit of added code complexity, it’s not a complete solution.

If we wanted to improve the error messages for our mistaken pattern match call, we could start by adding an assertion that the match input is a vector, either with explicit assert calls or with Clojure preconditions:

But how do we know what assertions to add? We might have happened upon that annoying error message because a coworker was blocked for half an hour trying to figure it out, but what happens next time? Have we provided good error messages for all the things someone could do wrong? What if someone assumes that our match is like case instead of cond, in that its last clause doesn’t need an :else prefix? As things stand, they might be in for a surprise:

So to provide a good error message for this mistake, we need an additional precondition that checks that there are an even number of expressions after the match input:

Are we finished now? Perhaps so. match (or at least our version of it) is a pretty simple macro—we haven’t provided many places for behavior to diverge. But the question of completion is fuzzier for macros with more complex input constraints, like this one that acts like defn but provides a default docstring where none is given:

Enumerating all of the possible things that can go wrong for a user here would be time-consuming. There’s a combinatorial number of things the user could do wrong, with the function name, docstring, function arguments, pre- and post-conditions, and metadata map. You may remember making some confusing mistakes around what the defn or ns macros allowed when you were first learning Clojure. In situations like these, the idea of illuminated macros is particularly compelling, and we’ll look at that next.

Illuminated Macros

The idea of illuminated macros, as far as I can tell, was introduced in a talk of the same name at Clojure/Conj 2013^[37] by Jonathan Claggett and Chris Houser, where they discussed the pain of learning to use complex and under-documented macros, and a potential solution, based in part on research in Scheme by Culpepper & Felleison.^[38] They introduced a library called seqex^[39] that aims to help automatically create both good documentation and good error messages for macros that are written using its tooling.

Do you remember when you were still learning the syntax for defn, and where the docstring goes in relation to the argument list and any preconditions? The built-in defn macro does a pretty good job of providing helpful error messages, but it’s had many hours of thought put into it, and it’s not perfect:

A seqex-based implementation of defn goes even a bit further and provides both additional context around errors and generated grammar documentation:

The syndoc syntax documentation for this macro looks just like a BNF^[40] grammar, and because it’s automatically generated from the same code that provides the error messages, it’s bound to stay in sync even if the allowable macro inputs change!

This is a pretty big win for anyone who’s struggled with poor macro usability. It’s not without cost, though: syntax/defn takes work to implement, even though it happens to be built on top of clojure.core/defn. seqex, as you may have guessed based on its name or problem domain, uses tools that the authors have dubbed “sequence expressions,” which are for sequences what regular expressions are for strings. Ignoring the most complex parts of syntax/defn, the destructuring logic, there’s still a lot of thought that’s been put into structuring the sequence expressions:

This style of macro-writing hasn’t yet taken the Clojure community by storm, but that may change because of the documentation and error-generating benefits. Quite recently, another entrant has appeared on the sequence expression scene: Christophe Grand’s elegant and powerful seqexp.^[41] At version 0.2.1, it’s more of a single-purpose tool, currently only handling the sequence expression part of the problem, and not the generation of error messages or documentation. I wouldn’t be surprised to see those kinds of features built on top of seqexp in the future, though. Because most macros tend to be fairly focused and because the cost of switching over to an illuminated approach is high, we’re likely to keep writing macros in the normal way for the foreseeable future. But as a user of complex macros, I’d sure like to have the benefits conferred by the illuminated macro approach.