Chapter 13. Why ClojureScript?
- Protocols underlying everything
- Stages of ClojureScript compilation
- Web Audio
- Advanced Google Closure compilation
- Google Closure externs files
- Differences between compile and runtime
ClojureScript is a redesign and reimplementation of Clojure and its compiler, written in a combination of Clojure and ClojureScript itself. Although the original purpose of the ClojureScript compiler was primarily to generate JavaScript to target runtimes and devices supporting it, in the long term this may be the least important feature of its design. After all, Clojure’s own compiler, which currently generates JVM bytecode, could have been altered to produce JavaScript as well.
In this chapter, we’ll examine the major design differences between Clojure and ClojureScript, and explore the implications of these differences. Even though ClojureScript is newer and designed with all the knowledge gleaned from the development of Clojure, Clojure is by no means abandoned. Instead, ClojureScript is an alternative implementation of Clojure that targets any platform that runs JavaScript. ClojureScript cleverly reuses parts of Clojure such as the reader and macroexpansion machinery. This implies ClojureScript’s design isn’t superior to Clojure’s in every context, so we’ll discuss that as well.
The differences between Clojure and ClojureScript are listed on the GitHub wiki (http://mng.bz/kKZK), but many of them result from the differences between JavaScript and the JVM. They may be important details when writing programs, but they don’t speak to the motivations that drove the redesign and thus are perhaps a less rich topic. Although some of these more trivial differences will come up naturally in this chapter, our focus will be on design decisions that separate concepts into simpler parts or take better advantage of parts Clojure has already separated:
- Clojure’s protocols provide good separation of interfaces from implementations. In the next section, we’ll look at how ClojureScript makes more use of protocols than Clojure does.
- In section 13.2, we’ll look at how the ClojureScript compiler has analyze and emit phases separated by regular data values.
- Finally, in section 13.3, we’ll examine how ClojureScript clarifies the distinction between compile time and runtime.
Let’s begin by comparing the approaches taken by Clojure and ClojureScript in both implementing and defining the interface to count.
13.1. Implementation vs. interface
Many of the core abstractions provided by Clojure are defined using Java interfaces, whereas ClojureScript uses protocols for the same purposes. ClojureScript’s approach provides much more power to programmers, as you’ll see in this section. One such abstraction is Clojure’s count function, which returns the number of items in a collection. Although it’s defined in core.clj, the definition calls the static method clojure .lang.RT/count. Following is the implementation of this method in Clojure 1.5.1:
The methods count and countFrom form a hand-coded type-based dispatch, where the type of the collection indicates which specific behavior count exhibits. The first type checked is clojure.lang.Counted 
, an interface provided by Clojure specifically to indicate that a collection knows its size so count can return in constant or near-constant time; a map is an example. Many of Clojure’s collection types implement Counted directly, and you are free to do so in any appropriate collection types you create. But many other types existed before Clojure’s
Counted interface was created, and although they provide methods for computing their size, they do so using different methods. Thus
you have the fallback option 
and the various else if clauses of countFrom 
to dispatch to the appropriate functionality for these other types. Handling all cases in a single chain of else if works, of course, and is relatively fast at runtime. In particular, count delegates to countFrom so count can remain small enough that the JVM sees it as a good candidate for inlining.
But despite being sufficient, the design is closed—if you discover another type (either in Java or in a third-party library) that would be a good candidate to implement count, you have a limited and rather poor set of options. These may include wrapping the object in a new object of your own that implements Counted and delegates at runtime, patching Clojure to add the type to countFrom, and giving up and not using count. The first of these is a popular option, but it has several drawbacks. Wrapped objects aren’t equal to their unwrapped counterparts. Also, consider what happens if another programmer wraps your object again to implement a different interface. This is another example of the expression problem we mentioned back in section 1.4.3.[1]
1 You can learn more about the expression problem in the description by Philip Wadler at http://mng.bz/sRul.
The solution ClojureScript uses, not surprisingly, is its protocol facility, which if you’ll recall is a feature that allows you to describe interfaces for related functions (discussed more deeply in section 9.3.2). But whereas protocols allow you to solve the expression problem in your own code, they can’t help Clojure itself because the code that implements protocols is built on top of a good deal of Clojure. This is where ClojureScript comes in. As you’ll see in a moment, when given the opportunity to design ClojureScript, Rich and his team chose to put protocols at the bottom, building the other abstractions on top rather than the other way around as they did with Clojure. Thus the definition of count in ClojureScript looks a bit different:
The function count in ClojureScript delegates 
directly to the -count method of the ICounted protocol . This is then extended to JavaScript nil and arrays . The primary benefit of this approach is that you can make Clojure’s count function work with any existing JavaScript type. For example, listing 13.1 shows how you can make count work on Google’s rather interesting LinkedMap objects, without monkey patching or risking name collisions even if an object gets a method named count or -count. This is a substantial benefit, and ClojureScript does this in a way that has very good runtime performance, which is why
it was deemed appropriate to build ClojureScript’s internal behavior on top of protocols. Protocol implementations can be
dispersed to places in the code that may be far from each other, as you see in the large variation in line numbers in the
previous listing.
Listing 13.1. Extending ICounted to LinkedMap
The important work of telling count how to count LinkedMap is done by extend-type . In this example, the instance is created after the type is extended, but reversing the order of these would work fine. This listing demonstrates that it works by putting a couple entries in the LinkedMap and then using ClojureScript’s normal count method on it .
Another example of the flexibility provided by ClojureScript’s deep use of protocols is the IFn protocol, which takes the place of the IFn interface in Clojure: any object that satisfies IFn can be called as a function. In Clojure and ClojureScript, persistent maps can be called as functions taking one argument as a key to look up within themselves:
(def pm {:answer 42})
(pm :answer)
;=> 42
But Clojure doesn’t allow you to use Java’s Map objects in this way; and, similarly, ClojureScript doesn’t let you use Google’s Map objects as functions. This is probably for good reasons, perhaps having to do with functions being immutable, whereas while those host-provided collection objects aren’t. And in Clojure, nothing can be done but accept the wisdom of this decision: in Clojure, IFn is a Java interface and thus can’t be extended to the existing Java classes, a victim of Java’s unsolved expression problem.
In ClojureScript, you can ignore the value of keeping functions immutable, throw caution to the wind, and extend IFn to Google’s LinkedMap.
Listing 13.2. Extending IFn to LinkedMap
Regardless of whether this particular extension of IFn is a good idea, the separation of interface from implementation provided by protocols—and ClojureScript’s use of these as its most basic abstraction—hands a great deal of power to users of ClojureScript.
Another way ClojureScript grants power to its users is by separating the phases of the compiler. Let’s look at that next.
13.2. Compiler internals: analysis vs. emission
In this section, we’ll present an overview of the ClojureScript compiler by taking a small example expression through all the stages normally applied by the compiler. Then we’ll present a much more substantial ClojureScript program, which we’ll use to discuss compilation details you wouldn’t normally see in tiny programs, including advanced compilation and externs files.
13.2.1. Stages of compilation
When compiling Clojure code, there’s no official way[2] to see anything between the source code going in one end and the bytecode coming out the other end. ClojureScript, on the other hand, somewhat more officially separates the compiler’s analysis phase (where the input code becomes understood) from the emission phase (where the target language is specified). The data format between these phases is the output of the analysis phase, known as the abstract syntax tree (AST).
2 Although an exciting unofficial way is provided by Ambrose Bonnaire-Sergeant’s https://github.com/clojure/jvm.tools.analyzer contrib library.
A common misconception perpetuated about Lisp languages (and Clojure, of course) is that because source code is data, this data is an AST. But although AST is a fuzzy term whose specific meaning may vary depending on context, we think it’s not an accurate way to describe Clojure source code in either its textual or its read form. For example, the forms passed into a macro aren’t yet macroexpanded, and even after macroexpansion there is more analysis to be done.
Although there is no comprehensive documentation for either Clojure or ClojureScript’s AST, the ClojureScript version is just a tree of immutable data. We’ll show you how to examine the AST of a given ClojureScript form, and then you’ll do something useful with that information. We’ll begin with an initial tour of the compilation process and the AST. Next we’ll introduce a more substantial body of input code to work on, specifically some functions that use the Web Audio API to play music. Then you’ll use what you’ve learned about the AST to examine the audio example code to help you produce more compact JavaScript output.
Let’s begin by looking at the stages of compilation leading up to analysis, starting with reading, which converts text into Clojure data. This requires the ClojureScript compiler, which is packed by default in the ClojureScript distribution. Run lein repl in a directory that contains a project.clj file, like this:
(defproject joy/music "1.0.0"
:dependencies [[org.clojure/clojure "1.5.1"]
[org.clojure/clojurescript "0.0-1835"]])
Start with a string containing the textual code for a defn:
(require '[cljs.compiler :as comp]
'[cljs.analyzer :as ana])
(def code-string "(defn hello [x] (js/alert (pr-str 'greetings x)))")
To convert the code-string into Clojure data, read it with read-string:
(def code-data (read-string code-string)) code-data ;=> (defn hello [x] (js/alert (pr-str (quote greetings) x)))
The stark difference between text and data becomes clear when you begin to interact with it:
(first code-string) ;; A Character: ;=> ( (first code-data) ;; A Symbol: ;=> defn
As with all LISPs, Clojure and ClojureScript both separate the reading done here from the following compilation stages. But ClojureScript separates the compilation itself into analysis and emission more clearly than Clojure does. So the next step is to analyze the read code, generating an AST:
(def ast (ana/analyze (ana/empty-env) code-data))
The AST generated even from this small snippet is dauntingly large, so although you may choose to print it, we’re not going to include it verbatim in this book. Instead, we handle it gently and with respect. The AST is, as its name indicates, a tree, and each node of the tree is a persistent map. You can see the keys of the root node like this:
(keys ast) ;=> (:children :env :op :form :name :var :doc :init)
Because each node has so many keys, you can write a function, shown in the following listing, that summarizes the tree.
Listing 13.3. Definition of the print-ast function
The specific keys in each node depend on the node type, which is primarily indicated by its :op. Some other common keys are :form and :name . Every non-leaf node has a :children key, and although the components of its value are often available under more specific keys of the node, the existence of
the :children entry eases generic navigation of the tree. The leaves of the tree are left alone , and the entire tree is printed with pleasant indentation via pprint. The output of calling (print-ast ast) is as follows:
Note how each node’s :form value is the result of fully macroexpanding the outermost form at that point, whereas inner forms are left alone until deeper
into the tree. For example, defn became def at the root node , but its fn doesn’t become fn* until the next level deeper . We’ll come back to macroexpansion in the next section.
What’s important is that at this point, almost all the heavy lifting of the compilation process has been done, and most of it is unrelated to JavaScript. This AST can be used for a variety of things. Of course, it’s meant primarily to be used in emitting JavaScript:
(comp/emit ast)
; cljs.user.hello = (function hello(x){
; return alert(cljs.user.pr_str.call(null,
; new cljs.core.Symbol(null,"greetings","greetings",
; 432603411,null),x));
; });
;=> nil
It can also be used to generate other outputs or to identify patterns that can be optimized. We have another use in mind, but first we need a more substantial example.
13.2.2. Web Audio
“Ah, music,” he said, wiping his eyes. “A magic far beyond all we do here!”
Professor Albus Dumbledore[3]
3 Harry Potter and the Sorcerer’s Stone, by J. K. Rowling (Scholastic, 1999).
Web Audio is a W3C Working Draft (W3C 2013) for synthesizing real-time audio streams in the browser. Currently, only WebKit browsers like Chrome and Safari support the draft as written, but despite it being at the cutting edge of browser support, ClojureScript can use it just fine. Bear in mind that we don’t plan to cover the Web Audio API in depth.
To easily get your ClojureScript compiled and loaded into the browser, update project.clj (refer to the preface for what this file does and means) as shown in the following listing to include the cljsbuild plug-in and related configuration options.
Listing 13.4. project.clj for music
You also need a small amount of HTML to load the JavaScript that the ClojureScript compiler will produce. Put the following HTML in a file named music.html in the same directory as project.clj.
Listing 13.5. HTML contents of music.html
<!DOCTYPE html>
<html lang="en">
<head><title>Web Audio with ClojureScript</title></head>
<body>
<button onclick="joy.music.go()">Play</button>
<script src="dev-target/all.js"></script>
</body>
</html>
Now all you need is the ClojureScript that this HTML is expecting. Begin with the functions in the next listing, which use host interop to manipulate Web Audio. These functions belong in a file named src/cljs/joy/music.cljs. The src directory belongs in the project root, next to project.clj and music.html.
Listing 13.6. Web Audio functions in ClojureScript
Many of these functions take ctx and an AudioContext argument. This is an object the browser uses to track real mutable resources, so unfortunately none of these functions are
pure. The functions soft-attack and sine-tone each construct and return a specific kind of AudioNode. Each AudioNode in the Web Audio API can have inputs and outputs, and the connect-to 
function supports hooking these together conveniently. The woo function 
does exactly this, creating and connecting Audio-Nodes to produce an output that sounds something like a glass harp, lingering for a bit longer than the specified note’s duration.
One more node is created at 
: a compressor that prevents you from accidentally over-driving your speakers.
The final function in this listing, play! 
, plays an entire tune based on its two parameters. The first, note-fn, should be a function like woo that takes a single note and returns an AudioNode that plays that note at the appropriate time. The second parameter to play! is a sequence of notes, each of which is a map with keys :cent, :duration, :delay, and :volume. A cent is a number that describes the pitch of a note relative to a chromatic scale, where the distance from one note on the scale
to the next is 100 cents. The cent is used by sine-tone . The duration and delay here are given in seconds and are used in soft-attack and sine-tone. The volume is on a scale from 0 to 1 as used at . The delay indicates how long to wait from the beginning of the tune until this note is played. Thus to play a single note
that lasts a little longer than a second, you can call play! like this:
(play! woo [{:cent 1100, :duration 1, :delay 0, :volume 0.4}])
To try that, add it to the end of your music.cljs file, compile it by running lein cljsbuild once, and then open music.html in your WebKit-based web browser. When the page has finished loading, you should be greeted by a tone. Make sure your volume is turned up!
You can change the tone to an arpeggio chord like this:
(play! woo [{:cent 1100, :duration 1, :delay 0.0, :volume 0.4}
{:cent 1400, :duration 1, :delay 0.2, :volume 0.4}
{:cent 1800, :duration 1, :delay 0.4, :volume 0.4}])
But these notes aren’t as pleasant as a tune, so next let’s build up some pure functions to help construct melodies. Add the following functions to your music.cljs file.
Listing 13.7. Tune-construction functions
Starting with magical-theme near the bottom of the listing, you can see a tune described somewhat conveniently as a sequence of pairs, where each pair
is a vector of tone and duration. This time, each tone is given in hundreds of cents so you don’t have to type as many zeros,
and the duration is given in beats so you can speed up or slow down the playback later.
Although these pairs are succinct for defining a tune, they’re not the format expected by play! and thus need to be transformed. The pair-to-note function does the first part of this, returning a map for a pair, computing a cent, and setting the volume. The consecutive-notes function takes a sequence of these notes and adds a :delay field to each of them such that they will play one after another. This is no good for generating chords, but it’s handy for melodies.
Finally, notes takes the entire sequence of pairs, uses the other functions just described, and converts the beat-based duration and delay
values to seconds as needed by play!. With all these together, the go function has what it needs to call play! and play your tune.
If you compile this with lein cljsbuild once and reload music.html in your browser, the tune shouldn’t play right away. Nothing has called the go function yet. To play the tune, click the Play button on the web page. The onclick handler of that button calls the JavaScript function joy.music.go, which is generated from the go function in the joy.music ClojureScript namespace. Voilà, music!
This go function, as well as all the other ClojureScript functions you’ve written, plus all the other functions provided by ClojureScript to support the entire language runtime, are compiled into your dev-target/all.js file. Would you be surprised to learn that this file is well over 700 KB, at least on our computer? Although this may not be a lot for the runtime system of an entire language, you can do better.
13.2.3. Advanced compilation
By itself, the ClojureScript compiler generates JavaScript code that, simply speaking, is bulky. The reason for the size of the generated JavaScript code is that the ClojureScript compiler is naive (by design) about the structure and content of the code it compiles. Rather than attempt to create a compiler that was smart about the code that it produces, the ClojureScript creators decided to defer the smarts of code compression and elision to another tool developed by Google, which we’ll discuss in brief shortly.
An optional (but on by default) part of the ClojureScript build process is the Google tool called the Closure compiler (note the s)—we’ll call it GClosure for clarity. This compiler reads in JavaScript, groks it quite deeply, and writes new JavaScript back out. If you look at the dev-target/all.js file, you’ll see one style of JavaScript it can produce, as specified in project.clj: {:output-to "dev-target/all.js", :optimizations :whitespace, :pretty-print true}. In this mode, GClosure pulls in all the files it’s asked to, removes whitespace and comments, fixes the indentation, and concatenates the files into a single output file, 700 KB worth in this case.
But GClosure also has an advanced optimization mode in which functions and variables, both local and global, are renamed to shorter names and dead code is eliminated. You can try this mode by adding another item to the :builds value of project.clj:
{:source-paths ["src/cljs"]
:compiler
{:output-to "prod-target/all.js"
:optimizations :advanced
:pretty-print false}}
Now, when you run lein cljsbuild once, in addition to an updated dev-target/all.js file you’ll also get a new prod-target/all.js file. This file is about 54 KB—dramatically less than the 700 KB produced by whitespace-optimization mode. To try it, update the HTML to point at it:
<script src="prod-target/all.js"></script>
Reload the HTML in your browser, click the Play button, and ... nothing happens. If you check your JavaScript console, you may see an error like this:
Uncaught ReferenceError: joy is not defined
What a dismal world, where there is no definition of joy. This is where GClosure’s function renaming gets you into trouble. The problem is that the joy.music namespace and the go function in it have been renamed; but because GClosure never sees the HTML file, it doesn’t know that the call to joy.music.go() in the onclick handler needs to be renamed. So instead of renaming go in the HTML file, you can use export to tell GClosure to skip renaming the go function everywhere. Any time you have ClojureScript that wants to expose a specific var name to other code, you need to mark it as export, like this:
(defn ^:export go [] (play! woo (notes magical-theme)))
Now, after rebuilding and reloading, you’ll get a different error. The code trips your browser’s missing-feature detectors, and an alert pops up saying, “Sorry, this browser doesn’t seem to support AudioContext.” You know this to be a lie, because the music was working fine in the previous section. What’s actually going on is more trouble with renaming. In this case, it’s the mirror image of the problem with go. The code is attempting to use public names from the AudioContext API, but when GClosure processes the code, it renames calls to that API. To prevent this, you need to use an externs file to tell GClosure which names to leave alone. Specify the externs file to use by adding it to the :compiler section of project.clj from listing 13.4 so it looks like this:
{:output-to "prod-target/all.js"
:optimizations :advanced
:externs ["externs.js"]
:pretty-print false}
Then provide an externs.js file in your project root directory. The first thing this example tries to do is detect whether the browser has an AudioContext or webkit-AudioContext property in its window object. To make sure these aren’t renamed by GClosure, add references to them in the externs.js file:
var window = {};
window.AudioContext = function(){};
window.webkitAudioContext = function(){};
This tells GClosure about three names that it should leave alone. The first the ubiquitous JavaScript global, window. This isn’t strictly necessary because GClosure already knows not to rename references to window, but we include it for completeness. The next two lines name the properties AudioContext and webkitAudioContext. Globals and properties are the two kinds of entries GClosure understands in an externs file.
With the project.clj and externs.js files updated, you can try rebuilding and reloading the program. The cljsbuild plug-in doesn’t generally notice changes to externs files, so you need to touch the music.cljs file, perhaps by forcing your editor to write it to disk again, before rebuilding and reloading. This time the browser should get past the feature sniffing, but then it fails again this time with
Uncaught TypeError: Object #<AudioContext> has no method 'ub'
Your error may have something other than ub, but it’s no doubt just as meaningless. So you’ve found a couple names that must be in the externs file, but apparently more are needed. How can you find all the entries needed for this ClojureScript example to run? One solution would be to go through the Web Audio API documentation and hand-write an externs.js file, updating it in the future as new versions of the API come out.
Another solution would be to carefully analyze your code by hand, identify public globals and properties used, and continue to add them manually to the externs file. If the program still didn’t work, you could assume you missed one and repeat the process. But we already showed how to use the ClojureScript compiler to analyze source code, so let’s see if you can use that to automate the creation of externs.js.
13.2.4. Generating an externs.js file
In the previous example of using the ClojureScript analyzer, you were analyzing a single form. To analyze the music example, you need to read an entire file and analyze each of the forms in it. Functions for doing this are shown in the following listing.
Listing 13.8. Using analyze to generate an AST for a ClojureScript file
You already saw how large the AST is even for a small piece of code, so it’s wise to handle the return value of file-ast with care:
(count (file-ast "src/cljs/joy/music.cljs"))
;=> 11
(first (file-ast "src/cljs/joy/music.cljs"))
;=> {:requires nil, :form (ns joy.music) ...}
When finding specific uses of JavaScript interop that may need to have their names protected, you want to find all uses of interop, but you don’t really care about the nested forms in which they’re found. So you can define a function that turns the entire AST into a single flat sequence of nodes:
(defn flatten-ast [ast] (mapcat #(tree-seq :children :children %) ast)) (def flat-ast (flatten-ast (file-ast "src/cljs/joy/music.cljs"))) (count flat-ast) ;=> 473
By exploring the AST yourself, you could no doubt find the interop forms you’re seeking. There’s not currently any comprehensive documentation for the ClojureScript AST, so exploring examples and reading the compiler source code are generally the only options for gaining that kind of knowledge. Or we can tell you: interop nodes in the AST use :dot in their :opentry, and a name in either a :method or :field entry. You can find these names with a get-interop-used function:
(defn get-interop-used
"Return a set of symbols representing the method and field names
used in interop forms in the given sequence of AST nodes."
[flat-ast]
(set (keep #(some % [:method :field]) flat-ast)))
(get-interop-used flat-ast)
;=> #{noteOn value webkitAudioContext AudioContext gain
createDynamicsCompressor frequency detune linearRampToValueAtTime
destination currentTime createOscillator createGainNode connect
noteOff}
The symbols returned by get-interop-used look like names from the Web Audio API that must remain unmodified in the final JavaScript, which is encouraging. Aren’t you glad you didn’t have to find them all by squinting at code and using trial and error? Now you just need to use these to generate the appropriate lines of JavaScript to populate the externs file.
Fortunately, GClosure isn’t currently smart enough to determine the actual class used in your application’s source code at each method call or field usage, so it ignores the class name in the externs file. This means you can use a dummy name for the class when defining method and field externs:
(defn externs-for-interop [syms]
(apply str
"var DummyClass={};
"
(map #(str "DummyClass." % "=function(){};
")
syms)))
It’s time to try this and see if the music will live again. First generate the externs.js file for the music project:
(spit "externs.js" (externs-for-interop (get-interop-used flat-ast)))
As mentioned earlier, changes to externs files aren’t enough to trigger a rebuild, so you need to touch your music.cljs file. With that done, regenerate the optimized JavaScript:
lein cljsbuild once
Reload in the browser, click the Play button, and you should once again hear your little melody.
A quick check of the latest prod-target/all.js shows a working file that weighs in at just 60 KB, down from the first working dev-target/all.js file that was over 700 KB: an order of magnitude improvement. But it did take a lot of effort to get GClosure’s advanced mode to generate working code. You’ll have to decide for yourself which situations call for advanced mode and which don’t.
We showed you how to examine the AST to find the methods and fields that were used, but there is another category of names that must sometimes be protected via externs files: class names. Extracting a useful set of class names from the AST is a bit trickier because they can look to the ClojureScript compiler like namespace names instead. The mechanisms required to do this are beyond the scope of this section, but the necessary code is provided in the joyofclojure github repository.
This may be a good time to step back for a moment to note the benefits and costs generated by the ClojureScript AST being made of regular Clojure data types. One cost is that you have to put extra thought into dealing with lazy sequences when used in proximity to side effects like reading from a mutable input stream. On the other hand, you can take advantage of familiar and powerful functions like take-while, keep, and some and techniques like using hash maps as functions to succinctly compute the results you want. Contrast this with the work that would be required if each node of the AST were a different class with its data tucked behind type-specific accessors. ClojureScript provides interesting options by giving you access to the workings of the compiler and providing an AST made up of normal data.
All this analysis, emission, and GClosure work is done at compile time. Next we’ll look at the separation of compile time from runtime.
13.3. Compile vs. run
LISP programmers know the value of everything and the cost of nothing.
Alan Perlis[4]
4 “Epigrams on Programming,” SIGPLAN Notices 17, no. 9 (1982): 7-13.
One of the defining characteristics of all but the most minimal LISPs is that you can execute arbitrary code at compile time via macros. In Clojure, this blurs and interleaves the lines between runtime and compile environments. The compiler evaluates user functions, which may in turn be defined in terms of macros. The runtime environment provides eval, which in turn invokes the compiler. ClojureScript draws sharper lines between these pieces, resulting in interesting implications.
A runtime without eval or a compiler has no need for all the mechanics of analysis, optimization, and emission. Part of analysis is macroexpansion, so that isn’t needed either. Without macroexpansion, there is no need for the runtime environment to even be aware of the concept of macros, let alone specific macros and the namespaces where they live. Without macros, there is no need for any functions that are used exclusively by those macros.
You can see one aspect of this separation in ClojureScript’s ns operator, which has two different requires directives: :requires and :require-macros. The former declares which namespaces must be made available at runtime in order for the current namespace to run, and the latter declares which namespaces must be made available at compile time in order for the current namespace to compile.
The separation is solidified even more by the differences between the JVM and JavaScript. Everything ClojureScript does at compile time, including macroexpansion and calling the functions used by macros, uses Clojure and runs in the JVM; whereas everything named by :requires is compiled to JavaScript and is run in the browser or other JavaScript environment.
This can be easily explored using the earlier music example. Although the Web Audio functions soft-attack, sine-tone, and so on (shown in listing 13.6) use the Web Audio API and thus must run in the browser, the functions pair-to-note, consecutive-notes, and notes (listing 13.7) purely manipulate Clojure data and thus could be run at compile time so the browser would see only the results, not the functions for computing them. To do this, you need to port these functions from ClojureScript to Clojure and make your project aware of them. This requires a few steps:
1. Add a line to project.clj to add a new Clojure source directory: :source-paths ["src/clj"].
2. Copy the functions from listing 13.7 into a new file, src/clj/joy/macro_tunes.clj.
3. Add a header to the top of the file: (ns joy.macro-tunes).
4. Add a macro definition to the bottom of the file:
(defmacro magical-theme-macro [] (vec (magical-theme))).
At this point the port is complete except for a couple of bugs we’ll track down later, but the existing ClojureScript code isn’t using this new macro. Fix that next:
1. In src/cljs/joy/music.cljs, add a :require-macros line so your header looks like this: (ns joy.music (:require-macros [joy.macro-tunes :as mtunes])).
2. Make the go function use the new macro by replacing the call to (magical-theme) with one to (mtunes/magical-theme-macro).
You’re now in a position to try the new arrangement of the code. But a couple of bugs are lurking—code that was correct ClojureScript but is incorrect Clojure. Over the next few paragraphs, we’ll show the error messages, explain the differences between Clojure and ClojureScript that cause the messages, and describe how to fix these issues.
The first problem can be discovered by running lein cljsbuild once, which generates a rather inscrutable message having to do with a null pointer. The mistake is in the consecutive-notes function, in listing 13.7, where you call reductions with only two arguments, causing it to seed the parameters to the anonymous function with a nil. The function destructures, assigning nil to both delay and duration. These are then added together. This was happening all along in ClojureScript, but that meant it was JavaScript’s semantics for addition that were in play; and in JavaScript, adding null to null is perfectly legal and returns zero such that subsequent additions work fine.
But this code is now running at compile time in Clojure, where adding nils is a throwable offense. Fix this by adding a seed value to the reductions call, shown in the following listing.
Listing 13.9. consecutive-notes function fixed to work in Clojure
You may need to touch or resave the music.cljs file to make cljsbuild recognize that anything needs to be rebuilt. Another attempt to build then uncovers the next (and final) bug in the Clojure port:
No method in multimethod 'emit-constant' for dispatch value: class clojure.lang.Ratio
This is because you’re doing division on integers. Because JavaScript only has floating-point numbers, dividing any two things results in a float. But in Clojure, dividing two integers returns a ratio. Note the subtle but real difference between the environment in which the compiler and macros are running and the one in which the compiled code runs. ClojureScript doesn’t know how to take the ratios your macros create and emit them as JavaScript, so you have to help it. In notes of listing 13.7, replace the last two uses of / with (comp double /) so that it looks as shown next.
Listing 13.10. notes function fixed to work in Clojure
Touch music.cljs, and your next rebuild should produce clean results. Reload the browser, and the Play button should work once again. The runtime behavior is identical to what it was before you ported half the code to Clojure, but the compiled results are different in an important way. If you compare prod-target/all.js before and after the port to Clojure, you’ll see that where there used to be JavaScript literals for vector pairs and functions for processing them, now there is instead JavaScript that expresses the sequence of note maps directly. There could be use cases for doing this kind of port in order to reduce the size of the final .js file or to hide proprietary algorithmic code from being sent to browsers over the network.
But although there may be such use cases, our main point here is more general. ClojureScript makes the distinction between compile time and runtime a good deal sharper than it is in Clojure. But they’re distinct even in Clojure. One kind of distinction is demonstrated by how you use build tools like Leiningen and Maven to resolve dependencies at compile time. Although an uber jar[5] built by such tools has all the current dependencies and can even eval new Clojure code at runtime, there is no guarantee the runtime system will be able to add new dependencies the way you do all the time during development. The systems where you develop and build generally have access to servers that provide dependencies for download, but the systems running the deployed application may not. Hopefully the fact that ClojureScript’s build and runtime environments are so clearly distinct can help you think more clearly about the distinctions in Clojure.
5 A jar file containing all of an application’s relevant class files, resources, and dependencies.
ClojureScript’s clearer separation has other practical benefits. You could add a top-level call to (go) at the end of music.cljs without any fear that the ClojureScript compiler would attempt to play music at compile time. It wouldn’t make any sense, because no Web Audio API is available in the JVM where the compiler is executing. Yet the compiled JavaScript would have the equivalent call so that as soon as the all.js file was done loading in the browser, the tune would start playing. Contrast this with Clojure, where each top-level form is evaluated immediately after it’s compiled, even when ahead of time (AOT) compilation is being used. This is sometimes described as the compiler “launching missiles,” although in our example it would be more likely to launch music.
Another benefit is that separation of the sort ClojureScript provides allows for cross-compilation without an emulator. You’ve been doing this all along in this chapter, compiling to JavaScript but only needing a JavaScript interpreter to test the results. Imagine if you had this ability for compiling to other runtimes such as the CLR. Right now, in order to compile a Clojure project for CLR, you either need to use ClojureCLR, where the compiler itself is running the CLR runtime, or you need to use a JVM emulator running inside the CLR.
These circumstances are inherent to the behavior promised by Clojure, where each top level form is evaluated immediately upon compilation and the very next top-level form can use the results. As is often the case, a layer of software restricts its flexibility by promising specific behavior to the next layer. By changing this promise, ClojureScript brings within reach interesting possibilities such as the ones we’ve just discussed.
This separation is similar to that of many classic compiled languages, where cross-compilers are common and REPLs aren’t. The most obvious cost of this approach has already been mentioned: no eval at runtime. This has direct consequences on tools like the REPL. A Clojure REPL is easy to write because each of the parts named happens in the same environment. A ClojureScript REPL can do the read, print, and loop parts in the JVM, but eval needs special treatment. The easiest solution is to use a JavaScript interpreter like Rhino, which runs inside the JVM. You can experiment with this easily by running
lein trampoline cljsbuild repl-rhino
This is often insufficient, because a runtime environment is more than just the language interpreter. Rhino provides neither a DOM nor a Web Audio API. To address this, the cljsbuild tool also provides a couple ways of setting up communication between the compilation environment in the JVM and the runtime environment in a browser. But however you solve this, it’s unlikely to ever be as convenient as sharing exactly the same runtime as Clojure.
Another cost of separating the environments can be seen in the amount of code you had to move when moving some of the music example to compile time. In order for magical-theme-macro to work without failure, all the functions it used had to be explicitly defined at compile time by moving them into a .clj file. Contrast this with Clojure, where it’s easy to define a macro that uses functions at compile time that were previously only used at runtime, without having to change those functions in any way. It’s a significant benefit that despite being technically separate, the compile-time environment is similar to the runtime, because this allows many functions to be ported by copying. It’s even likely that in the future there will be an officially supported way to make a single namespace available to both the runtime and compile-time environments.
13.4. Summary
ClojureScript’s design is another example of how teasing apart things that were previously tangled together results in powerful new capabilities. Using protocols to define the abstractions provided by the language and opening access to the analysis phase both provide additional power with little or no cost. In fact, you were able to use the AST to construct the externs file needed to use advanced Closure compilation in the Web Audio example. Finally, separating runtime more fully from compile time provides benefits that are critical especially for JavaScript, and ClojureScript’s tooling goes a long way to mitigate the costs that incurs.
You’ve seen several examples already of treating code as data, such as when looking at the stages of compilation in this chapter. We’ll explore this topic more thoroughly in the next chapter.