The most comprehensive automation technique is to remotely control a web browser. Remote control means that a program does what a human can do using a real web browser—opening the browser to a given page, clicking on links, filling in inputs, and so on.

One of the most popular remote control frameworks is Selenium (http://docs.seleniumhq.org/). Selenium provides many web drivers, which are programmatic adapters that hook into a real web browser and trigger actions through the normal user interface. Selenium supports a diverse array of environments, providing web drivers on different operating systems for various browsers, including Chrome, Safari, Firefox, and Internet Explorer.

Automating our test infrastructure with a remote control tool such as Selenium involves two basic steps: open and run the test driver page and then, infer whether or not the tests have passed. As an example, we could write a Selenium script that opens a browser window to the Notes application test driver page notes/test/test.html in the code samples. The Selenium script could then scrape the report page HTML to check for a telltale string such as failures: 0 in the DOM and terminate the script with an appropriate success/failure exit code.

Thus, remote-controlled tools such as Selenium are quite powerful because they can do anything a real browser can do, just automatically. And, with a cross-platform compatible tool such as Selenium, we can run tests on nearly all modern browser/operating system combinations from a single script.

The remote controlled approach does have a few downsides, the first of which is that the test tools can be relatively slow. Scripts can take seconds or even minutes to hook into a target browser and run a test driver page. Additionally, these frameworks require a real web browser and a desktop windowing system. This can be an issue for build/continuous integrations that are headless, meaning they have no graphical user interface or window environment installed by default.

An alternative automation approach is to replace the web browser with a test-friendly simulation of the browser environment and state. Typically, browser simulation libraries provide implementations of the JavaScript API within a browser such as DOM objects (for example, window and document), CSS selectors, and JSON interfaces.

JSDom (https://github.com/tmpvar/jsdom) is a prevalent simulation library that provides a fairly complete browser environment. JSDom is written in JavaScript and packaged as a Node.js module. Because Node.js can be easily scripted, JSDom offers us a good starting point for integrating and running Backbone.js tests from the command line.

Test automation is such a common use case that several test-friendly libraries have been written around JSDom. One such library is Zombie.js (http://zombie.labnotes.org/), which provides convenient browser abstractions and integration with various test frameworks, including Mocha. Using a library such as Zombie.js, we could write a Node.js script that creates a fake browser simulation, navigates to our Backbone.js test driver page, and scrapes the test result HTML to check if any tests failed. For a more in-depth treatment of testing JavaScript web applications with Zombie.js and Mocha, see Using Node.js for UI Testing by Pedro Teixeira (http://www.packtpub.com/testing-nodejs-web-uis/book).

Browser simulation libraries are fast because they run simulation code in the same underlying JavaScript engine as the test code without external dependencies (for example, on a real web browser executable). Simulation libraries are often quite extensible, as the simulation JavaScript code runs in the same process as the application and the tests.

However, simulations suffer from a few key drawbacks. One primary issue is that simulations can deviate from the true environment in a real web browser. Complicated browser interactions such as heavily chained event triggers or complex DOM manipulations can potentially break the simulation or behave differently than a real browser. Additionally, a browser simulation library provides only a single browser environment implementation and thus cannot test the quirks and differences across various real web browsers.

Between remote controlled browsers and simulation libraries are headless web browsers. A headless browser takes a real web browser and gets rid of the user interface, leaving only the JavaScript engine and environment. What remains is a command line tool that can navigate to web pages, execute JavaScript within the browser environment, and communicate through non-graphical interfaces such as alerts and console logging.

One of the most popular headless toolkits is PhantomJS (http://phantomjs.org/), which is based on the WebKit open source browser (http://www.webkit.org/) that powers browsers such as Safari. PhantomJS enhances WebKit with scripting support and a JavaScript API.

Integrating Backbone.js application tests with a headless browser is analogous to configuring a remote-controlled browser. Conveniently, PhantomJS ships with native support for a wide array of test infrastructures and offers third-party adapters for many others. See https://github.com/ariya/phantomjs/wiki/Headless-Testing for more test support details.

Headless web tools have a mix of some of the best features of the previous automation approaches, including the following:

At the same time, headless browsers incur some performance penalties from starting up and running the browser engine. They also forgo cross-browser capabilities, because headless tools are tied to a specific web browser engine implementation. Considering the overall advantages and disadvantages, headless frameworks provide a good compromise between the many mutually exclusive automation features.

Capitalizing on the benefits of various approaches, many frameworks aggregate different automation schemes into a single package. For example, the following test frameworks can programmatically drive tests in major web browsers and PhantomJS:

Aggregation frameworks are desirable because they allow a single test collection to be reused in different automation environments, although some tools are more difficult to set up and maintain than a single automation tool.