Running Jasmine tests manually by running SpecRunner.html on every browser can be a time-consuming process; this is why automating Jasmine tests is important. In order to automate Jasmine tests, we can use Karma (http://karma-runner.github.io).
Karma is one of the best modern JavaScript test runners that can be used to automate JavaScript tests. Karma is based on Node.js and is distributed as a node package. Karma provides an easy-to-use command-line interface that we will illustrate in detail in the following sections.
Karma includes a web server that can capture one or more browser(s), execute JavaScript tests on the captured browsers, and finally, report the test results of every browser in the command-line interface. In order to capture a browser in Karma, you can execute one of the following two methods:
- Make the browser(s) that you want to capture visiting Karma server URL (usually, it is
http://${karma_server_ip}:9876/). - In the configuration file, you can specify the browser(s) to launch automatically when the Karma server starts (check the Karma Configuration section). Doing this configuration will save a lot of time spent on executing your JavaScript tests manually on the different browsers.
The following sections will show you how we will use Karma with Jasmine in detail.
In order to work with Karma, you need to make sure that you have Node.js installed in your operating system. In order to install Node.js in Windows and Mac, you can download their installers from http://nodejs.org/download/; for Linux, you can use the Node Version Manager (NVM) from https://github.com/creationix/nvm. Currently, Karma works perfectly with the latest stable versions of Node.js (0.8.x and 0.10.x).
It is recommended that you install Karma and all of its plugins that our project needs in the project directory. In order to install Karma in our project, execute the following command directly under the www directory of our weather project:
> npm install karma --save-dev
Then, we can install karma-Jasmine 2.0 (to run the Jasmine 2.0 code over Karma) and karma-chrome-launcher (to launch the Chrome browser automatically when requested by the Karma configuration) plugins from the command-line interface as follows:
> npm install karma-jasmine@2_0 karma-chrome-launcher --save-dev
In order to avoid typing the full path of Karma every time you execute a karma command, it is recommended that you install Karma CLI globally by executing the following command:
> npm install -g karma-cli
We can generate the initial Karma configuration using CLI by executing the following command and answering the Karma configuration questions:
> karma init config.js Which testing framework do you want to use? Press tab to list possible options. Enter to move to the next question. > jasmine Do you want to use Require.js? This will add Require.js plugin. Press tab to list possible options. Enter to move to the next question. > no Do you want to capture any browsers automatically? Press tab to list possible options. Enter empty string to move to the next question. > Chrome > What is the location of your source and test files? You can use glob patterns, eg. "js/*.js" or "test/**/*Spec.js". Enter empty string to move to the next question. >jqueryMobile/jquery-1.10.2.min.js >js/api/*.js > tests/spec/*.js > Should any of the files included by the previous patterns be excluded ? You can use glob patterns, eg. "**/*.swp". Enter empty string to move to the next question. > Do you want Karma to watch all the files and run the tests on change ? Press tab to list possible options. > yes Config file generated at "${somePath}/config.js".
The following code snippet shows the generated Karma config.js file:
module.exports = function(config) {
config.set({
basePath: '',
frameworks: ['jasmine'],
files: [
'jqueryMobile/jquery-1.10.2.min.js',
'js/api/*.js',
'tests/spec/*.js'
],
exclude: [
],
preprocessors: {
},
reporters: ['progress'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: true,
browsers: ['Chrome'],
singleRun: false
});
};The following table explains the meaning of the generated configuration attributes briefly:
In order to start running our developed Jasmine test using Karma, we can start the Karma server specifying our configuration file as a parameter as follows:
> karma start config.js
This will automatically start a Chrome browser instance and execute our developed Jasmine tests on it. Finally, you will find output results like the following in the console:
INFO [karma]: Karma v0.12.19 server started at http://localhost:9876/ INFO [launcher]: Starting browser Chrome INFO [Chrome 36.0.1985 (Mac OS X 10.9.2)]: Connected on socket PF7lhJWBohNMJqOlUdnP with id 78567722 ... Some test results information here... Executed 6 of 6 SUCCESS (1.575 secs / 1.57 secs)
If we want Karma to test our Jasmine code on more browsers, we can simply:
- Specify more browsers in the
browsersattribute of the configuration file - Install the browser launcher plugin in our
appdirectory
For example, if we want to test our Jasmine code in Firefox, we will do the following:
- In the
config.jsfile, add'Firefox'to the browser's attribute as follows:browsers: ['Chrome', 'Firefox'] - Install the Firefox launcher plugin in our
appdirectory as follows:> npm install karma-firefox-launcher --save-dev - Run the
karma start config.jscommand again to see the test results in the CLI for both Chrome and Firefox.
In order to run our tests on a mobile browser, we can make the mobile browser visit the URL of the Karma server. The following screenshot shows the tests run on an Android mobile browser.
Running the tests on an Android mobile browser
The following are the test results in the Android browser that are displayed in the console:
INFO [Android 4.1.2 (Android 4.1.2)]: Connected on socket Rv85bR0dfNpt5S8ecBnc with id manual-7523 ... Some test results information here ... Android 4.1.2 (Android 4.1.2): Executed 6 of 6 SUCCESS (1.094 secs / 1.02 secs)
Karma, by default, outputs test results in the console. In order to output the test results in an XML JUnit report, we need to use the Karma JUnit reporter plugin (https://github.com/karma-runner/karma-junit-reporter).
In order to install Karma JUnit reporter plugin, execute the following command:
> npm install karma-junit-reporter --save-dev
Then, we need to add the plugin configuration in our config.js file, as highlighted in the following code snippet:
module.exports = function(config) {
config.set({
reporters: ['progress', 'junit'],
// The default configuration
junitReporter: {
outputFile: 'test-results.xml',
suite: ''
}
});
};This means that the JUnit reporter will output the test results in the 'test-results.xml' file. In order to see the XML JUnit report, execute karma start config.js again, and you will find the XML JUnit report, as shown in the following code:
<?xml version="1.0"?>
<testsuites>
...
<testsuite name="Chrome 36.0.1985 (Mac OS X 10.9.2)" package="" timestamp="2014-08-11T10:42:53" id="0" hostname="IBMs-MacBook-Pro-2.local" tests="6" errors="0" failures="0" time="1.017">
<properties>
<property name="browser.fullName" value="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36"/>
</properties>
<testcase name="should NOT be able to save a user with an empty user name" time="0.004" classname="Chrome 36.0.1985 (Mac OS X 10.9.2).UserService"/>
<testcase name="should NOT be able to save a user with invalid email" time="0.001" classname="Chrome 36.0.1985 (Mac OS X 10.9.2).UserService"/>
<testcase name="should NOT be able to save a user with a user name more than 6 characters" time="0" classname="Chrome 36.0.1985 (Mac OS X 10.9.2).UserService"/>
<testcase name="should be able to save and load a valid user" time="0.002" classname="Chrome 36.0.1985 (Mac OS X 10.9.2).UserService"/>
<testcase name="should be able to get weather information for a valid place" time="0.419" classname="Chrome 36.0.1985 (Mac OS X 10.9.2).WeatherService"/>
<testcase name="should NOT be able to get weather information for an invalid place" time="0.591" classname="Chrome 36.0.1985 (Mac OS X 10.9.2).WeatherService"/>
<system-out> ... </system-out>
<system-err/>
</testsuite>
</testsuites>Adding to JUnit XML reports, Karma can generate code coverage reports. In order to generate code coverage using Karma, we can use the Karma coverage plugin (https://github.com/karma-runner/karma-coverage).
In order to install the Karma coverage plugin, execute the following command:
> npm install karma-coverage --save-dev
Then, we need to add the plugin configuration in our config.js file, as highlighted in the following code snippet:
module.exports = function(config) {
config.set({
// ...
reporters: ['progress', 'coverage'],
preprocessors: {
'js/api/*.js': ['coverage']
},
coverageReporter: {
type : 'html',
dir : 'coverage/'
}
});
};This previous configuration means that the Karma code coverage plugin will generate code coverage report(s) for our JavaScript source files, 'js/api/*.js', and output the code coverage results under the 'coverage' directory in an HTML format. Note that every browser will have its own directory under the 'coverage' directory, including its code coverage report. The following screenshot shows the code coverage report for the Chrome browser:
Code coverage report
As shown in the preceding screenshot, the code coverage is generated for code statements, branches, and functions.
Having the ability to execute Karma tests from the command-line interface allows us to fully automate running Karma tests using CI tools. In order to integrate Karma tests with Jenkins (a popular CI tool), we need to perform the following steps in the Jenkins project configuration:
- Select Execute Shell from Add build step (or Execute Windows batch command if you are using Windows), then specify the location of the shell file that starts the Karma server and run the tests. The shell file can have the following commands:
#!/bin/bash cd weather/www export PATH=$PATH:/usr/local/bin karma start config.js --single-run --browsers PhantomJS
The previous shell script code starts the Karma server in a single run mode and specifies PhantomJS (http://phantomjs.org) as the browser that will execute JS tests. PhantomJS is a very light-weight, headless browser that can be a good choice for CI environments. In order to work with PhantomJS in Karma, we have to install its launcher plugin from CLI as follows:
> npm install karma-phantomjs-launcher --save-dev - Select Publish JUnit test result report from Post-build Actions, and in test report XML, specify the path of the XML JUnit report,
'weather/www/test-results.xml'. Jenkins fortunately recognizes the JUnit XML format, which we generate in our JavaScript tests.After making these main changes and building the Jenkins CI project, we will find the JavaScript test results shown in the project dashboard after some builds have been done, as shown in the following screenshot:
JavaScript test results in the Jenkins project dashboard
Note
You can get the complete source code of the weather application with its JavaScript tests from the book page or from GitHub using https://github.com/hazems/cordova-js-unit-testing.