Chapter 23
Offline Applications and Client-Side Storage
WHAT’S IN THIS CHAPTER?
- Setting up offline detection
- Using the offline cache
- Storing data in the browser
One of HTML5’s focus areas is enabling offline web applications. An offline web application still works even when there is no Internet connection available to the device. This focus is based on the desire of web application developers to better compete with traditional client applications that may be used so long as the device has power.
For web applications, creating an offline experience requires several steps. The first step is to ensure that the application knows whether an Internet connection is available or not in order to perform the correct operation. Then, the application still needs to have access to a subset of resources (images, JavaScript, CSS, and so on) in order to continue working properly. The last piece is a local data storage area that can be written to and read from regardless of Internet availability. HTML5 and other associated JavaScript APIs make offline applications a reality.
Since the first step for offline applications is to know whether or not the device is offline, HTML5 defines a navigator.onLine property that is true when an Internet connection is available or false when it’s not. The idea is that the browser should be aware if the network is available or not and return an appropriate indicator. In practice, navigator.onLine is a bit quirky across browsers:
- Internet Explorer 6+ and Safari 5+ correctly detect that the network connection has been lost and switch navigator.onLine to false.
- Firefox 3+ and Opera 10.6+ support navigator.onLine, but you must manually set the browser to work in offline mode via the File Work Offline menu item.
- Chrome through version 11 permanently has navigator.onLine set to true. There is an open bug to fix this.
Given these compatibility issues, navigator.onLine cannot be used as the sole determinant for network connectivity. Even so, it is useful in case errors do occur during requests. You can check the status as follows:
if (navigator.onLine){
//work as usual
} else {
//perform offline behavior
}
OnLineExample01.htm
Along with navigator.onLine, HTML5 defines two events to better track when the network is available or not: online and offline. Each event is fired as the network status changes from online to offline or vice versa, respectively. These events fire on the window object:
EventUtil.addHandler(window, "online", function(){
alert("Online");
});
EventUtil.addHandler(window, "offline", function(){
alert("Offline");
});
OnlineEventsExample01.htm
For determining if an application is offline, it’s best to start by looking at navigator.onLine to get the initial state when the page is loaded. After that, it’s best to use the events to determine when network connectivity changes. The navigator.onLine property also changes as the events fire, but you would need to manually poll for changes to this property to detect a network change.
Offline detection is supported in Internet Explorer 6+ (navigator.onLine only), Firefox 3, Safari 4, Opera 10.6, Chrome, Safari for iOS 3.2, and WebKit for Android.
The HTML5 application cache, or appcache for short, is designed specifically for use with offline web applications. The appcache is a cache area separate from the normal browser cache. You specify what should be stored in the page’s appcache by providing a manifest file listing the resources to download and cache. Here’s a simple manifest file:
CACHE MANIFEST #Comment file.js file.css
In its simplest form, a manifest file lists out the resources to be downloaded so that they are available offline.
There are a lot of options for setting up this file, and these are beyond the scope of this book. Please see http://html5doctor.com/go-offline-with-application-cache/ for a full description of the options.
The manifest file is associated with a page by specifying its path in the manifest attribute of <html>, for example:
<html manifest="/offline.manifest">
This code indicates that /offline.manifest contains the manifest file. The file must be served with a content type of text/cache-manifest to be used.
While the appcache is primarily a way for designated resources to be cached for offline use, it does have a JavaScript API that allows you to determine what the appcache is doing. The primary object for this is applicationCache. This object has one property, status, which indicates the current status of the appcache as one of the following constants:
- 0 for uncached, meaning that there is no appcache associated with the page.
- 1 for idle, meaning the appcache is not being updated.
- 2 for checking, meaning the appcache manifest file is being downloaded and checked for updates.
- 3 for downloading, meaning the appcache is downloading resources specified in the manifest file.
- 4 for update ready, meaning that the appcache was updated with new resources and all resources are downloaded and may be put into use via swapCache().
- 5 for obsolete, meaning the appcache manifest file is no longer available and so the appcache is no longer valid for the page.
There are also a large number of events associated with the appcache to indicate when its status has changed. The events are:
- checking — Fires when the browser begins looking for an update to the appcache.
- error — Fires if there’s an error at any point in the checking or downloading sequence.
- noupdate — Fires after checking if the appcache manifest hasn’t changed.
- downloading — Fires when the appcache resources begin downloading.
- progress — Fires repeatedly as files are being downloaded into the appcache.
- updateready — Fires when a new version of the page’s appcache has been downloaded and is ready for use with swapCache().
- cached — Fires when the appcache is complete and ready for use.
These events fire generally in this order when the page is loaded. You can also trigger the appcache to go through this sequence of checking for updates again by calling update():
applicationCache.update();
Once update() is called, the appcache goes to check if the manifest file is updated (fires checking) and then proceeds as if the page had just been loaded. If the cached event fires, it means the new appcache contents are ready for use and no further action is necessary. If the updateready event fires, that means a new version of the appcache is available and you need to enable it using swapCache():
EventUtil.addHandler(applicationCache, "updateready", function(){
applicationCache.swapCache();
});
The HTML5 appcache is supported in Firefox 3+, Safari 4+, Opera 10.6, Chrome, Safari for iOS 3.2+, and WebKit for Android. Firefox through version 4 throws an error when swapCache() is called.
Along with the emergence of web applications came a call for the ability to store user information directly on the client. The idea is logical: information pertaining to a specific user should live on that user’s machine. Whether that is login information, preferences, or other data, web application providers found themselves searching for ways to store data on the client. The first solution to this problem came in the form of cookies, a creation of the old Netscape Communications Corporation and described in a specification titled Persistent Client State: HTTP Cookies (still available at http://curl.haxx.se/rfc/cookie_spec.html). Today, cookies are just one option available for storing data on the client.
Cookies
HTTP cookies, commonly just called cookies, were originally intended to store session information on the client. The specification called for the server to send a Set-Cookie HTTP header containing session information as part of any response to an HTTP request. For instance, the headers of a server response may look like this:
HTTP/1.1 200 OK Content-type: text/html Set-Cookie: name=value Other-header: other-header-value
This HTTP response sets a cookie with the name of "name" and a value of "value". Both the name and the value are URL-encoded when sent. Browsers store such session information and send it back to the server via the Cookie HTTP header for every request after that point, such as the following:
GET /index.html HTTP/1.1 Cookie: name=value Other-header: other-header-value
This extra information being sent back to the server can be used to uniquely identify the client from which the request was sent.
Restrictions
Cookies are, by nature, tied to a specific domain. When a cookie is set, it is sent along with requests to the same domain from which it was created. This restriction ensures that information stored in cookies is available only to approved recipients and cannot be accessed by other domains.
Since cookies are stored on the client computer, restrictions have been put in place to ensure that cookies can’t be used maliciously and that they won’t take up too much disk space. The total number of cookies per domain is limited, although it varies from browser to browser. For example:
- Internet Explorer 6 and lower enforced a limit of 20 cookies per domain.
- Internet Explorer 7 and later have a limit of 50 cookies per domain. Internet Explorer 7 initially shipped with support for a maximum of 20 cookies per domain, but that was later updated with a patch from Microsoft.
- Firefox limits cookies to 50 per domain.
- Opera limits cookies to 30 per domain.
- Safari and Chrome have no hard limit on the number of cookies per domain.
When cookies are set above the per-domain limit, the browser starts to eliminate previously set cookies. Internet Explorer and Opera begin by removing the least recently used (LRU) cookie to allow space for the newly set cookie. Firefox seemingly randomly decides which cookies to eliminate, so it’s very important to mind the cookie limit to avoid unintended consequences.
There are also limitations as to the size of cookies in browsers. Most browsers have a byte-count limit of around 4096 bytes, give or take a byte. For best cross-browser compatibility, it’s best to keep the total cookie size to 4095 bytes or less. The size limit applies to all cookies for a domain, not per cookie.
If you attempt to create a cookie that exceeds the maximum cookie size, the cookie is silently dropped. Note that one character typically takes one byte, unless you’re using multibyte characters.
Cookie Parts
Cookies are made up of the following pieces of information stored by the browser:
- Name — A unique name to identify the cookie. Cookie names are case-insensitive, so myCookie and MyCookie are considered to be the same. In practice, however, it’s always best to treat the cookie names as case-sensitive because some server software may treat them as such. The cookie name must be URL-encoded.
- Value — The string value stored in the cookie. This value must also be URL-encoded.
- Domain — The domain for which the cookie is valid. All requests sent from a resource at this domain will include the cookie information. This value can include a subdomain (such as www.wrox.com) or exclude it (such as .wrox.com, which is valid for all subdomains of wrox.com). If not explicitly set, the domain is assumed to be the one from which the cookie was set.
- Path — The path within the specified domain for which the cookie should be sent to the server. For example, you can specify that the cookie be accessible only from http://www.wrox.com/books/ so pages at http://www.wrox.com won’t send the cookie information, even though the request comes from the same domain.
- Expiration — A time stamp indicating when the cookie should be deleted (that is, when it should stop being sent to the server). By default, all cookies are deleted when the browser session ends; however, it is possible to set another time for the deletion. This value is set as a date in GMT format (Wdy, DD-Mon-YYYY HH:MM:SS GMT) and specifies an exact time when the cookie should be deleted. Because of this, a cookie can remain on a user’s machine even after the browser is closed. Cookies can be deleted immediately by setting an expiration date that has already occurred.
- Secure flag — When specified, the cookie information is sent to the server only if an SSL connection is used. For instance, requests to https://www.wrox.com should send cookie information, whereas requests to http://www.wrox.com should not.
Each piece of information is specified as part of the Set-Cookie header using a semicolon-space combination to separate each section, as shown in the following example:
HTTP/1.1 200 OK
Content-type: text/html
Set-Cookie: name=value; expires=Mon, 22-Jan-07 07:10:24 GMT; domain=.wrox.com
Other-header: other-header-value
This header specifies a cookie called "name" that expires on Monday, January 22, 2007, at 7:10:24 GMT and is valid for www.wrox.com and any other subdomains of wrox.com such as p2p.wrox.com.
The secure flag is the only part of a cookie that is not a name-value pair; the word "secure" is simply included. Consider the following example:
HTTP/1.1 200 OK
Content-type: text/html
Set-Cookie: name=value; domain=.wrox.com; path=/; secure
Other-header: other-header-value
Here, a cookie is created that is valid for all subdomains of wrox.com and all pages on that domain (as specified by the path argument). This cookie can be transmitted only over an SSL connection because the secure flag is included.
It’s important to note that the domain, path, expiration date, and secure flag are indications to the browser as to when the cookie should be sent with a request. These arguments are not actually sent as part of the cookie information to the server; only the name-value pairs are sent.
Cookies in JavaScript
Dealing with cookies in JavaScript is a little complicated because of a notoriously poor interface, the BOM’s document.cookie property. This property is unique in that it behaves very differently depending on how it is used. When used to retrieve the property value, document.cookie returns a string of all cookies available to the page (based on the domain, path, expiration, and security settings of the cookies) as a series of name-value pairs separated by semicolons, as in the following example:
name1=value1;name2=value2;name3=value3
All of the names and values are URL-encoded and so must be decoded via decodeURIComponent().
When used to set a value, the document.cookie property can be set to a new cookie string. That cookie string is interpreted and added to the existing set of cookies. Setting document.cookie does not overwrite any cookies unless the name of the cookie being set is already in use. The format to set a cookie is as follows, which is the same format used by the Set-Cookie header:
name=value; expires=expiration_time; path=domain_path; domain=domain_name; secure
Of these parameters, only the cookie’s name and value are required. Here’s a simple example:
document.cookie = "name=Nicholas";
This code creates a session cookie called "name" that has a value of "Nicholas". This cookie will be sent every time the client makes a request to the server; it will be deleted when the browser is closed. Although this will work, as there are no characters that need to be encoded in either the name or the value, it’s a best practice to always use encodeURIComponent() when setting a cookie, as shown in the following example:
document.cookie = encodeURIComponent("name") + "=" +
encodeURIComponent("Nicholas");
To specify additional information about the created cookie, just append it to the string in the same format as the Set-Cookie header, like this:
document.cookie = encodeURIComponent("name") + "=" +
encodeURIComponent("Nicholas") + "; domain=.wrox.com; path=/";
Since the reading and writing of cookies in JavaScript isn’t very straightforward, functions are often used to simplify cookie functionality. There are three basic cookie operations: reading, writing, and deleting. These are all represented in the CookieUtil object as follows:
var CookieUtil = {
get: function (name){
var cookieName = encodeURIComponent(name) + "=",
cookieStart = document.cookie.indexOf(cookieName),
cookieValue = null;
if (cookieStart > -1){
var cookieEnd = document.cookie.indexOf(";", cookieStart);
if (cookieEnd == -1){
cookieEnd = document.cookie.length;
}
cookieValue = decodeURIComponent(document.cookie.substring(cookieStart
+ cookieName.length, cookieEnd));
}
return cookieValue;
},
set: function (name, value, expires, path, domain, secure) {
var cookieText = encodeURIComponent(name) + "=" +
encodeURIComponent(value);
if (expires instanceof Date) {
cookieText += "; expires=" + expires.toGMTString();
}
if (path) {
cookieText += "; path=" + path;
}
if (domain) {
cookieText += "; domain=" + domain;
}
if (secure) {
cookieText += "; secure";
}
document.cookie = cookieText;
},
unset: function (name, path, domain, secure){
this.set(name, "", new Date(0), path, domain, secure);
}
};
CookieUtil.js
The CookieUtil.get() method retrieves the value of a cookie with the given name. To do so, it looks for the occurrence of the cookie name followed by an equal sign in document.cookie. If that pattern is found, then indexOf() is used to find the next semicolon after that location (which indicates the end of the cookie). If the semicolon isn’t found, this means that the cookie is the last one in the string, so the entire rest of the string should be considered the cookie value. This value is decoded using decodeURIComponent() and returned. In the case where the cookie isn’t found, null is returned.
The CookieUtil.set() method sets a cookie on the page and accepts several arguments: the name of the cookie, the value of the cookie, an optional Date object indicating when the cookie should be deleted, an optional URL path for the cookie, an optional domain for the cookie, and an optional Boolean value indicating if the secure flag should be added. The arguments are in the order in which they are most frequently used, and only the first two are required. Inside the method, the name and value are URL-encoded using encodeURIComponent(), and then the other options are checked. If the expires argument is a Date object, then an expires option is added using the Date object’s toGMTString() method to format the date correctly. The rest of the method simply builds up the cookie string and sets it to document.cookie.
There is no direct way to remove existing cookies. Instead, you need to set the cookie again — with the same path, domain, and secure options — and set its expiration date to some time in the past. The CookieUtil.unset() method handles this case. It accepts four arguments: the name of the cookie to remove, an optional path argument, an optional domain argument, and an optional secure argument.
These arguments are passed through to CookieUtil.set() with the value set to a blank string and the expiration date set to January 1, 1970 (the value of a Date object initialized to 0 milliseconds). Doing so ensures that the cookie is removed.
These methods can be used as follows:
//set cookies
CookieUtil.set("name", "Nicholas");
CookieUtil.set("book", "Professional JavaScript");
//read the values
alert(CookieUtil.get("name")); //"Nicholas"
alert(CookieUtil.get("book")); //"Professional JavaScript"
//remove the cookies
CookieUtil.unset("name");
CookieUtil.unset("book");
//set a cookie with path, domain, and expiration date
CookieUtil.set("name", "Nicholas", "/books/projs/", "www.wrox.com",
new Date("January 1, 2010"));
//delete that same cookie
CookieUtil.unset("name", "/books/projs/", "www.wrox.com");
//set a secure cookie
CookieUtil.set("name", "Nicholas", null, null, null, true);
CookieExample01.htm
These methods make using cookies to store data on the client easier by handling the parsing and cookie string construction tasks.
Subcookies
To get around the per-domain cookie limit imposed by browsers, some developers use a concept called subcookies. Subcookies are smaller pieces of data stored within a single cookie. The idea is to use the cookie’s value to store multiple name-value pairs within a single cookie. The most common format for subcookies is as follows:
name=name1=value1&name2=value2&name3=value3&name4=value4&name5=value5
Subcookies tend to be formatted in query string format. These values can then be stored and accessed using a single cookie, rather than using a different cookie for each name-value pair. The result is that more structured data can be stored by a website or web application without reaching the per-domain cookie limit.
To work with subcookies, you need a new set of methods. The parsing and serialization of subcookies are slightly different and a bit more complicated because of the expected subcookie usage. To get a subcookie, for example, you need to follow the same basic steps to get a cookie, but before decoding the value, you need to find the subcookie information as follows:
var SubCookieUtil = {
get: function (name, subName){
var subCookies = this.getAll(name);
if (subCookies){
return subCookies[subName];
} else {
return null;
}
},
getAll: function(name){
var cookieName = encodeURIComponent(name) + "=",
cookieStart = document.cookie.indexOf(cookieName),
cookieValue = null,
cookieEnd,
subCookies,
i,
parts,
result = {};
if (cookieStart > -1){
cookieEnd = document.cookie.indexOf(";", cookieStart);
if (cookieEnd == -1){
cookieEnd = document.cookie.length;
}
cookieValue = document.cookie.substring(cookieStart +
cookieName.length, cookieEnd);
if (cookieValue.length > 0){
subCookies = cookieValue.split("&");
for (i=0, len=subCookies.length; i < len; i++){
parts = subCookies[i].split("=");
result[decodeURIComponent(parts[0])] =
decodeURIComponent(parts[1]);
}
return result;
}
}
return null;
},
//more code here
};
SubCookieUtil.js
There are two methods for retrieving subcookies: get() and getAll(). Whereas get() retrieves a single subcookie value, getAll() retrieves all subcookies and returns them in an object whose properties are equal to the subcookie names and the values are equal to the subcookie values. The get() method accepts two arguments: the name of the cookie and the name of the subcookie. It simply calls getAll() to retrieve all of the subcookies and then returns just the one of interest (or null if the cookie doesn’t exist).
The SubCookieUtil.getAll() method is very similar to CookieUtil.get() in the way it parses a cookie value. The difference is that the cookie value isn’t immediately decoded. Instead, it is split on the ampersand character to get all subcookies into an array. Then, each subcookie is split on the equal sign so that the first item in the parts array is the subcookie name, and the second is the subcookie value. Both items are decoded using decodeURIComponent() and assigned on the result object, which is returned as the method value. If the cookie doesn’t exist, then null is returned.
These methods can be used as follows:
//assume document.cookie=data=name=Nicholas&book=Professional%20JavaScript
//get all subcookies
var data = SubCookieUtil.getAll("data");
alert(data.name); //"Nicholas"
alert(data.book); //"Professional JavaScript"
//get subcookies individually
alert(SubCookieUtil.get("data", "name")); //"Nicholas"
alert(SubCookieUtil.get("data", "book")); //"Professional JavaScript"
SubCookiesExample01.htm
To write subcookies, you can use two methods: set() and setAll(). The following code shows their constructs:
var SubCookieUtil = {
set: function (name, subName, value, expires, path, domain, secure) {
var subcookies = this.getAll(name) || {};
subcookies[subName] = value;
this.setAll(name, subcookies, expires, path, domain, secure);
},
setAll: function(name, subcookies, expires, path, domain, secure){
var cookieText = encodeURIComponent(name) + "=",
subcookieParts = new Array(),
subName;
for (subName in subcookies){
if (subName.length > 0 && subcookies.hasOwnProperty(subName)){
subcookieParts.push(encodeURIComponent(subName) + "=" +
encodeURIComponent(subcookies[subName]));
}
}
if (cookieParts.length > 0){
cookieText += subcookieParts.join("&");
if (expires instanceof Date) {
cookieText += "; expires=" + expires.toGMTString();
}
if (path) {
cookieText += "; path=" + path;
}
if (domain) {
cookieText += "; domain=" + domain;
}
if (secure) {
cookieText += "; secure";
}
} else {
cookieText += "; expires=" + (new Date(0)).toGMTString();
}
document.cookie = cookieText;
},
//more code here
};
SubCookieUtil.js
The set() method accepts seven arguments: the cookie name, the subcookie name, the subcookie value, an optional Date object for the cookie expiration day/time, an optional cookie path, an optional cookie domain, and an optional Boolean secure flag. All of the optional arguments refer to the cookie itself and not to the subcookie. In order to store multiple subcookies in the same cookie, the path, domain, and secure flag must be the same; the expiration date refers to the entire cookie and can be set whenever an individual subcookie is written. Inside the method, the first step is to retrieve all of the subcookies for the given cookie name. The logical OR operator is used to set subcookies to a new object if getAll() returns null. After that, the subcookie value is set on the subcookies object and then passed into setAll().
The setAll() method accepts six arguments: the cookie name, an object containing all of the subcookies, and then the rest of the optional arguments used in set(). This method iterates over the properties of the second argument using a for-in loop. To ensure that the appropriate data is saved, use the hasOwnProperty() method to ensure that only the instance properties are serialized into subcookies. Since it’s possible to have a property name equal to the empty string, the length of the property name is also checked before being added to the result. Each subcookie name-value pair is added to the subcookieParts array so that they can later be easily joined with an ampersand using the join() method. The rest of the method is the same as CookieUtil.set().
These methods can be used as follows:
//assume document.cookie=data=name=Nicholas&book=Professional%20JavaScript
//set two subcookies
SubCookieUtil.set("data", "name", "Nicholas");
SubCookieUtil.set("data", "book", "Professional JavaScript");
//set all subcookies with expiration date
SubCookieUtil.setAll("data", { name: "Nicholas", book: "Professional JavaScript" },
new Date("January 1, 2010"));
//change the value of name and change expiration date for cookie
SubCookieUtil.set("data", "name", "Michael", new Date("February 1, 2010"));
SubCookiesExample01.htm
The last group of subcookie methods has to do with removing subcookies. Regular cookies are removed by setting the expiration date to some time in the past, but subcookies cannot be removed as easily. In order to remove a subcookie, you need to retrieve all subcookies contained within the cookie, eliminate just the one that is meant to be removed, and then set the value of the cookie back with the remaining subcookie values. Consider the following:
var SubCookieUtil = {
//more code here
unset: function (name, subName, path, domain, secure){
var subcookies = this.getAll(name);
if (subcookies){
delete subcookies[subName];
this.setAll(name, subcookies, null, path, domain, secure);
}
},
unsetAll: function(name, path, domain, secure){
this.setAll(name, null, new Date(0), path, domain, secure);
}
};
SubCookieUtil.js
The two methods defined here serve two different purposes. The unset() method is used to remove a single subcookie from a cookie while leaving the rest intact; whereas the unsetAll() method is the equivalent of CookieUtil.unset(), which removes the entire cookie. As with set() and setAll(), the path, domain, and secure flag must match the options with which a cookie was created. These methods can be used as follows:
//just remove the "name" subcookie
SubCookieUtil.unset("data", "name");
//remove the entire cookie
SubCookieUtil.unsetAll("data");
If you are concerned about reaching the per-domain cookie limit in your work, subcookies are an attractive alternative. You will have to more closely monitor the size of your cookies to stay within the individual cookie size limit.
Cookie Considerations
There is also a type of cookie called HTTP-only. HTTP-only cookies can be set either from the browser or from the server but can be read only from the server, because JavaScript cannot get the value of HTTP-only cookies.
Since all cookies are sent as request headers from the browser, storing a large amount of information in cookies can affect the overall performance of browser requests to a particular domain. The larger the cookie information, the longer it will take to complete the request to the server. Even though the browser places size limits on cookies, it’s a good idea to store as little information as possible in cookies, to avoid performance implications.
The restrictions on and nature of cookies make them less than ideal for storing large amounts of information, which is why other approaches have emerged.
It is strongly recommended to avoid storing important or sensitive data in cookies. Cookie data is not stored in a secure environment, so any data contained within may be accessible by others. You should avoid storing data such as credit card numbers or personal addresses in cookies.
Internet Explorer User Data
In Internet Explorer 5, Microsoft introduced the concept of persistent user data via a custom behavior. User data allows you to store up to 128KB of data per document and up to 1MB of data per domain. To use persistent user data, you first must specify the userData behavior as shown here on an element using CSS:
<div style="behavior:url(#default#userData)" id="dataStore"></div>
Once an element is using the userData behavior, you can save data onto it using the setAttribute() method. In order to commit the data into the browser cache, you must then call save() and pass in the name of the data store to save to. The data store name is completely arbitrary and is used to differentiate between different sets of data. Consider the following example:
var dataStore = document.getElementById("dataStore");
dataStore.setAttribute("name", "Nicholas");
dataStore.setAttribute("book", "Professional JavaScript");
dataStore.save("BookInfo");
UserDataExample01.htm
In this code, two pieces of information are saved on the <div> element. After setAttribute() is used to store that data, the save() method is called with a data store name of "BookInfo". The next time the page is loaded, you can use the load() method with the data store name to retrieve the data as follows:
dataStore.load("BookInfo");
alert(dataStore.getAttribute("name")); //"Nicholas"
alert(dataStore.getAttribute("book")); //"Professional JavaScript"
UserDataExample01.htm
The call to load() retrieves all of the information from the "BookInfo" data store and makes it available on the element; the information is not available until explicitly loaded. If getAttribute() is called for a name that either doesn’t exist or hasn’t been loaded, then null is returned.
You can explicitly remove data from the element by using the removeAttribute() method and passing in the attribute name. Once removed, save() must be called again, to commit the changes as shown here:
dataStore.removeAttribute("name");
dataStore.removeAttribute("book");
dataStore.save("BookInfo");
UserDataExample01.htm
This code removes two data attributes and then saves those changes to the cache.
The accessibility restrictions on Internet Explorer user data are similar to the restrictions on cookies. In order to access a data store, the page on which the script is running must be from the same domain, on the same directory path, and using the same protocol as the script that saved data to the store. Unlike with cookies, you cannot change accessibility restrictions on user data to a wider range of consumers. Also unlike cookies, user data persists across sessions by default and doesn’t expire; data needs to be specifically removed using removeAttribute() in order to free up space.
As with cookies, Internet Explorer user data is not secure and should not be used to store sensitive information.
Web Storage
Web Storage was first described in the Web Applications 1.0 specification of the Web Hypertext Application Technical Working Group (WHAT-WG). The initial work from this specification eventually became part of HTML5 before being split into its own specification. Its intent is to overcome some of the limitations imposed by cookies when data is needed strictly on the client side, with no need to continuously send data back to the server. The two primary goals of Web Storage are:
- To provide a way to store session data outside of cookies.
- To provide a mechanism for storing large amounts of data that persists across sessions.
The original Web Storage specification includes definitions for two objects: localStorage and globalStorage. These objects are available as a property of window in Internet Explorer 8+, Firefox 3.5+, Safari 3.1+, Chrome 4+, and Opera 10.5+.
Firefox 2 and 3 had partial implementations of Web Storage that were based on earlier work where an object called globalStorage was implemented instead of localStorage.
The Storage Type
The Storage type is designed to hold name-value pairs up to a maximum size (determined by the browser). An instance of Storage acts like any other object and has the following additional methods:
- clear() — Removes all values; not implemented in Firefox.
- getItem(name) — Retrieves the value for the given name.
- key(index) — Retrieves the name of the value in the given numeric position.
- removeItem(name) — Removes the name-value pair identified by name.
- setItem(name, value) — Sets the value for the given name.
The getItem(), removeItem(), and setItem() methods can be called directly or indirectly by manipulating the Storage object. Since each item is stored on the object as a property, you can simply read values by accessing the property with dot or bracket notation, set the value by doing the same, or remove it by using the delete operator. Even so, it’s generally recommended to use the methods instead of property access to ensure you don’t end up overwriting one of the already available object members with a key.
You can determine how many name-value pairs are in a Storage object by using the length property. It’s not possible to determine the size of all data in the object, although Internet Explorer 8 provides a remainingSpace property that retrieves the amount of space, in bytes, that is still available for storage.
The Storage type is capable of storing only strings. Nonstring data is converted into a string before being stored.
The sessionStorage Object
The sessionStorage object stores data only for a session, meaning that the data is stored until the browser is closed. This is the equivalent of a session cookie that disappears when the browser is closed. Data stored on sessionStorage persists across page refreshes and may also be available if the browser crashes and is restarted, depending on the browser vendor. (Firefox and WebKit support this, but Internet Explorer does not.)
Because the sessionStorage object is tied to a server session, it isn’t available when a file is run locally. Data stored on sessionStorage is accessible only from the page that initially placed the data onto the object, making it of limited use for multipage applications.
Since the sessionStorage object is an instance of Storage, you can assign data onto it either by using setItem() or by assigning a new property directly. Here’s an example of each of these methods:
//store data using method
sessionStorage.setItem("name", "Nicholas");
//store data using property
sessionStorage.book = "Professional JavaScript";
SessionStorageExample01.htm
Writing to storage has slight differences from browser to browser. Firefox and WebKit implement storage writing synchronously, so data added to storage is committed right away. The Internet Explorer implementation writes data asynchronously, so there may be a lag between the time when data is assigned and the time that the data is written to disk. For small amounts of data, the difference is negligible. For large amounts of data, you’ll notice that JavaScript in Internet Explorer resumes execution faster than in other browsers, because it offloads the actual disk write process.
You can force disk writing to occur in Internet Explorer 8 by using the begin() method before assigning any new data, and the commit() method after all assignments have been made. Consider the following example:
//IE8 only sessionStorage.begin(); sessionStorage.name = "Nicholas"; sessionStorage.book = "Professional JavaScript"; sessionStorage.commit();
This code ensures that the values for "name" and "book" are written as soon as commit() is called. The call to begin() ensures that no disk writes will occur while the code is executed. For small amounts of data, this process isn’t necessary; however, you may wish to consider this transactional approach for larger amounts of data such as documents.
When data exists on sessionStorage, it can be retrieved either by using getItem() or by accessing the property name directly. Here’s an example of each of these methods:
//get data using method
var name = sessionStorage.getItem("name");
//get data using property
var book = sessionStorage.book;
SessionStorageExample01.htm
You can iterate over the values in sessionStorage using a combination of the length property and key() method, as shown here:
for (var i=0, len = sessionStorage.length; i < len; i++){
var key = sessionStorage.key(i);
var value = sessionStorage.getItem(key);
alert(key + "=" + value);
}
SessionStorageExample01.htm
The name-value pairs in sessionStorage can be accessed sequentially by first retrieving the name of the data in the given position via key() and then using that name to retrieve the value via getItem().
It’s also possible to iterate over the values in sessionStorage using a for-in loop:
for (var key in sessionStorage){
var value = sessionStorage.getItem(key);
alert(key + "=" + value);
}
Each time through the loop, key is filled with another name in sessionStorage; none of the built-in methods or the length property will be returned.
To remove data from sessionStorage, you can use either the delete operator on the object property or the removeItem() method. Here’s an example of each of these methods:
//use delete to remove a value - won't work in WebKit
delete sessionStorage.name;
//use method to remove a value
sessionStorage.removeItem("book");
SessionStorageExample01.htm
It’s worth noting that as of the time of this writing, the delete operator doesn’t remove data in WebKit, whereas removeItem() works correctly across all supporting browsers.
The sessionStorage object should be used primarily for small pieces of data that are valid only for a session. If you need to persist data across sessions, then either globalStorage or localStorage is more appropriate.
The globalStorage Object
The globalStorage object is implemented in Firefox 2. As part of the original Web Storage specification, its purpose is to persist data across sessions and with specific access restrictions. In order to use globalStorage, you need to specify the domains for which the data should be available. This is done using a property via bracket notation, as shown in the following example:
//save value
globalStorage["wrox.com"].name = "Nicholas";
//get value
var name = globalStorage["wrox.com"].name;
GlobalStorageExample01.htm
Here, a storage area for the domain wrox.com is accessed. Whereas the globalStorage object itself is not an instance of Storage, the globalStorage["wrox.com”] specification is and can be used accordingly. This storage area is accessible from wrox.com and all subdomains. You can limit the subdomain by specifying it as follows:
//save value
globalStorage["www.wrox.com"].name = "Nicholas";
//get value
var name = globalStorage["www.wrox.com"].name;
GlobalStorageExample01.htm
The storage area specified here is accessible only from a page on www.wrox.com, excluding other subdomains.
Some browsers allow more general access restrictions, such as those limited only by top-level domains (TLDs) or by allowing global access, such as in the following example:
//store data that is accessible to everyone - AVOID!
globalStorage[""].name = "Nicholas";
//store data available only to domains ending with .net - AVOID!
globalStorage["net"].name = "Nicholas";
Even though these are supported, it is recommended to avoid using generally accessible data stores, to prevent possible security issues. It’s also possible that because of security concerns, this ability will be either removed or severely limited in the future, so applications should not rely on this type of functionality. Always specify a domain name when using globalStorage.
Access to globalStorage areas is limited by the domain, protocol, and port of the page making the request. For instance, if data is stored for wrox.com while using the HTTPS protocol, a page on wrox.com accessed via HTTP cannot access that information. Likewise, a page accessed via port 80 cannot share data with a page on the same domain and use the same protocol that is accessed on port 8080. This is similar to the same-origin policy for Ajax requests.
Each property of globalStorage is an instance of Storage. Therefore, it can be used as in the following example:
globalStorage["www.wrox.com"].name = "Nicholas";
globalStorage["www.wrox.com"].book = "Professional JavaScript";
globalStorage["www.wrox.com"].removeItem("name");
var book = globalStorage["www.wrox.com"].getItem("book");
GlobalStorageExample01.htm
If you aren’t certain of the domain name to use ahead of time, it may be safer to use location.host as the property name. For example:
globalStorage[location.host].name = "Nicholas";
var book = globalStorage[location.host].getItem("book");
GlobalStorageExample01.htm
The data stored in a globalStorage property remains on disk until it’s removed via either removeItem() or delete, or until the user clears the browser’s cache. This makes globalStorage ideal for storing documents on the client or persisting user settings.
The localStorage Object
The localStorage object superceded globalStorage in the revised HTML5 specification as a way to store persistent client-side data. Unlike with globalStorage, you cannot specify any accessibility rules on localStorage; the rules are already set. In order to access the same localStorage object, pages must be served from the same domain (subdomains aren’t valid), using the same protocol, and on the same port. This is effectively the same as globalStorage[location.host].
Since localStorage is an instance of Storage, it can be used in the same manner as sessionStorage. Here are some examples:
//store data using method
localStorage.setItem("name", "Nicholas");
//store data using property
localStorage.book = "Professional JavaScript";
//get data using method
var name = localStorage.getItem("name");
//get data using property
var book = localStorage.book;
LocalStorageExample01.htm
Data that is stored in localStorage follows the same rules as data stored in globalStorage, because the data is persisted until it is specifically removed via JavaScript or the user clears the browser’s cache.
To equalize for browsers that support only globalStorage, you can use the following function:
function getLocalStorage(){
if (typeof localStorage == "object"){
return localStorage;
} else if (typeof globalStorage == "object"){
return globalStorage[location.host];
} else {
throw new Error("Local storage not available.");
}
}
GlobalAndLocalStorageExample01.htm
Then, the following initial call to the function is all that is necessary to identify the correct location for data:
var storage = getLocalStorage();
GlobalAndLocalStorageExample01.htm
After determining which Storage object to use, you can easily continue storing and retrieving data with the same access rules across all browsers that support Web Storage.
The storage Event
Whenever a change is made to a Storage object, the storage event is fired on the document. This occurs for every value set using either properties or setItem(), every value removal using either delete or removeItem(), and every call to clear(). The event object has the following four properties:
- domain — The domain for which the storage changed.
- key — The key that was set or removed.
- newValue — The value that the key was set to, or null if the key was removed.
- oldValue — The value prior to the key being changed.
Of these four properties, Internet Explorer 8 and Firefox have implemented only the domain property. WebKit doesn’t support the storage event as of the date of this writing.
You can listen for the storage event using the following code:
EventUtil.addHandler(document, "storage", function(event){
alert("Storage changed for " + event.domain);
});
StorageEventExample01.htm
The storage event is fired for all changes to sessionStorage and localStorage but doesn’t distinguish between them.
Limits and Restrictions
As with other client-side data storage solutions, Web Storage also has limitations. These limitations are browser-specific. Generally speaking, the size limit for client-side data is set on a per-origin (protocol, domain, and port) basis, so each origin has a fixed amount of space in which to store its data. Analyzing the origin of the page that is storing the data enforces this restriction.
Most desktop browsers have a 5MB per-origin limit for localStorage. Chrome and Safari have a per-origin limit of 2.5MB. Safari for iOS and WebKit for Android also have a limit of 2.5MB.
The limits for sessionStorage vary across browsers. Many browsers have no limit on sessionStorage data, while Chrome, Safari, Safari for iOS, and WebKit for Android have a limit of 2.5MB. Internet Explorer 8+ and Opera have a limit of 5MB for sessionStorage.
For more information about Web Storage limits, please see the Web Storage Support Test at http://dev-test.nemikor.com/web-storage/support-test/.
IndexedDB
The Indexed Database API, IndexedDB for short, is a structured data store in the browser. IndexedDB came about as an alternative to the now-deprecated Web SQL Database API (not covered in this book because of its deprecated state). The idea behind IndexedDB was to create an API that easily allowed the storing and retrieval of JavaScript objects while still allowing querying and searching.
IndexedDB is designed to be almost completely asynchronous. As a result, most operations are performed as requests that will execute later and produce either a successful result or an error. Nearly every IndexedDB operation requires you to attach onerror and onsuccess event handlers to determine the outcome.
Once fully supported, there will be a global indexedDB object that serves as the API host. While the API is still in flux, browsers are using vendor prefixes, so the object in Internet Explorer 10 is called msIndexedDB, in Firefox 4 it’s called mozIndexedDB, and in Chrome it’s called webkitIndexedDB. This section uses indexedDB in the examples for clarity, so you may need to include the following code before each example:
var indexedDB = window.indexedDB || window.msIndexedDB || window.mozIndexedDB || window.webkitIndexedDB;
IndexedDBExample01.htm
Databases
IndexedDB is a database similar to databases you’ve probably used before such as MySQL or Web SQL Database. The big difference is that IndexedDB uses object stores instead of tables to keep track of data. An IndexedDB database is simply a collection of object stores grouped under a common name.
The first step to using a database is to open it using indexedDB.open() and passing in the name of the database to open. If a database with the given name already exists, then a request is made to open it; if the database doesn’t exist, then a request is made to create and open it. The call to indexDB.open() returns an instance of IDBRequest onto which you can attach onerror and onsuccess event handlers. Here’s an example:
var request, database;
request = indexedDB.open("admin");
request.onerror = function(event){
alert("Something bad happened while trying to open: " +
event.target.errorCode);
};
request.onsuccess = function(event){
database = event.target.result;
};
IndexedDBExample01.htm
In both event handlers, event.target points to request, so these may be used interchangeably. If the onsuccess event handler is called, then the database instance object (IDBDatabase) is available in event.target.result and stored in the database variable. From this point on, all requests to work with the database are made through the database object itself. If an error occurs, an error code stored in event.target.errorCode indicates the nature of the problem as one of the following (these error codes apply to all operations):
- IDBDatabaseException.UNKNOWN_ERR (1) — The error is unexpected and doesn’t fall into an available category.
- IDBDatabaseException.NON_TRANSIENT_ERR (2) — The operation is not allowed.
- IDBDatabaseException.NOT_FOUND_ERR (3) — The database on which to perform the operation is not found.
- IDBDatabaseException.CONSTRAINT_ERR (4) — A database constraint was violated.
- IDBDatabaseException.DATA_ERR (5) — Data provided for the transaction doesn’t fulfill the requirements.
- IDBDatabaseException.NOT_ALLOWED_ERR (6) — The operation is not allowed.
- IDBDatabaseException.TRANSACTION_INACTIVE_ERR (7) — An attempt was made to reuse an already completed transaction.
- IDBDatabaseException.ABORT_ERR (8) — The request was aborted and so did not succeed.
- IDBDatabaseException.READ_ONLY_ERR (9) — Attempt to write or otherwise change data while in read-only mode.
- IDBDatabaseException.TIMEOUT_ERR (10) — The operation could not be completed in the amount of time available.
- IDBDatabaseException.QUOTA_ERR (11) — Not enough remaining disk space.
By default, a database has no version associated with it, so it’s a good idea to set the initial version when starting out. To do so, call the setVersion() method and pass in the version as a string. Once again, this creates a request object on which you’ll need to assign event handlers:
if (database.version != "1.0"){
request = database.setVersion("1.0");
request.onerror = function(event){
alert("Something bad happened while trying to set version: " +
event.target.errorCode);
};
request.onsuccess = function(event){
alert("Database initialization complete. Database name: " + database.name +
", Version: " + database.version);
};
} else {
alert("Database already initialized. Database name: " + database.name +
", Version: " + database.version);
}
IndexedDBExample01.htm
This example tries to set the version of the database to “1.0”. The first line checks the version property to see if the database version has already been set. If not, then setVersion() is called to create the version change request. If that request is successful, then a message is displayed indicating that the version change is complete. (In a real implementation, this is where you would set up your object stores. See the next section for details.)
If the database version is already “1.0”, then a message is displayed stating that the database has already been initialized. This basic pattern is how you can tell if the database you want to use has already been set up with appropriate object stores or not. Over the course of a web application, you may have many different versions of the database as you update and modify the data structures.
Object Stores
Once you have established a connection to the database, the next step is to interact with object stores. If the database version doesn’t match the one you expect then you likely will need to create an object store. Before creating an object store, however, it’s important to think about the type of data you want to store.
Suppose that you’d like to store user records containing username, password, and so on. The object to hold a single record may look like this:
var user = {
username: "007",
firstName: "James",
lastName: "Bond",
password: "foo"
};
Looking at this object, you can easily see that an appropriate key for this object store is the username property. A username must be globally unique, and it’s probably the way you’ll be accessing data most of the time. This is important because you must specify a key when creating an object store. Here’s how you would create an object store for these users:
var store = db.createObjectStore("users", { keyPath: "username" });
IndexedDBExample02.htm
The keyPath property of the second argument indicates the property name of the stored objects that should be used as a key.
Since you now have a reference to the object store, it’s possible to populate it with data using either add() or put(). Both of these methods accept a single argument, the object to store, and save the object into the object store. The difference between these two occurs only when an object with the same key already exists in the object store. In that case, add() will cause an error while put() will simply overwrite the object. More simply, think of add() as being used for inserting new values while put() is used for updating values. So to initialize an object store for the first time, you may want to do something like this:
//where users is an array of new users
var i=0,
len = users.length;
while(i < len){
store.add(users[i++]);
}
IndexedDBExample02.htm
Each call to add() or put() creates a new update request for the object store. If you want verification that the request completed successfully, you can store the request object in a variable and assign onerror and onsuccess event handlers:
//where users is an array of new users
var i=0,
request,
requests = [],
len = users.length;
while(i < len){
request = store.add(users[i++]);
request.onerror = function(){
//handle error
};
request.onsuccess = function(){
//handle success
};
requests.push(request);
}
Once the object store is created and filled with data, it’s time to start querying.
Transactions
Past the creation step of an object store, all further operations are done through transactions. A transaction is created using the transaction() method on the database object. Any time you want to read or change data, a transaction is used to group all changes together. In its simplest form, you create a new transaction as follows:
var transaction = db.transaction();
With no arguments specified, you have read-only access to all object stores in the database. To be more optimal, you can specify one or more object store names that you want to access:
var transaction = db.transaction("users");
This ensures that only information about the users object store is loaded and available during the transaction. If you want access to more than one object store, the first argument can also be an array of strings:
var transaction = db.transaction(["users", "anotherStore"]);
As mentioned previously, each of these transactions accesses data in a read-only manner. To change that, you must pass in a second argument indicating the access mode. These constants are accessible on IDBTransaction as READ_ONLY (0), READ_WRITE (1), and VERSION_CHANGE (2). While Internet Explorer 10+ and Firefox 4+ implement IDBTransaction, Chrome supports it via webkitIDBTransaction, so the following code is necessary to normalize the interface:
var IDBTransaction = window.IDBTransaction || window.webkitIDBTransaction;
IndexedDBExample03.htm
With that setup, you can specify the second argument to transaction():
var transaction = db.transaction("users", IDBTransaction.READ_WRITE);
IndexedDBExample03.htm
This transaction is capable of both reading and writing into the users object store.
Once you have a reference to the transaction, you can access a particular object store using the objectStore() method and passing in the store name you want to work with. You can then use add() and put() as before, as well as get() to retrieve values, delete() to remove an object, and clear() to remove all objects. The get() and delete() methods each accept an object key as their argument, and all five of these methods create a new request object. For example:
var request = db.transaction("users").objectStore("users").get("007");
request.onerror = function(event){
alert("Did not get the object!");
};
request.onsuccess = function(event){
var result = event.target.result;
alert(result.firstName); //"James"
};
IndexedDBExample02.htm
Because any number of requests can be completed as part of a single transaction, the transaction object itself also has event handlers: onerror and oncomplete. These are used to provide transaction-level state information:
transaction.onerror = function(event){
//entire transaction was cancelled
};
transaction.oncomplete = function(event){
//entire transaction completed successfully
};
Keep in mind that the event object for oncomplete doesn’t give you access to any data returned by get() requests, so you still need an onsuccess event handler for those types of requests.
Querying with Cursors
Transactions can be used directly to retrieve a single item with a known key. When you want to retrieve multiple items, you need to create a cursor within the transaction. A cursor is a pointer into a result set. Unlike traditional database queries, a cursor doesn’t gather all of the result set up front. Instead, a cursor points to the first result and doesn’t try to find the next until instructed to do so.
Cursors are created using the openCursor() method on an object store. As with other operations with IndexedDB, the return value of openCursor() is a request, so you must assign onsuccess and onerror event handlers. For example:
var store = db.transaction("users").objectStore("users"),
request = store.openCursor();
request.onsuccess = function(event){
//handle success
};
request.onfailure = function(event){
//handle failure
};
IndexedDBExample04.htm
When the onsuccess event handler is called, the next item in the object store is accessible via event.target.result, which holds an instance of IDBCursor when there is a next item or null when there are no further items. The IDBCursor instance has several properties:
- direction — A numeric value indicating the direction the cursor should travel in. The default is IDBCursor.NEXT (0) for next. Other values include IDBCursor.NEXT_NO_DUPLICATE (1) for next without duplicates, IDBCursor.PREV (2) for previous, and IDBCursor.PREV_NO_DUPLICATE (3) for previous without duplicates.
- key — The key for the object.
- value — The actual object.
- primaryKey — The key being used by the cursor. Could be the object key or an index key (discussed later).
You can retrieve information about a single result using the following:
request.onsuccess = function(event){
var cursor = event.target.result;
if (cursor){ //always check
console.log("Key: " + cursor.key + ", Value: " +
JSON.stringify(cursor.value));
}
};
Keep in mind that cursor.value in this example is an object, which is why it is JSON encoded before being displayed.
A cursor can be used to update an individual record. The update() method updates the current cursor value with the specified object. As with other such operations, the call to update() creates a new request, so you need to assign onsuccess and onerror if you want to know the result:
request.onsuccess = function(event){
var cursor = event.target.result,
value,
updateRequest;
if (cursor){ //always check
if (cursor.key == "foo"){
value = cursor.value; //get current value
value.password = "magic!"; //update the password
updateRequest = cursor.update(value); //request the update be saved
updateRequest.onsuccess = function(){
//handle success;
};
updateRequest.onfailure = function(){
//handle failure
};
}
}
};
You can also delete the item at that position by calling delete(). As with update(), this also creates a request:
request.onsuccess = function(event){
var cursor = event.target.result,
value,
deleteRequest;
if (cursor){ //always check
if (cursor.key == "foo"){
deleteRequest = cursor.delete(); //request the value be deleted
deleteRequest.onsuccess = function(){
//handle success;
};
deleteRequest.onfailure = function(){
//handle failure
};
}
}
};
Both update() and delete() will throw errors if the transaction doesn’t have permission to modify the object store.
Each cursor makes only one request by default. To make another request, you must call one of the following methods:
- continue(key) — Moves to the next item in the result set. The argument key is optional. When not specified, the cursor just moves to the next item; when provided, the cursor will move to the specified key.
- advance(count) — Moves the cursor ahead by count number of items.
Each of these methods causes the cursor to reuse the same request, so the same onsuccess and onfailure event handlers are reused until no longer needed. For example, the following iterates over all items in an object store:
request.onsuccess = function(event){
var cursor = event.target.result;
if (cursor){ //always check
console.log("Key: " + cursor.key + ", Value: " +
JSON.stringify(cursor.value));
cursor.continue(); //go to the next one
} else {
console.log("Done!");
}
};
The call to continue() triggers another request and onsuccess is called again. When there are no more items to iterate over, onsuccess is called one last time with event.target.result equal to null.
Key Ranges
Working with cursors may seem suboptimal since you’re limited in the ways data can be retrieved. Key ranges are used to make working with cursors a little more manageable. A key range is represented by an instance of IDBKeyRange. The standard version is IDBKeyRange, which is supported in Internet Explorer 10+ and Firefox 4+, while Chrome supports this type with webkitIDBKeyRange. As with other types related to IndexedDB, you’ll first need to create a local copy, taking these differences into account:
var IDBKeyRange = window.IDBKeyRange || window.webkitIDBKeyRange;
There are four different ways to specify key ranges. The first is to use the only() method and pass in the key you want to retrieve:
var onlyRange = IDBKeyRange.only("007");
This range ensures that only the value with a key of "007" will be retrieved. A cursor created using this range is similar to directly accessing an object store and calling get("007").
The second type of range defines a lower bound for the result set. The lower bound indicates the item at which the cursor should start. For example, the following key range ensures the cursor starts at the key "007" and continues until the end:
//start at item "007", go to the end
var lowerRange = IDBKeyRange.lowerBound("007");
If you want to start at the item immediately following the value at "007", then you can pass in a second argument of true:
//start at item after "007", go to the end
var lowerRange = IDBKeyRange.lowerBound("007", true);
The third type of range is an upper bound, indicating the key you don’t want to go past by using the upperBound() method. The following key ensures that the cursor starts at the beginning and stops when it gets to the value with key "ace":
//start at beginning, go to "ace"
var upperRange = IDBKeyRange.upperBound("ace");
If you don’t want to include the given key, then pass in true as the second argument:
//start at beginning, go to the item just before "ace"
var upperRange = IDBKeyRange.upperBound("ace", true);
To specify both a lower and an upper bound, use the bound() method. This method accepts four arguments, the lower bound key, the upper bound key, an optional Boolean indicating to skip the lower bound, and an optional Boolean indicating to skip the upper bound. Here are some examples:
//start at "007", go to "ace"
var boundRange = IDBKeyRange.bound("007", "ace");
//start at item after "007", go to "ace"
var boundRange = IDBKeyRange.bound("007", "ace", true);
//start at item after "007", go to item before "ace"
var boundRange = IDBKeyRange.bound("007", "ace", true, true);
//start at "007", go to item before "ace"
var boundRange = IDBKeyRange.bound("007", "ace", false, true);
Once you have defined a range, pass it into the openCursor() method and you’ll create a cursor that stays within the constraints:
var store = db.transaction("users").objectStore("users"),
range = IDBKeyRange.bound("007", "ace");
request = store.openCursor(range);
request.onsuccess = function(event){
var cursor = event.target.result;
if (cursor){ //always check
console.log("Key: " + cursor.key + ", Value: " +
JSON.stringify(cursor.value));
cursor.continue(); //go to the next one
} else {
console.log("Done!");
}
};
This example outputs only the values between keys "007" and "ace", which are fewer than the previous section’s example.
Setting Cursor Direction
There are actually two arguments to openCursor(). The first is an instance of IDBKeyRange and the second is a numeric value indicating the direction. These constants are specified as constants on IDBCursor as discussed in the querying section. Firefox 4+ and Chrome once again have different implementations, so the first step is to normalize the differences locally:
var IDBCursor = window.IDBCursor || window.webkitIDBCursor;
Normally cursors start at the first item in the object store and progress toward the last with each call to continue() or advance(). These cursors have the default direction value of IDBCursor.NEXT. If there are duplicates in the object store, you may want to have a cursor that skips over the duplicates. You can do so by passing IDBCursor.NEXT_NO_DUPLICATE into openCursor() as the second argument:
var store = db.transaction("users").objectStore("users"),
request = store.openCursor(null, IDBCursor.NEXT_NO_DUPLICATE);
Note that the first argument to openCursor() is null, which indicates that the default key range of all values should be used. This cursor will iterate through the items in the object store starting from the first item and moving toward the last while skipping any duplicates.
You can also create a cursor that moves backward through the object store, starting at the last item and moving toward the first by passing in either IDBCursor.PREV or IDBCursor.PREV_NO_DUPLICATE (the latter, of course, to avoid duplicates). For example:
var store = db.transaction("users").objectStore("users"),
request = store.openCursor(null, IDBCursor.PREV);
IndexedDBExample05.htm
When you open a cursor using IDBCursor.PREV or IDBCursor.PREV_NO_DUPLICATE, each call to continue() or advance() moves the cursor backward through the object store instead of forward.
Indexes
For some data sets, you may want to specify more than one key for an object store. For example, if you’re tracking users by both a user ID and a username, you may want to access records using either piece of data. To do so, you would likely consider the user ID as the primary key and create an index on the username.
To create a new index, first retrieve a reference to the object store and then call createIndex(), as in this example:
var store = db.transaction("users").objectStore("users"),
index = store.createIndex("username", "username", { unique: true });
The first argument to createIndex() is the name of the index, the second is the name of the property to index, and third is an options object containing the key unique. This option should always be specified so as to indicate whether or not the key is unique across all records. Since username may not be duplicated, this index is not unique.
The returned value from createIndex() is an instance of IDBIndex. You can also retrieve the same instance via the index() method on an object store. For example, to use an already existing index named "username", the code would be:
var store = db.transaction("users").objectStore("users"),
index = store.index("username");
An index acts a lot like an object store. You can create a new cursor on the index using the openCursor() method, which works exactly the same as openCursor() on an object store except that the result.key property is filled in with the index key instead of the primary key. Here’s an example:
var store = db.transaction("users").objectStore("users"),
index = store.index("username"),
request = index.openCursor();
request.onsuccess = function(event){
//handle success
};
An index can also create a special cursor that returns just the primary key for each record using the openKeyCursor() method, which accepts the same arguments as openCursor(). The big difference is that event.result.key is the index key and event.result.value is the primary key instead of the entire record.
var store = db.transaction("users").objectStore("users"),
index = store.index("username"),
request = index.openKeyCursor();
request.onsuccess = function(event){
//handle success
//event.result.key is the index key, event.result.value is the primary key
};
You can also retrieve a single value from an index by using get() and passing in the index key, which creates a new request:
var store = db.transaction("users").objectStore("users"),
index = store.index("username"),
request = index.get("007");
request.onsuccess = function(event){
//handle success
};
request.onfailure = function(event){
//handle failure
};
To retrieve just the primary key for a given index key, use the getKey() method. This also creates a new request but result.value is equal to the primary key value rather than the entire record:
var store = db.transaction("users").objectStore("users"),
index = store.index("username"),
request = index.getKey("007");
request.onsuccess = function(event){
//handle success
//event.result.key is the index key, event.result.value is the primary key
};
In the onsuccess event handler in this example, event.result.value would be the user ID.
At any point in time, you can retrieve information about the index by using properties on the IDBIndex object:
- name — The name of the index.
- keyPath — The property path that was passed into createIndex().
- objectStore — The object store that this index works on.
- unique — A Boolean indicating if the index key is unique.
The object store itself also tracks the indexes by name in the indexNames property. This makes it easy to figure out which indexes already exist on an object using the following code:
var store = db.transaction("users").objectStore("users"),
indexNames = store.indexNames,
index,
i = 0,
len = indexNames.length;
while(i < len){
index = store.index(indexNames[i++]);
console.log("Index name: " + index.name + ", KeyPath: " + index.keyPath +
", Unique: " + index.unique);
}
This code iterates over each index and outputs its information to the console.
An index can be deleted by calling the deleteIndex() method on an object store and passing in the name of the index:
var store = db.transaction("users").objectStore("users");
store.deleteIndex("username");
Since deleting an index doesn’t touch the data in the object store, the operation happens without any callbacks.
Concurrency Issues
While IndexedDB is an asynchronous API inside of a web page, there are still concurrency issues. If the same web page is open in two different browser tabs at the same time, it’s possible that one may attempt to upgrade the database before the other is ready. The problematic operation is in setting the database to a new version, and so calls to setVersion() can be completed only when there is just one tab in the browser using the database.
When you first open a database, it’s important to assign an onversionchange event handler. This callback is executed when another tab from the same origin calls setVersion(). The best response to this event is to immediately close the database so that the version upgrade can be completed. For example:
var request, database;
request = indexedDB.open("admin");
request.onsuccess = function(event){
database = event.target.result;
database.onversionchange = function(){
database.close();
};
};
You should assign onversionchange after every successful opening of a database.
When you are calling setVersion(), it’s also important to assign an onblocked event handler to the request. This event handler executes when another tab has the database open while you’re trying to update the version. In that case, you may want to inform the user to close all other tabs before attempting to retry setVersion(). For example:
var request = database.setVersion("2.0");
request.onblocked = function(){
alert("Please close all other tabs and try again.");
};
request.onsuccess = function(){
//handle success, continue on
};
Remember, onversionchange will have been called in the other tab(s) as well.
By always assigning these event handlers, you will ensure your web application will be able to better handle concurrency issues related to IndexedDB.
Limits and Restrictions
Many of the restrictions on IndexedDB are exactly the same as those for Web Storage. First, IndexedDB databases are tied to the origin (protocol, domain, and port) of the page, so the information cannot be shared across domains. This means there is a completely separate data store for www.wrox.com as for p2p.wrox.com.
Second, there is a limit to the amount of data that can be stored per origin. The current limit in Firefox 4+ is 50MB per origin while Chrome has a limit of 5MB. Firefox for mobile has a limit of 5MB and will ask the user for permission to store more than that if the quota is exceeded.
Firefox imposes an extra limitation that local files cannot access IndexedDB databases at all. Chrome doesn’t have this restriction. When running the examples from this book locally, be sure to use Chrome.
Offline web applications and client-side data storage are a big part of the Web’s future. Browsers now have the ability to detect when a user has gone offline and fire events in JavaScript so that your application can respond. The application cache allows you to specify which files should be made available offline. A JavaScript API is available for determining what the application cache state is and how it may be changing.
This chapter also covered the following aspects of client-side storage:
- Traditionally, such storage was limited to using cookies, small pieces of information that could be set from the client or server and transmitted along with every request.
- JavaScript provides access to cookies through document.cookie.
- The limitations placed on cookies make them okay for storing small amounts of data but inefficient for storing large amounts.
Internet Explorer provides a behavior called user data that can be applied to an element on the page as follows:
- Once applied, the element can load data from a named data store and make the information accessible via the getAttribute(), setAttribute(), and removeAttribute() methods.
- The data must be explicitly saved to a named data store using the save() method for it to persist between sessions.
Web Storage defines two objects to save data: sessionStorage and localStorage. The former is used strictly to save data within a browser session, because the data is removed once the browser is closed. The latter is used to persist data across sessions and based on cross-domain security policies.
IndexedDB is a structured data storage mechanism similar to an SQL database. Instead of storing data in tables, data is stored in object stores. Object stores are created by defining a key and then adding data. Cursors are used to query object stores for particular pieces of data, and indexes may be created for faster lookups on particular properties.
With all of these options available, it’s possible to store a significant amount of data on the client machine using JavaScript. You should use care not to store sensitive information, because the data cache isn’t encrypted.