Web APIs

Unlike humans, code doesn’t have a pair of eyes that want to read an article or view a picture. Code needs to view that “something” in a format that it can read (parse). This is where a data interchange format (like JSON, shown in Example 6-1) steps into the picture.

{
    "dt": 1433383200,
    "temp": {
        "day": 293.5,
        "min": 293.5,
        "max": 293.5,
        "night": 293.5,
        "eve": 293.5,
        "morn": 293.5
    },
    "pressure": 1015.06,
    "humidity": 98,
    "weather": [
        {
            "id": 802,
            "main": "Clouds",
            "description": "scattered clouds",
            "icon": "03n"
        }
    ],
    "speed": 2.86,
    "deg": 134,
    "clouds": 44
}

The JSON weather data example can be “read” by any code capable of parsing JSON. This JSON resource can be requested with a URL (the code example is a subset of the full JSON document):

http://api.openweathermap.org/data/2.5/forecast/daily?lat=35&lon=139&cnt=10&mode=json

Though many public web APIs like OpenWeatherMap are for “reading,” many APIs such as the PayPal API are more interactive. A web API is a set of instructions and standards for interacting with a service over HTTP. That interaction can include create, read, update, and delete (CRUD) operations and the web API will have a reference outlining these instructions and standards.

For example, according to the PayPal API reference, I can create a new invoice with the PayPal API by posting JSON to the URL:

https://api.sandbox.paypal.com/v1/invoicing/invoices

In Example 6-2, we have JSON representing an invoice to be sent as a request to the PayPal API.

{
    "merchant_info": {
        "email": "[email protected]",
        "first_name": "Bob",
        "last_name": "Bobberson",
        "business_name": "Bob Equipment, LLC",
        "phone": {
            "country_code": "001",
            "national_number": "5555555555"
        },
        "address": {
            "line1": "123 Fake St.",
            "city": "Somewhere",
            "state": "OR",
            "postal_code": "97520",
            "country_code": "US"
        }
    },
    "billing_info": [
        {
            "email": "[email protected]"
        }
    ],
    "items": [
        {
            "name": "Widgets",
            "quantity": 20,
            "unit_price": {
                "currency": "USD",
                "value": 89
            }
        }
    ],
    "note": "Special Widgets Order!",
    "payment_term": {
        "term_type": "NET_45"
    },
    "shipping_info": {
        "first_name": "Some",
        "last_name": "Guy",
        "business_name": "Not applicable",
        "address": {
            "line1": "456 Real Fake Dr",
            "city": "Some Place",
            "state": "OR",
            "postal_code": "97501",
            "country_code": "US"
        }
    }
}

With the PayPal API, once the invoice is created, I can request (read), update, and delete that invoice.

Operations done behind the scenes in JavaScript, such as the request for the weather data, are referred to as asynchronous. Asynchronous operations are operations that take place in the background without interrupting the main transmission.

In the case of JavaScript asynchronous operations, the “main transmission” would be the display of the web browser. For example, a news web page might include a sidebar with real-time weather data. While you’re reading your news article, code in the background could asynchronously update the weather display every 60 seconds. This operation would not require the page to reload, nor would it interrupt your page scrolling to read your article. The only thing on the page to change would be the sidebar with the weather data.

The asynchronous (background) operations of JavaScript are referred to as AJAX. AJAX stands for Asynchronous JavaScript and XML. When we are making behind-the-scenes requests for JSON, this would technically be Asynchronous JavaScript and JSON (AJAJ). However, the term “AJAX” has been used for so long that it has become less of an acronym and more of a word to describe any asynchronous operations in JavaScript. Here in this book and elsewhere, you will see the term AJAX used to describe Asynchronous JavaScript and JSON. This is not done in error.

Let’s take a look at how we can achieve AJAX with JSON and the JavaScript XMLHttpRequest.

The JavaScript XMLHttpRequest

Though the JavaScript XMLHttpRequest sounds like it has to do with XML, we use it for making HTTP requests. During the time it was named with XML in it, XML was the star data interchange format for making these types of requests. However, XMLHttpRequest is not restricted to XML. We use it to request JSON despite its name.

Earlier, I began several phrases with “Code that fetches.” The JavaScript XMLHttpRequest is code that fetches a resource. This phraseology is common when talking about HTTP requests, though in reality the code isn’t chasing through cyberspace like a dog fetching a ball. The code sits where it is and politely asks for the resource to be delivered.

If we return to the restaurant example, we can imagine the JavaScript code sitting at a restaurant table. The JavaScript code doesn’t go to the kitchen to request JSON, nor does it yell across the restaurant. There is protocol in a restaurant. That protocol requires you to summon a waiter who writes down your order and carries it to the kitchen.

There is also protocol that allows code to send requests to web servers that may be hundreds or thousands of miles away. The very foundation of data communication that allows us to visit our favorite websites is based on a protocol called HyperText Transfer Protocol (HTTP). When we type http://www.cutelittlekittens.com into our Internet browser, we are using the HyperText Transfer Protocol to make a request for a resource. In the case of the cute kittens website, that resource is an HTML page with cute kitten images embedded into it. Code that needs to make a request for a JSON resource uses the same protocol (HTTP).

In JavaScript, the code that uses the protocol to make this request is the XMLHttpRequest. JavaScript is an object-oriented language, so naturally the XMLHttpRequest is an object. Once it is created by using the syntax new XmlHttpRequest() (Example 6-3), and assigned to a variable, it has functions that can be called to request resources from a location.

var myXmlHttpRequest = new XMLHttpRequest();

The XMLHttpRequest is an object, and Example 6-3 shows the creation of a new XMLHttpRequest object. Even if you don’t know JavaScript, if you’ve read the book up to this point, your knowledge of JSON will help you visualize a JavaScript object. After all, JSON is based on the literal notation of JavaScript objects.

We say that a JavaScript object has properties. These “properties” are simply name-value pairs. The properties of the XMLHttpRequest are named onreadystatechange, readyState, response, responseText, responseType, responseXML, status, statusText, timeout, ontimeout, upload, and withCredentials. Note the inconsistency on the casing of these properties. Some use camelCase and others are all lowercase. This may cause hairpulling in your future if you decide to write your own XMLHttpRequest code, as you will not only need to remember what the properties are called, but the different casing.

One of the key differences between a typical object in programming and a JSON object is that JSON lacks executable instructions. JSON is composed only of properties because it is meant for data interchange. A typical object in programming will also include functions (or methods). For example, I can describe my shoes in JSON with name-value pairs such as: "color": "pink" and "brand": "crocs". What I cannot do in JSON is give life to those shoes, by calling a function (or method) such as shoe.walk().

The available functions of the XMLHttpRequest that we are most interested in are:

• open(method, url, async (optional), user (optional), password (optional))
• send()

The available properties of the XMLHttpRequest that we are most interested in are:

onreadystatechange

Has a value of a function that we can define in our code.

readyState

Returns a value ranging from 0–4 that represents a status code.

status

Returns the HTTP status code (like 200 for success).

responseText

When the response is successful, this will contain the body of the response as text (JSON if this is what we are requesting).

For those of you not familiar with JavaScript, here is a mind-bender: a property can have a function for a value. This is possible in JavaScript because a function is an object. An object is a type of data, and therefore can be assigned to a variable (property), modified, and passed around. In the programming world, these are called first-class functions. The onreadystatechange property has a value of a function.

Example 6-4 creates a new XMLHttpRequest object and gets JSON from the OpenWeatherMap API.

var myXMLHttpRequest = new XMLHttpRequest();
var url = "http://api.openweathermap.org/data/2.5/weather?lat=35&lon=139";

myXMLHttpRequest.onreadystatechange = function() {
    if (myXMLHttpRequest.readyState === 4 && myXMLHttpRequest.status === 200) {
        var myObject = JSON.parse(myXMLHttpRequest.responseText);
        var myJSON = JSON.stringify(myObject);
    }
}
myXMLHttpRequest.open("GET", url, true);
myXMLHttpRequest.send();

In the example, on the second line of code I’m creating a variable that contains the URL for a JSON resource. I then create a function and assign it to the onreadystatechange property of myXMLHttpRequest. This function will be executed every time the readyState property changes. In the function, I check to see if the readyState is 4 (code for “done”) and that the HTTP status is 200 (code for success). If both of those things are true, I then parse the JSON into a JSON object.

Two terms that you will often hear about when JSON is turned into text from an object and then from an object back into text are serialization and deserialization. Serialization is the act of converting the object into text. Deserialization is the act of converting the text back into an object.

In Example 6-5, the JavaScript is deserializing with JSON.parse(). In the responseText, the JSON is simply text as a data interchange format. Once it is parsed by JSON.parse(), it is no longer JSON, but a JavaScript object.

var myJSON = JSON.parse(myXMLHttpRequest.responseText);

This deserialization with JSON.parse() is necessary because the JSON is not yet an object. Remember that JSON stands for JavaScript Object Notation. While it is in its JSON form, it is a literal representation of an object in the form of text. In order for JSON to become a real object, it must be deserialized. In JavaScript, we can also serialize the JSON with JSON.stringify().

In Example 6-6, the myObject variable gets the deserialized JSON. This is now an object. The myJSON variable gets the serialized object. This is now JSON.

// the JSON response deserialized 
var myObject = JSON.parse(myXMLHttpRequest.responseText); 

// The object serialized
var myJSON = JSON.stringify(myObject); 

Finally, the last two lines in my example are setting up the request and then sending it via the HTTP protocol (Example 6-7).

myXMLHttpRequest.open("GET", url, true);
myXMLHttpRequest.send();

You may find it strange that the code that handles the JSON response appears before the request is sent. I even talked about it before I talked about the request being set up and sent. The function value of the onreadystatechange property is an EventHandler. The underlying JavaScript engine (not my code) has logic to access the properties value (my function) every time the ready state changes. The ready states from 0–4 are:

0 for UNSENT

The state before the open() function has been executed.

1 for OPENED

The state after the open() function has been executed but before send() has been executed.

2 for HEADERS_RECEIVED

The state after the send() function has been executed and headers and status are available.

3 for LOADING

The headers have been received but the response text is still being retrieved.

4 for DONE

Complete; the full message with headers and body has been received.

In my example, I used a URL that belongs to the OpenWeatherMap API. This is a web service that provides weather data through an API. The API has URLs for both XML and JSON resources for a variety of weather data. My example showed how I requested and received JSON from this API on the client side with JavaScript. However, Internet browsers have rules about sharing for security purposes. The same-origin policy states that a website should only have these sort of background requests with sites on the same domain. This is to protect users because unless they know what Internet browser developer tools are, they can’t see these requests happening.

The code I shared in my example could be executed in a script on a site with the URL http://www.mycoolsite.com. This is the origin domain because it is the URL of the original request. The URL for the API is:

http://api.openweathermap.org/data/2.5/weather?lat=35&lon=139

The domain http://api.openweathermap.org does not match the origin domain of http://www.mycoolsite.com. This clearly violates the same-origin policy. This background request and response is only made possible by some hoops that the makers of the OpenWeatherMap API jumped through. Let’s take a look at these hoops and loopholes to see how behind-the-scenes requests with public APIs are made possible.

Relationship Woes and Rules About Sharing

I realize I left you on a cliffhanger back there. I just showed you some code that does work, but violates same-origin Internet browser rules. It shouldn’t work, but it does, only because of a hoop that the makers of the OpenWeatherMap API jumped through. Sounds like a bad soap opera, where two people can’t make up their minds what their relationship is. Will the browser allow the request to the public web API? Stay tuned.

Access-Control-Allow-Credentials:true
Access-Control-Allow-Methods:GET, POST
Access-Control-Allow-Origin:*

Access-Control-Allow-Methods:POST
Access-Control-Allow-Origin:http://www.somebank.com

getTheAnimal(
    {
        "animal": "cat"
    }
);

function getTheAnimal(data) {
    var myAnimal = data.animal; // will be "cat"
}

var script = document.createElement("script");
script.type = "text/javascript";
script.src = "http://notarealdomain.com/animal.json";
document.getElementsByTagName('head')[0].appendChild(script);

script.src = "http://notarealdomain.com/animal.json?callback=getThing";

getThing(
    {
        "animal": "cat"
    }
);

Key Terms and Concepts

This chapter covered the following key terms:

Web API

A set of instructions and standards for interacting with a service over HTTP.

XMLHttpRequest

A JavaScript object, primarily used in AJAX programming, that retrieves data from a URL without reloading the page.

HyperText Transfer Protocol (HTTP)

The protocol that is the very foundation of data communication for the World Wide Web.

Serialization

The act of converting the object into text.

Deserialization

The act of converting the serialized text back into an object.

Same-origin policy

The Internet browser requires scripts to originate from the same domain for security.

Cross-origin resource sharing (CORS)

Allows resources (e.g., a JSON document) to be requested from another domain outside of the domain of the requestor by defining permission in the response headers.

JSON with padding (JSON-P)

Uses the <script> tag exemption from the same-origin policy to request JSON from servers without the same origin.

We also  discussed these key concepts:

• The relationship between the JavaScript XMLHttpRequest and a web API is a client–server relationship.
• XMLHttpRequest isn’t just for XML. We use it to request JSON resources as well.
• A website caters to humans, and a web API caters to code. Both serve resources over HTTP.
• The same-origin policy can cause relationship woes for client–server interactions between JavaScript and a JSON resource.
• A cross-domain XMLHttpRequest by a client is dependent on the server of the JSON resource for success.