Chapter 6. Documenting and versioning a Restlet application
Now that you’ve seen how to secure Restlet applications, it’s time to continue your exploration of the tasks required to get ready to roll out. This chapter covers how to document and version the RESTful web API of your Restlet application.
First we discuss the use cases, pitfalls, and recommendations when documenting and versioning your web API. Documenting is essential in helping a development team communicate during active development or maintenance and helping users learn how to use it. Versioning ensures that existing clients won’t break and alienate your user community.
We cover the Web Application Description Language (WADL) that’s well-suited for documentation of RESTful web APIs and the Restlet extension for WADL, including WadlApplication, WadlServerResource, and other artifacts. You learn how to progressively describe parts of a WADL document, such as the application title and description, the list of resources, their names, URI paths, methods, and representations. To illustrate the use of this extension you’ll reuse the sample RESTful mail application developed in previous chapters.
Finally you’ll see how easy it is to automatically convert the WADL documents into a user-friendly HTML document.
6.1. The purpose of documentation and versioning
This section explores the need for documenting and versioning your RESTful web APIs with common uses cases, pitfalls when tackling such a project, and best practices to follow.
6.1.1. Use cases
The first and most common use case is to provide human-readable documentation of your web API to developers on the client side so they can read it online or print it. After a learning phase they’ll continue to use it as a reference document while developing their client programs. Those developers are mainly looking for the URI entry points to your API resources, the supported methods, and a description of their representations.
The second use case is to coordinate among a large development team for construction or maintenance purposes. Imagine that you need to develop version 2.0 of an existing web API, without any previous knowledge. You’d naturally look for the equivalent of Javadocs for this API to learn not just how to use it but also how it was designed and implemented. This knowledge should be complementary to technical specifications and properly commented code as an essential deliverable of any RESTful web project.
The third use case is allowing the automatic generation of client programs, or at least part of them. This use case requires a formal and precise description of your REST API and is often criticized as reproducing the issues of the previous SOAP-based web services. But it can be provided for convenience purposes, in addition to a regular HTTP access.
Finally the need to maintain several versions of the same API occurs when the existing clients aren’t under the control of the project, such as for open web APIs, and when you don’t want to break clients by forcing them to use a newer API.
6.1.2. Pitfalls
In theory there’s no need to generate client SDKs for a RESTful web API because clients should automatically discover the API through hypermedia-driven navigation; nor is there a need to document it because it should be self-describing. A website is a good comparison: web browsers don’t need to read a user guide to display or interact with websites. This principle is hard to respect in practice, though, unless you limit yourself to standard hypermedia-driven media types such as HTML/XHTML, Atom/ AtomPub, or RDF (we discuss this topic in greater detail in chapter 10 when covering Restlet support for hypermedia and the Semantic Web).
But when you use your own XML- or JSON-based media types (for example, by using XStream or Jackson extensions provided by Restlet), you gain in terms of productivity. This is particularly true for the first phases of your project, where iterations are short and reactivity is key, but you’ll reduce the capacity of your clients to adapt to future changes of your API. Change is common among web APIs, even those labeled “REST,” but by creating new XML or JSON dialects you introduce coupling between clients and servers that can lead to brittle interfaces and the need for proper versioning to prevent evolution headaches. Also most if not all web APIs available today provide users with documentation to help learn and interact with them.
Another common problem you’ll face when documenting a web API is the need to keep it synchronized with the code developed. This is true during the elaboration and construction phases of a project, but also later on when evolutions are being pushed into production. Having documentation that doesn’t reflect the reality of your API will certainly cause trouble for your users.
Let’s review some best practices to help you solve or work around these issues.
6.1.3. Recommendations
If you’re developing an open web API that can be consumed by clients outside your control, the first recommendation is to reuse existing and proven hypermedia types such as HTML/XHTML, Atom/AtomPub, and RDF and extend them if necessary. This design choice will make your API more RESTful, easier to document and consume (higher-level clients for those media types exist in many environments), more reusable, and easier to evolve. But this choice comes with a development cost: you won’t be able to use transparent and bi-directional serialization mechanisms such as those provided by XStream, Jackson, JiBX, or JAXB.
If your API relies on custom media types not specifically designed with hypermedia and independent evolution of client and server in mind, the second recommendation is to add a version number to your URIs such as http://api.mycompany.com/v1.0/. This way you’ll be able to maintain older versions of your API while offering a new version in a parallel URI space for API enhancements.
If you use custom media types for productivity reasons, try to make it as hypermediacapable as possible, at least by providing hyperlinks to related resources. One way to do it is to add HREF attributes to XML elements, as in HTML. Hyperlinking, as opposed to forms and scripting, is rather easy to support, so make sure to use it extensively (for example, to point to items in a collection resource).
Regarding documentation, unless you target only browsers used by humans you’ll need to provide more formal documentation and keep it perpetually up to date, ideally by maintaining it close to your source code, as for Javadocs. If you target robots and other programmatic clients only, then consider using HTTP content negotiation to provide alternative HTML representations of your resources. Doing so will be useful for the developer of those clients to navigate, learn, and test your web API.
In HTTP, the standard way to request a description of a resource is via the OPTIONS method. If you invoke it on a root URI, or with the “*” special URI, it should return a description of the target URI subspace, typically of the whole application.
At this point you may feel like it adds a lot of work to follow those recommendations in a real project, so let’s look at how the Restlet Framework can help. In the next section we introduce WADL and explain how Restlet supports it to document your web APIs.
6.2. Introducing WADL
WADL is a useful XML vocabulary designed by Marc Hadley as a more RESTful way to describe web applications than its WSDL 1.1 predecessor, which had roots in the SOAP world. Note that WSDL 2.0 added better support for RESTful web APIs but still isn’t as elegant as WADL for our purpose. WADL was submitted in 2009 to the W3C by Sun Microsystems [4].
To get more concrete, the following listing shows a simple WADL example for the mail application developed in previous chapters. Though the example is small, it already provides useful information, such as the list of resources, their URIs, and the supported HTTP methods.
Listing 6.1. Sample WADL description
This example is pretty simple, but WADL can also be used to describe expected request and response messages, including query parameters and representation media types. Let’s now explore the Restlet extension for WADL provided in the org.restlet.ext.wadl.jar file. This extension is composed of a set of description classes such as ApplicationInfo, ResourcesInfo, ResourceInfo, RequestInfo, and so on corresponding to the WADL information model. In addition the WadlRepresentation class can be used to either parse an existing WADL document or generate one based on a given ApplicationInfo or ResourceInfo instance.
The two most useful classes, WadlApplication and WadlServerResource, can help to produce WADL descriptions.
6.3. The WadlApplication class
The most common use case for this extension is to describe a complete Restlet application as a WADL document. That’s the purpose of the WadlApplication class, which extends org.restlet.Application (illustrated in figure 6.1) and intercepts incoming calls on the base URI of the application with the OPTIONS method. It provides several protected methods, such as createWadlRepresentation(ApplicationInfo), that can be extended to customize their default behavior, even though in most cases you won’t need to look into this.
Figure 6.1. The WadlApplication class can be used to describe an application in WADL.
Imagine that you want to describe the API of the sample mail application developed in chapter 3. You update the MailServerApplication to make it extend Wadl-Application and see how it behaves. To test this let’s write a simple client program that outputs the WADL document retrieved on the console.
public static void main(String[] args) throws Exception {
ClientResource service = new ClientResource("http://localhost:8111");
System.out.println(service.options().getText());
}
You could also use any kind of HTTP client, such as curl or a web browser via the tunnel service, using the URI http://localhost:8111?method=options. If you first launch the MailServerComponent, including the modified MailServerApplication, and then test the client program, the console will display (as you may have guessed) the same WADL document as the one in listing 6.1. With little effort, you already have a useful result!
By automatically introspecting the application, the WadlApplication class was able to detect the available resources and their URI templates, the methods supported by each, and the request and response representations supported. Note that the name and description properties of the application are also automatically extracted into the WADL documentation in the root application element.
This introspection process is capable of recursively traversing filters and routers until it reaches a leaf ServerResource subclass and then instantiates them—which, by the way, can help you detect implementation issues. As you know, other sorts of leaves can be found in a Restlet routing graph, typically Restlet instances. For those situations, to provide a description of those leaves in the resulting WADL document, you can either have the object implement the WadlDescribable interface or wrap it into a WadlWrapper instance. You’ll then only have to override the getResourceInfo() method and build your own WADL resource descriptor using the ResourceInfo class available in the WADL extension.
Next we introduce the WadlServerResource class to further describe each resource of the mail application in WADL.
6.4. The WadlServerResource class
Even though the WadlApplication can already discover quite a bit of useful information on your server resources, there’s a limit to what can be guessed during introspection. To go beyond this limit, you need to explicitly provide additional information. For this purpose the WADL extension includes WadlServerResource, a subclass of ServerResource, which can be extended to describe anything that’s supported by the WADL specification.
In this section we introduce the class and its properties, update the server resources from the sample mail application to improve their description, and illustrate how descriptions of a single resource can also be retrieved.
6.4.1. Overview of properties and methods
To discover this essential class of the WADL extension, you will first get an overview of its main properties and methods. We already covered the ones inherited from ServerResource in section 2.5. For this purpose we illustrate this class in figure 6.2 as a UML class diagram, including its three properties and its sets of methods.
Figure 6.2. The WadlServerResource class can be used to further describe resources in WADL.
First the autoDescribing Boolean property indicates whether the resource should directly support the OPTIONS method. By default it’s enabled and allows a client to retrieve a WADL description snippet only for the resource that’s the target of the request, and not the whole web API. We illustrate this possibility later in the section.
The two other name and description properties allow easy documentation of your resources with the possibility of using dynamic information from the request. Note that this dynamic aspect wouldn’t be possible if you used regular Javadocs comments or special Java annotations.
Now to get an overview of the class methods. As you can see, many describe*() methods should be viewed as callbacks to be invoked by a parent WadlApplication, giving you a chance to customize the values returned by default.
For example if you merely want to update the description of the GET method with additional documentation, you need to override only describeGet(MethodInfo), invoking super.describeGet(methodInfo) (unless you want to completely bypass the default behavior), and then add your own values to the methodInfo parameter.
Two methods, createHtmlRepresentation(ApplicationInfo) and createWadl-Representation(ApplicationInfo), allow you to customize the WADL and HTML representation generated when OPTIONS is directly invoked on a WadlServerResource subclass. Finally, canDescribe(Method) allows you to remove the description of a specific method—for example, if the user doesn’t have the required role.
For further explanations, including on the default behavior, refer to the Javadocs. You’ll now put this useful class into practice.
6.4.2. Improving description of existing server resources
To improve the description of the sample mail application, you’ll make the three server resources extend the WadlServerResource class instead of ServerResource. Start with the AccountServerResource and reuse the accountID attribute in the name and description properties, as illustrated in the following listing.
Listing 6.2. WADL-enhanced account resource with a dynamic name and description
The following listing continues updating the sample resources to provide static documentation for the AccountsServerResource class.
Listing 6.3. WADL-enhanced accounts server resource with static documentation
public class AccountsServerResource extends WadlServerResource implements
AccountsResource {
private static final List<String> accounts
= new CopyOnWriteArrayList<String>();
@Override
protected void doInit()throws ResourceException {
setName("Mail accounts resource");
setDescription(
"The resource containing the list of mail accounts");
}
public static List<String> getAccounts(){
return accounts;
}
public String represent(){
StringBuilder result = new StringBuilder();
for (String account : getAccounts()) {
result.append((account == null) ? "" : account).append('
'),
}
return result.toString();
}
public String add(String account) {
getAccounts().add(account);
return Integer.toString(getAccounts().indexOf(account));
}
}
Finally you take care of the RootServerResource in listing 6.4, which provides a name and a description but also sets the autoDescribing property to false. Why do this? If you remember, WadlApplication is intercepting OPTIONS calls to the root URI of the application to automatically describe the whole web API exposed to clients. This action is done when a client requests the root URI of the application and there is no other reply by underlying resources.
If you let the RootServerResource autodescribe itself in reply to OPTIONS calls, then you’d only be able to retrieve the snippet covering the root resource, not the whole API description.
Listing 6.4. WADL-enhanced root server resource
public class RootServerResource extends WadlServerResource implements
RootResource {
@Override
protected void doInit() throws ResourceException {
setAutoDescribing(false);
setName("Root resource");
setDescription("The root resource of the mail server application");
}
public String represent() {
return "Welcome to the " + getApplication().getName() + " !";
}
}
Now that you’ve associated a name and description to each resource, let’s see how you could describe representations more precisely than with a text/plain media type. For this, you’ll override some of the describe*() methods provided by the WadlServerResource. First override the describe(ApplicationInfo) method in AccountsServerResource to add a global description of the textual representation that you use for accounts, as illustrated in the following snippet. Note that it sets an account identifier on this representation description to be able to reference it in other descriptions:
@Override
protected void describe(ApplicationInfo applicationInfo) {
super.describe(applicationInfo);
RepresentationInfo rep =
new RepresentationInfo(MediaType.TEXT_PLAIN);
rep.setIdentifier("account");
applicationInfo.getRepresentations().add(rep);
DocumentationInfo doc = new DocumentationInfo();
doc.setTitle("Account");
doc.setTextContent("Simple string containing the account ID");
rep.getDocumentations().add(doc);
}
Then you want to describe the fact that both AccountsServerResource and AccountServerResource rely on this textual media type to represent an account in the sample application. For this purpose you’ll override another describe(...) method as illustrated in this snippet:
@Override
protected RepresentationInfo describe(MethodInfo methodInfo,
Class<?> representationClass, Variant variant) {
RepresentationInfo result = super.describe(methodInfo,
representationClass, variant);
result.setReference("account");
return result;
}
Finally you want to describe the remaining representation type returned by the GET method of RootServerResource. Again, you override the same method, but because this representation isn’t reused elsewhere, it’s directly described:
@Override
protected RepresentationInfo describe(MethodInfo methodInfo,
Class<?> representationClass, Variant variant) {
RepresentationInfo result = super.describe(methodInfo,
representationClass, variant);
result.setMediaType(MediaType.TEXT_PLAIN);
result.setIdentifier("root");
DocumentationInfo doc = new DocumentationInfo();
doc.setTitle("Mail application");
doc.setTextContent("Simple string welcoming the user " +
"to the mail application");
result.getDocumentations().add(doc);
return result;
}
It’s been a while since you tested the WADL description of the application, so launch the test client again. The result displayed in the following listing shows some nice improvements to the documentation.
Listing 6.5. Improved WADL application description
<?xml version="1.0" standalone="yes"?>
<?xml-stylesheet type="text/xsl" href="wadl2html.xslt"?>
<application xmlns="http://wadl.dev.java.net/2009/02">
<doc title="RESTful Mail Server application">
Example application for 'Restlet in Action' book
</doc>
<representation id="account" mediaType="text/plain">
<doc title="Account">Simple string containing the account ID</doc>
</representation>
<resources base="http://localhost:8111/">
<resource>
<doc title="Root resource">
The root resource of the mail server application
</doc>
<method name="GET">
<response>
<representation id="root" mediaType="text/plain">
<doc title="Mail application">
Simple string welcoming the user to the
mail application
</doc>
</representation>
</response>
</method>
</resource>
<resource path="accounts/">
<doc title="Mail accounts resource">
The resource containing the list of mail accounts
</doc>
<method name="GET">
<response>
<representation href="#account"/></response>
</method>
<method name="POST">
<request>
<representation href="#account"/></request>
<response>
<representation href="#account"/></response>
</method>
</resource>
<resource path="accounts/{accountId}">
<doc title="Mail account resource">
The resource describing a mail account</doc>
<method name="DELETE"/>
<method name="GET">
<response>
<representation href="#account"/></response>
</method>
<method name="PUT">
<request>
<representation href="#account"/></request>
</method>
</resource>
</resources>
</application>
Let’s see if it’s as easy to describe a single resource instead of the whole application.
6.4.3. Describing a single resource
Earlier in the chapter we mentioned describing a single resource instead of the whole application. You can now demonstrate this feature without any additional development. Modify the test client to address the collection of accounts and see the result:
public static void main(String[] args) throws Exception {
ClientResource service = new ClientResource("http://localhost:8111");
System.out.println(service.getChild("/accounts/").options().getText());
}
In the following listing you can see the resulting WADL description that should be displayed in your console.
Listing 6.6. WADL resource description snippet
<?xml version="1.0" standalone="yes"?>
<?xml-stylesheet type="text/xsl" href="wadl2html.xslt"?>
<application xmlns="http://wadl.dev.java.net/2009/02">
<doc title="Mail accounts resource"/>
<representation id="account" mediaType="text/plain">
<doc title="Account">Simple string containing the account ID</doc>
</representation>
<resources>
<resource path="accounts/">
<doc title="Mail accounts resource">
The resource containing the list of mail accounts
</doc>
<method name="GET">
<response>
<representation href="#account"/></response>
</method>
<method name="POST">
<request>
<representation href="#account"/></request>
<response>
<representation href="#account"/></response>
</method>
</resource>
</resources>
</application>
You can also override other WadlServerResource methods, such as describeGet (MethodInfo) to describe the GET methods; see figure 6.2 and Javadocs of the WADL extension for details.
So far we’ve retrieved WADL descriptions in their original XML form, but that’s not easily readable by a human using a browser to learn about your cool web API. There’s another built-in feature of this WADL extension: the ability to automatically convert WADL documents to an HTML equivalent.
6.5. Automatic conversion to HTML
The WADL documents you’ve produced so far are nice for formal descriptions or when those descriptions need to be automatically processed, but in general your users will look for something more user-friendly, typically a web page.
For this purpose the WADL extension embeds an XSLT document that was developed by Marc Nottingham from Yahoo! This XSLT transformation can be automatically applied by the WadlApplication and WadlServerResource classes, relying on standard HTTP content negotiation.
Note
Because this WADL-to-HTML stylesheet uses the nonstandard EXSLT functions library, you need to have a recent version of Xalan-Java in your classpath instead of the default one bundled in Java SE. Version 2.6.0 has been tested successfully, but version 2.4.1 and above should work fine as well.
If you open http://localhost:8111/?method=options in your browser, you’ll see the web page in figure 6.3. Try http://localhost:8111/accounts/?method=options to see an HTML snippet describing the AccountServerResource only. The application knows it has to return HTML due to the browser preferences set in the Accept HTTP header.
Figure 6.3. The partial WADL documentation converted to HTML
Note that the method query parameter is interpreted by the TunnelService discussed in previous chapters and allows the sending of OPTIONS requests from your browser. If you want to use the GET method instead, override the WadlApplication.handle(Request, Response) method and invoke the WADL generation logic directly via the wadlRepresent(Request, Response) method.
With no further development you can provide a clear documentation of the API that will always stay synchronized with your source code.
The WADL extension can also use a WADL document—for example, provided by a third-party tool—as an input for configuring a Restlet application. This isn’t as powerful as the XML configuration mechanism introduced in chapter 3, especially a configuration based on Spring, but this can come in handy in some situations. For additional details, have a look at the constructors of the WadlComponent and Wadl-Application classes.
6.6. Summary
This chapter explored how to document and version a RESTful web API exposed by a server-side Restlet application. We presented common use cases, such as the need to learn an API and have a reference document to look at when using it, or the need to coordinate among a large team of developers. Versioning is also used when the web API or the technical environment can’t transparently handle the independent evolution of clients and servers.
You also saw main pitfalls such as coupling too many clients and servers in opposition to REST principles and recommendations such as using existing media types like HTML, Atom, or RDF built to respect REST’s hypermedia principle.
This led you to WADL and its support in the Restlet Framework. You learned how to use the WadlApplication class as an alternative parent class to org.restlet.Application in order to automatically describe the existing sample mail application. You then used the WadlServerResource class and saw how it complements Wadl-Application to provide more detailed descriptions of resources and representations that constitute your API and to generate a snippet description for a single resource instead of the whole API.
Finally you saw how to convert the XML documents produced by the Restlet extension for WADL into user-friendly HTML documents that display nicely in a web browser.
The second part of this book ends with a chapter packed with Restlet recipes and best practices. Chapter 7 discusses handling of web forms, cookies, static files, error pages feeds, and redirections, as well as how to improve performance and how to modularize your Restlet application to make it ready for production rollout.