Web services have had a lot of publicity proclaiming them the new “new thing”, but they haven't yet achieved the universal use that would justify that publicity. There are several reasons behind that. One factor was the end of the Y2K upgrade spending, followed a few months later by the collapse of dot-communism. Together these led to a multi-year world-wide recession in the computer industry. Companies have been deferring new IT investments where they can.
In other words, the fact that web services aren't yet ubiquitous may have more to do with the economic climate than with the technology. But another factor is that web services often aren't secure. There is no trivial universally accepted security solution. You can prevent people snooping on the bits by encrypting them using SSL (Secure Sockets Layer, as used in https: protocols everywhere). But we also need a way to ensure that only authorized clients call the server. There are several alternatives for this (such as authentication tokens), but industry has yet to converge on one.
Regardless, organizations are now increasing their use of web services. Two of the premier websites on the internet, Google and Amazon, have launched web services Beta programs. These two initiatives are not related to each other, and they have taken slightly different technical approaches.
The goals of the two Beta programs are the same though. Amazon and Google are both predicting that web services are going to grow greatly in significance.
Amazon and Google are planting a flag in the ground, saying that they intend to be part of that growth, and help shape the future of web services. For you as a programmer, the benefit is getting early exposure to a technology that's got a great future.
In this chapter we'll give you a clear picture of what web services are, and the problems they can solve for an organization. We will show how client programs send XML requests to web services. The work that a web service server does to fulfill a request is beyond our scope here. We will look at some XML that is sent back in response, but not how it is generated or transformed. In the second section, we'll look at the Google web services beta, and walk through an example that uses it. The Google package is quite reasonable, and most programmers could figure it out for themselves. It saves your time, if I put the steps together in a list, and point out the pitfalls. We'll end up with a way to do google searches from inside one of your programs.
The third section describes the beta program for Amazon web services. There are some enormously talented people working hard at Amazon to bring the program to you free of charge. But, ironically for a company working in the books-and-information-online business, Amazon's documentation isn't good enough and there are errors in the beta release that prevent it from working out of the box. Overall Amazon has done a good job though.
The current form of this chapter will have a limited lifetime, starting in June 2004. Beta programs come to an end. My belief is that Amazon and Google and many other companies will soon establish permanent web services interfaces. At that point I'll update the chapter, and may put it on my website at afu.com/jj6. Let's get started by describing what web services in general do for you.
Chapter 27 described XML. That mark-up language lets you label pieces of data with a description of what they are. If you give an XML file to another program (including a program outside your organization) it can easily find within the file the pieces in which it is interested. Web services consist of XML files, plus a way of sending them to a server that will execute a program (probably a servlet) based on the request in the XML, and send back the results.
If you've never worked in IT you may doubt this, but if you have worked in IT, you'll immediately recognize it as true. A significant IT expense for large organizations is the cost of moving data from one system to another. You store data for system A in a database, then requirements change and some of the data needs to be shared with System B in real time. But systems A and B are hosted on different computers, and use databases from different vendors. Just getting them to agree on the format of a calendar date is a problem. Actually moving the bulk data reliably and repeatedly is an expensive and fragile undertaking.
Some companies try to avoid the costs of data migration by moving everything into a gigantic data repository known as a data warehouse. That has some advantages and some drawbacks of its own. Other companies, like Vitria Inc., have built a substantial business out of providing software channels that reliably feed data from one system to another.
Today's browser-based web architecture has two potential improvements for business data use:
-
Get rid of manual steps. At present, people read web pages. In a business environment, you want to eliminate manual steps. They are error-prone and can cause delays. If web pages are being used for data retrieval (e.g. inventory levels), it is desirable to turn the person-to-computer process into a computer-to-computer process.
-
Integrate “islands of data”. When data comes from several websites (e.g. comparing the cost and timing of a trip by rail versus airplane) a person has to take notes, summarize and re-package the data. It can introduce clerical errors, and does not scale well when the number of alternatives or trips increases.
People have written “screen-scraper” programs that address these issues, but each one of those is a custom job. Screen-scrapers work in intranets, but may need modification when they poll external websites that change. Now that the world is connected by the internet, there has to be a better way to publish and move data.
Anything that automates the flow of data can directly reduce IT costs. For example, a company can permit trusted suppliers view its internal inventory levels, so suppliers can replenish stocks automatically under a just-in-time delivery system. Web services provide the basic messaging and service-description functions for this kind of electronic relationship.
Bill Gates recently described the infrastructure needed for web services. He identified four necessary pieces:
-
The hardware and software for communication. The internet provides this.
-
A standard way for services to describe what they do, and to find each other on the internet. Web Services Description Language (WSDL) supplies the first, and UDDI, the Universal Description Discovery and Integration standard, the second.
-
A common format for data. XML provides this.
-
A protocol for servers and clients to exchange XML messages with each other. There are two or three contenders here: REST (fast, simple, but basic), SOAP (slower, more complex, but more capable), and XML-RPC (somewhere in the middle between REST and SOAP). Like the email protocols or the FTP protocol, a web services protocol says “If you talk to me on this port, and say these kinds of sentences, then I'll understand what you need and give it to you.”
This is the promise of web services: it moves the web beyond being a gigantic reading room, and lets it become an API for client/server or peer-to-peer programs. By using XML over HTTP connections, the client and server are completely decoupled. They don't have to match in hardware, architecture, or operating system.
When Bill Gates outlined these components for web services, the client/server independence was a major disadvantage for him. All his past software frameworks (like COM, DCOM, and .Net) have been Windows-only. Microsoft supports web services, even though they do not reinforce the Microsoft monopoly, because they give Bill something else he really wants. Web services encourage the move from “software as product” to “software as service”, where apps-on-tap software can be accessed remotely and rented per-use or per-time-period.
Deutsche Telekom uses Web Services today to connect about 100 content providers with 5 million mobile phone service subscribers. By 2015, when Moore's Law will make it economically feasible, tiny computers may run the light switches, pet doors, and fridges in our homes. They may all have IP addresses, and they may all use web services to talk to other authorized systems.
Information architects have great hopes that web services will one day make it easy to create aggregate systems by combining smaller web services. People always have these kind of hopes for every new software technology. Web services are strong enough to stand on their own regardless of whether this particular ambition is realized.
This chapter will show you how to write a client program that makes calls to a web service. Let's list the steps to be done by such a program. The service could be as simple as doing a Google search. The point here is that, as a web service, the search results come back in a format which is meant to be easy for a program to use. If a person is doing a web search interactively, use a browser, not a web service.
-
Identify the server that offers the resource that you want. This may be done by querying using UDDI, the Universal Description Discovery and Integration standard.
-
Obtain the XML file (written in Web Services Description Language) that describes how to make a request to the server. The description will include the URL, and what kind of fields are needed in the XML file that will be sent over and in the answer that will be sent back.
-
Run the WSDL file through a tool that converts it into a library of Java calls. The premier tool for doing this translation is Axis—the Apache Extensible Interaction System. Axis is open source, and can do a lot of other things as well.
-
Write your code to call into the library produced by Axis. Your code will provide a number of parameters. The library will assemble these into an XML file, put a SOAP (Simple Object Access Protocol) envelope around it, and send it via an HTTP GET or POST to the web service URL.
-
The web service will process the SOAP request, create an XML file with the results, and send it back to your system. In the case of Google, this will be a search request. In the case of Amazon, this will be information about a book, or the confirmation of a step in completing a book purchase.
-
The web services client library on your system will handle the incoming XML, and pass it on to your code as a return value. You may get raw XML, which you can further process using XSLT—Extensible Stylesheet Language Transformations. XSLT is a way of converting XML into HTML or plain text or XML that uses another vocabulary. Or the library may do all the work of converting the XML response into something your program finds easier to deal with.
This is the theory, and in practice some of these steps may not be necessary. If you already know that you intend to search Google or Amazon, you don't need a UDDI enquiry. If you want to find several alternative providers, e.g. to convert some foreign currency, a UDDI enquiry will tell you who offers the service.
The profusion of evolving standards, and the consequent alphabetti spaghetti, present a formidable barrier to a programmer trying to master this technology. Realizing this, both Amazon and Google provide some client software that does some or most of the hard parts. That's quite likely to become a standard feature of web services in future. With this theoretical background in mind, let's go on to look at the Google web service.