There are many compelling reasons to use Web services. It seems as if everyone is at least playing with Web services. Almost every software vendor is building support for Web services into its platforms, languages, and tools. Web services enable any-to-any integration, supporting any programming language, any runtime platform, and any network transport. Technologies such as SOAP and WSDL are simpler to use than traditional integration middleware technologies, and they offer much more flexibility. When combined with domain-specific industry standards, Web services enable unprecedented dynamic interaction. More to the point, they can make it easier for your partners and your customers to do business with you. Best of all, the low cost, pervasiveness, and simplicity of the technology lets your existing staff do more with less.
Web services can be used for many types of applications. Nearly every application requires some integration effort, so I'm sure you can find a way to use Web services in almost any project. After you get a little experience under your belt, you'll very likely adopt Web services as a standard integration technology. When you're just getting started, though, it's always best to limit your scope. Start small. Spend some time learning about the technology. Then you can apply your learning to larger projects. One nice feature of Web services is that you can use them incrementally. There's no need to tackle an enormous project all at once. I recommend that you use Web services in places where they are likely to have a big impact.
Which applications would benefit most from Web services? Where should you start? What are the key criteria that should ring a bell in your head and make you think, “This is a job for Web services”?
The first and most obvious bell ringer is the need to connect applications from incompatible environments, such as Windows and UNIX, or .NET and J2EE. Web services support heterogeneous integration. They support any programming language on any platform. One thing that's particularly useful about Web services is that you can use any Web services client environment to talk to any Web services server environment.
For example, JPMorgan uses Web services to connect Excel spreadsheets to UNIX-based financial data. JPMorgan operates the global wholesale businesses for J.P. Morgan Chase. JPMorgan is a leader in investment banking, asset management, private equity, custody and transaction services, middle market financial services, and e-finance. The firm has financial analysts in more than 50 countries around the world. These analysts needed a way to upload and download financial, forecast, and other relevant data used in their spreadsheets to and from various legacy application systems.
Knowing that it's difficult to find a single-vendor solution that would allow it to connect Excel with various UNIX-based systems, JPMorgan decided to use Web services. Web services permit the firm to use the right tool for each side of the equation. JPMorgan created a set of Web services using Systinet's Web Applications and Services Platform (WASP) to enable easy access to the legacy applications. Now the financial analysts can access these services from Excel using Visual Basic for Applications (VBA) macros and the Microsoft SOAP Toolkit, as shown in Figure 7-1.
The next bell ringer is any situation in which you have little or no knowledge of or control over the client applications that will be used to access the service. Because Web services don't require a specific software environment, you don't need to worry about compatibility issues.
For example, Con-Way Transportation Services uses Web services to support electronic exchange of shipping data with its customers and business partners. Con-Way is a $2 billion transportation company based in Ann Arbor, Michigan. More than two-thirds of its customers are small to medium-sized businesses. Con-Way wanted to provide these customers with a mechanism that would support tight integration with Con-Way's transportation systems. The challenge was that these customers use a variety of transportation applications on a variety of deployment platforms. Con-Way realized that it didn't have the option of asking these customers to install a proprietary API with limited deployment options to support integrated Con-Way business transactions. Instead Con-Way developed a set of Web APIs using IBM WebSphere. These APIs support invoicing, bill of lading, order pickup, and sales management services. Customers can interface with these services through the Con-Way Web site or use the Web APIs to connect directly from their corporate application systems. The Web APIs support any type of client application—in-house applications as well as packaged applications—as shown in Figure 7-2.
A third bell ringer is the need to support many types of client formats, such as browser clients, rich desktop clients, spreadsheets, wireless devices, interactive voice response (IVR) systems, and other business applications. A Web service returns its results in XML, and XML can be transformed into any number of formats to support different client formats.
For example, Wachovia uses Web services to support both browser-based clients and rich desktop clients for Einstein, its customer information system. Wachovia is a leading provider of financial services, with nine million U.S. customer households. Einstein is a GUI application that gives bank staff complete information about a customer, aggregating information from multiple backend systems. Some bank staff use a browser to access Einstein. Others require a richer desktop interface.
As shown in Figure 7-3, Einstein was developed as a multitier Web service application. The backend business functions and data sources are legacy applications implemented in CICS and DB2 on the mainframe. The middle tier, which accesses and aggregates the customer information, is implemented as a set of J2EE Web services using IBM WebSphere. The client environments are implemented using Microsoft .NET. The browser client is implemented using Microsoft .NET WebForms, and the desktop client is implemented using Microsoft .NET WinForms. Einstein's architecture also allows Wachovia to implement other types of client interfaces to support IVR systems, wireless handsets, two-way pagers, and other devices.
Web services can help you accomplish many types of business goals. You can use Web services to solve immediate tactical problems. You can use them to help you manage your software assets, leverage legacy applications, and reduce development costs. Web services can also help you optimize your business process and improve customer relationships.
The first and most basic way to use Web services is for simple point-to-point integration. For example, Cape Clear uses Web services to connect employees' e-mail clients with its CRM solution. Cape Clear is a Web services software startup. It uses Salesforce.com as its CRM solution. Salesforce.com provides a hosted CRM solution using an ASP-style model. Users typically interface with the CRM solution through a browser, recording customer contact information and correspondence.
Like most software startups, Cape Clear provides e-mail-based customer support. As a result, quite a bit of customer correspondence takes place via e-mail. But Salesforce.com didn't provide a simple, easy way for Cape Clear employees to log this correspondence in the Salesforce.com database. Users had to copy and paste the e-mail from Outlook into the Salesforce.com browser interface. Cape Clear found that lots of correspondence wasn't getting recorded.
Salesforce.com provides a programming API, so Cape Clear decided to eat its own dog food and address this problem using Web services. First Cape Clear used Cape Clear Studio to develop a Web service adapter for the Salesforce.com native programming API. This adapter accepts a SOAP request and translates it into the Salesforce.com native API. Next Cape Clear developed an Outlook macro using VBA and the Microsoft SOAP Toolkit. This Outlook macro adds a button to the standard Outlook tool bar labeled “Save to Salesforce.” As shown in Figure 7-4, when the user clicks on this button, the Outlook macro captures the e-mail message, packages it as a SOAP message, and sends it to the Salesforce.com adapter Web service. The Web service then forwards the e-mail using the native API to Salesforce.com, which logs it.
Figure 7-4. Cape Clear developed a VBA macro using Microsoft SOAP Toolkit that takes an Outlook e-mail and uses SOAP to pass it to the Salesforce Adapter Web service. The Adapter service then passes the message to Salesforce.com using the Salesforce API.
After seeing the advantages of using SOAP for integration, Salesforce.com has decided to develop its own set of Web APIs in addition to the native programming APIs. If you are a Salesforce.com customer, you won't need to build your own adapter Web services.
One of the most popular internal integration projects is enabling a consolidated view of information to make your staff more effective. For example, you probably have many people in your organization who interact with customers. Each time your staffs interact with the customer, you want to let them have access to all aspects of the customer relationship. Unfortunately, the customer relationship information is probably maintained in variety of systems. The good news is that a consolidated customer view provides a single point of access to all these systems.
You can use Web services to implement this type of consolidated view. For example, Coloplast is using Web services to improve its sales and customer support functions. Coloplast is a worldwide provider of specialized healthcare products and services. As part of an initiative to improve customer relationships, Coloplast wanted to set up a state-of-the-art call center system that would give customer representatives real-time access to complete customer histories and product information. The company selected Siebel Call Center as the base application, but it needed to connect this system to its backend AS/400-based ERP systems, which manage the sales, manufacturing, and distribution functions. It did so using Web services. Coloplast used Jacada Integrator to create Web services adapters for the legacy AS/400 application systems. As shown in Figure 7-5, Siebel Call Center uses these Web services to deliver a 360 degree view of customer relationships, including access to backend processes such as open order status, inventory information, customer credit checking, and special pricing. This solution improves efficiency and enhances employee and customer satisfaction.
Web services can make it easier to manage and maintain your legacy application assets. For example, AT&T estimates that Web services technology has reduced the time it takes to make modifications to some of its oldest application systems by 78 percent.
TIRKS (Trunks Inventory Record Keeping System) is a critical application for AT&T. TIRKS was first developed in the 1960s, and it is connected to more than 100 other application systems. Because of the brittleness of these application connections, every time AT&T makes a modification to TIRKS, it must also make a corresponding modification to the other systems. Using Web services, AT&T has developed much more flexible application connections that don't break every time a change is made to the application. AT&T is using IONA XMLBus to replace the more than 100 brittle application connections with a much smaller set of flexible, reusable Web APIs to TIRKS. Now each modification to TIRKS no longer requires the associated changes in all the other application systems.
One of the more popular ways to use Web services is to reduce redundant applications. A service can support many types of application clients. If you need to perform the same type of function via multiple applications, it makes a lot of sense to develop a single service shared by all these applications rather than duplicate the functionality in each application.
Reduction of redundant applications is a key objective of the U.S. government's E-Gov initiative. The U.S. government encompasses hundreds of federal agencies and bureaus, and there is significant overlap and redundancy of systems across these agencies. A 2001 study by the E-Gov Task Force analyzed the agencies to identify the various business activities performed by the government. The study identified 30 general lines of business, such as economic development, public safety, environmental management, and tax collection. On average, each agency is involved in 17 lines of business, and each line of business is performed by 19 agencies. Some lines of business—such as payroll, travel, HR, procurement, logistics, administration, and finance—are performed by every agency.
The U.S. government spent $48 billion on information technology in 2002 and will spend $52 billion in 2003. The Office of Management and Budget estimates that the government can save more than $1 billion annually in IT expenditures by aligning redundant IT investments across federal agencies. In addition, this alignment will save taxpayers several billion dollars annually by reducing operational inefficiencies, redundant spending, and excessive paperwork.
In October 2001, the President's Management Council approved 24 high-payoff government-wide initiatives that integrate agency operations and IT investments. One of those initiatives is E-Travel, which is being run by the U.S. General Services Administration (GSA). E-Travel delivers an integrated, government-wide, Web-based travel management service. Federal government employees make approximately four million air and rail trips each year, and until recently each agency and bureau managed its own travel department. Cumulatively, these various departments used four travel charge card providers, six online self-service reservation systems, 25 authorization and voucher processing systems, 40 travel agencies, and a unique payment reimbursement system for almost every bureau.
By consolidating these travel systems into a single, centralized travel management system, the U.S. government expects to save $300 million annually, achieving a 649 percent return on investment. In addition, the consolidated system will deliver a 70 percent reduction in the time it takes to process vouchers and reimbursements.
GSA delivered the first phase of E-Travel in December 2002—an online self-service reservation system. The total end-to-end travel management system is scheduled to be complete by December 2003. The system will use a service-oriented architecture, based on XML and Web services, to ensure easy integration with existing agency systems and future adaptability. The E-Travel team refers to this architecture as “Velcro integration,” indicating that modules can be easily replaced when necessary.
Web services can also be very useful as a means to manage and coordinate your portal initiatives. A portal is an integrated, Web-based view into a host of application systems. A portal contains a piece of application code (a portlet) for each backend application. A portlet contains the code that talks to the backend application as well as the code that displays the application in the portal.
Web services technology enhances portals in two ways. First, Web services deliver content to the portal as XML. It's then easy for a portal engine to take this XML content and display the information in a portal frame. It's also easy for the portal engine to reformat the XML content to support other client devices, such as wireless handsets or PDAs. Second, Web services technology defines a simple, consistent mechanism that portlets can use to access backend applications. This consistency allows you to create a framework to make it quicker and easier to add new content to your portal. Furthermore, as mentioned in Chapter 5, the new OASIS WSRP specification will allow you to add new content to the portal dynamically. Figure 7-6 shows an overview of WSRP.
Figure 7-6. WSRP lets you add new content to a portal dynamically. A content provider makes content available as a WSRP service, and the portal accesses the content using SOAP and delivers the result in the appropriate markup format.
Another goal of the U.S. government's E-Gov program is to get a handle on government portals. As of February 2003, the U.S. government was managing more than 22,000 Web sites with more then 35 million Web pages. These Web sites have been developed, organized, and managed using the same stovepipe mentality as used in the backend agency applications. Such decentralization and duplication make it difficult for citizens and communities to do business with the government. For example, a community that is attempting to obtain economic development grants must do a tremendous amount of research to learn about federal grants. There's no single source of information. More than 250 agencies administer grants, and you would have to file more than 1,000 forms (most with duplicate information) to apply for all of them. Some of these forms are available online; others aren't. Currently all forms must be filed by postal mail.
The government is working to consolidate this myriad of Web sites into a much more manageable number of portals, each providing a single point of entry to a particular line of business. Each portal will use Web services to access the backend applications that implement the business process. In many cases the government will consolidate backend applications to reduce redundant systems and to ensure a simpler experience for the portal users. For example, the forthcoming E-Grants portal will provide a single point of entry for anyone looking to obtain or administer federal grants. This site will help citizens learn about all available grants and allow them to apply for these grants online. The government expects to save $1 billion by simplifying grant administration as well as saving $20 million in postage.
All government portals will be coordinated through the FirstGov portal at http://www.firstgov.gov. From this one portal, citizens, businesses, and government agencies will have a single point of entry to all other government portals.
Web services can make it easier for your employees to share information and collaborate. For example, the University of Texas M.D. Anderson Cancer Center used Web services to implement a shared information retrieval system called ClinicStation. The center uses a unique collaborative approach to cancer treatment that makes it one of the most respected cancer centers in the United States. Rather than rely on a single physician to manage a patient's case, M.D. Anderson brings together a team of multidisciplinary specialists to collaborate on the best treatment for each individual.
Such collaboration requires a means to dynamically share patient information, such as the patient's chart, test results, x-rays, and other diagnostic images. Because the clinic spans multiple buildings, it's inefficient to try to assemble everyone in the same room to view physical images and discuss a course of treatment. Instead the clinical data is digitized so that it can be viewed electronically. One challenge, though, is that this clinical information is stored in 10 systems on a wide range of platforms. To bring all these systems together, ClinicStation uses Web services built with Microsoft .NET to provide access to all patient information from any browser throughout the center. Physicians can now collaborate over the phone while looking at patient records online.
M.D. Anderson developed this application in nine months using three in-house developers. The total hardware and software costs were less than $200,000. The center expects to save $6 million over the next three years, largely through increased clinical efficiency.
One of the most popular B2B applications is electronic procurement. Companies have been using Electronic Data Interchange (EDI) to automate purchasing applications for years. But EDI is expensive and often requires extensive customization. Web services technology can reduce the cost and time required to create these B2B connections.
Premier Farnell uses Web services technology to implement a B2B procurement system for its customers. Based in London, Premier Farnell is a small-order distributor of electronic components and industrial products to the design, maintenance, and engineering industries throughout Europe, North America, and Asia Pacific.
The Premier Farnell B2B trading solution, implemented using IONA Orbix E2A Web Services Integration Platform, supports customers using any electronic procurement system, including SAP, Oracle, Ariba, Commerce One, and custom systems. As shown in Figure 7-7, even if each of these systems sends a slightly different purchase order format, the Web service can handle the situation. It automatically converts all incoming purchase orders into the format required by the Premier Farnell systems.
Web services provide an excellent foundation for building a trading partner network. The Integrated Shipbuilding Environment Consortium (ISEC) is building a trading partner network for U.S. shipbuilders. ISEC is a project of the National Industrial Information Infrastructure Protocols Consortium (NIIIP), a group of information technology suppliers, industrial manufacturers, academic institutions, and standards organizations. NIIIP is defining an infrastructure based on Web services that can support the creation of a dynamic virtual enterprise: an ad hoc or long-term alliance of companies that work together on a project or opportunity. Although each member of the virtual enterprise operates its own internal business systems, the integration features of the virtual enterprise infrastructure make it seem as if all members belong to the same organization.
NIIIP's first project is focused on the U.S. shipbuilding industry. ISEC will use the NIIIP infrastructure to enable an integrated supply chain that will reduce costs and cycle time for commercial and Navy shipbuilders. Part of the ISEC project is the development of an open, shared parts catalog and a set of standard Web APIs for accessing it.
NIIIP will host a public UDDI registry for the ISEC community. The ISEC catalog service descriptions and Web APIs will be registered in this public registry. The U.S. shipbuilding suppliers will then be invited to implement catalogs that conform to the ISEC standards and to register their catalogs in the public registry. The shipyards will be able to query the public registry to obtain information about the various suppliers and obtain the binding information necessary to access the individual catalogs. Shipyards are also invited to replicate a subset of the public information in their own private UDDI registries and to customize the information for their specific needs. For example, a private shipyard UDDI registry might contain information about the specific subset of approved suppliers for that shipyard. A shipyard might also want to define a set of custom taxonomies to indicate contract history, negotiated discounts, or other information about supplier relationships. Figure 7-8 shows the relationship between the consortium-hosted public registry and a shipyard's private registry.
You can also use Web services to provide a programmatic interface to a business service that you license using the software-as-a-service business model. For the most part, I'm leery of promoting the association of Web services and software-as-a-service. Web services are Web APIs. You don't sell APIs. Instead you sell the business function that customers access through the APIs. As I mentioned in Chapter 6, it's hard to be successful using an ASP-style business model. Looking at history, we can see the secrets to a successful ASP model:
The service must be based on strategic intellectual property, something that your customers can't easily do themselves.
The service must provide a disruptive value proposition—a new and unique advantage that's dependent on the service provider model, such as aggregate information gained through collaboration.[1]
The service provider must establish and maintain a reputation for neutrality and trustworthiness.
The service provider must devise a reasonable revenue model that is comfortable for the customer.
My general take on software-as-a-service is that the business model must be viable on its own without Web services. A Web API is simply a better way to provide programmatic access to the service. If you think you have a new, viable idea for an ASP-style service, then you should provide Web APIs for that service. As I mentioned earlier in this chapter, Salesforce.com has added Web APIs to its already successful ASP model, making it easier for clients to integrate their in-house systems with the hosted CRM solution. Now let's look at another example.
Yahoo is an excellent example of a company that has been successful using the ASP model. Yahoo is the world's leading aggregator of content. The vast majority of Yahoo's clients access this content for free through the Yahoo public portal. As with most public portals, Yahoo generates revenue through advertising. But Yahoo also licenses this content to other businesses as a service. If you are a Yahoo enterprise service customer, you can display Yahoo content in your corporate portal, and users can personalize their corporate portal just as public users can personalize their my.yahoo.com portal.
Yahoo is extending its enterprise software service offering by adding a set of Web APIs. These Web APIs will let you integrate Yahoo content with your business applications. For example, you could integrate Yahoo content with your CRM application. When a salesperson looks up a customer contact in the contact management system, the application can send a query to Yahoo to retrieve and display the latest headlines about the customer. Although it's true that this information is available for free through the Yahoo portal, there's an obvious value to being able to integrate news with a CRM solution. Yahoo plans to license the Web APIs as part of the Yahoo enterprise service offering. Yahoo may also license these APIs to CRM application providers to enable a prepackaged Yahoo-ready solution.
As much as I like Web services, I want to caution you that they aren't always the appropriate solution. XML is tremendously versatile, but it isn't the most compact or efficient mechanism for transferring data. A SOAP message is much bigger than a comparable native binary message used with RPC, RMI, CORBA, or DCOM. It also takes a lot more time to process an XML message than a binary message. Even with the best-performing implementations, SOAP messaging can take 10 to 20 times longer than RMI or DCOM.[2] The performance differential gets worse as the message grows in size and complexity. You want to be cautious when trying to use Web services in situations with stringent requirements for real-time performance. I don't recommend using Web services to transfer very large files (> 10MB).
You shouldn't view SOAP as a total replacement for traditional middleware technologies, such as RPC, RMI, CORBA, and DCOM. These technologies still have an important place in application development. They were designed to provide a seamless, high-performance mechanism to communicate among various components within a homogeneous application system or service.
It's quite appropriate to use these technologies to build individual applications. For example, if you were developing in Java, you would probably want to build the application using J2EE component technologies, such as servlets and Enterprise JavaBeans. These components communicate with each other using RMI.
After you have developed your application, you probably want to expose it to the rest of the world using SOAP and WSDL. The traditional technologies weren't designed to integrate heterogeneous application systems, and they certainly weren't designed to communicate across the Internet. The point is that you want to use the right tool for the job. Web services were designed to support heterogeneous integration and Internet communication.
You might consider using SOAP in place of some proprietary messaging infrastructures, particularly if you are using these technologies to perform simple point-to-point integration. Keep in mind, though, that as of this writing, basic SOAP doesn't provide the same level of reliability as message queuing software, nor does it give you inherent notification facilities to support publish and subscribe functionality.[3]
A number of folks position Web services as the death knell for EAI software. My view is that if Web services can replace your EAI software, then EAI software is overkill for your project. EAI software does many things that SOAP, WSDL, and UDDI simply can't do by themselves. The software category known as EAI consists of a collection of various types of software that work together to deliver a comprehensive integration solution. EAI software includes messaging infrastructure, application adapters, data extraction and transformation tools, message brokers, and rules engines. Web services technology could replace the messaging infrastructure, but it can't replace the rest of the pieces. These other pieces, particularly the application adapters, are complementary to Web services technology. Most EAI vendors are adding support for Web services to their products. Over time I suspect that we'll see most of the EAI vendors adopt Web services technology in place of their current proprietary messaging infrastructure.
Because nearly every application project involves integration, you can use Web services almost anywhere. But when you have a hammer, everything looks like a nail. As with any situation, it's always a good idea to use the right tool for the job.
Web services are useful and versatile, but they have their limits. Let's review what they're good at:
Web services excel in connecting disparate systems. For example, you can use Web services to connect multiple platforms, such as Windows, Macintosh, Linux, UNIX, AS/400, and mainframe systems. You can also use Web services to connect various programming languages, such as Visual Basic, Excel, Java, C++, RPG, COBOL, and Perl.
Web services excel in dealing with uncontrolled conditions. For example, if you want to expose some business functionality to many different users, and particularly if you don't have any control over those users' systems, Web services give you the most flexible and versatile solution.
Web services excel in supporting multichannel clients. For example, you can use Web services to deliver the same application to rich desktop clients, browser users, mobile devices, and IVR systems.
Web services excel in dealing with dynamic conditions. If you anticipate that your system will need to adapt to changing conditions, Web services are a great choice. If you anticipate churn in your user base or if you anticipate the need to add or modify parts of your system, Web services can deal with these changes gracefully.
Web services excel in aggregating information. You can use Web services to pull together information from many sources for portal applications, customer care applications, and collaborative applications.
Web services excel in reducing redundancy. If you have many applications that perform essentially the same work, you can use Web services to centralize that functionality into a single set of shared services.
Although Web services are versatile, they aren't a complete replacement for all other types of middleware. Web services don't do all the things that other middleware systems do. Other middleware systems still have their place:
Web services don't provide a development component model, and therefore you don't build applications using Web services. You generally build applications using component technology based on your programming language, such as Java or Visual Basic. You want to use the native communication technology supplied by the language (for example, RMI or DCOM) to connect the various components within the application. You can then use Web services to expose the application functionality to other environments.
Web services don't perform as well as traditional middleware systems. If you have stringent performance and scalability requirements, Web services may not be appropriate for your project.
Web services don't do all the things that an EAI middleware solution does. If you need to extract, aggregate, transform, and route information for business process automation purposes, you'll need more capability than Web services offer. EAI middleware often complements Web services.
[1] Please see The Innovator's Dilemma, by Clayton M. Christenson, ISBN 0875845851, for an explanation of disruptive versus sustaining innovations and value propositions.
[2] Each new generation of Web services technology makes tremendous improvements in performance and scalability. It's quite possible that these performance issues may go away in subsequent generations.
[3] By “basic SOAP” I mean SOAP over HTTP. You can achieve the same level of reliability as MOM by transporting your SOAP messages using a reliable transport. The standards community is also developing SOAP extensions to support reliable messaging. I expect most SOAP implementations to support reliable messaging by early 2003.