Web services are becoming more and more present in today’s development projects. Indeed, they can prove to be an excellent choice for integrating loosely coupled systems.

Web services are often used to integrate different systems, possibly developed by different organizations. If you are on the receiving (client) end of a web service integration project, it is in your best interest to make sure that the web services you are calling return the data you expect them to return.

In this chapter, we look at SoapUI, a tool that can help you test your web services. SoapUI^[15] is a powerful open source tool written by Eviware that helps you test your web services in a variety different scenarios. SoapUI supports a wide range of web service protocols, including Soap 1.1 and 1.2, MTOM, and SAAJ, making it suitable for testing a wide range of web services with different requirements. SoapUI also allows you to perform functional testing and put your web services under load and compliance tests along with some code generation capabilities that makes web service development easier.

Based on material contributed by: Masoud Kalali

SoapUI is a particularly rich tool. Its features fall into two main areas, which together cover a good part of the web service development lifecycle. These two areas are:

Let’s look at each of these in more detail.

SoapUI is written in Java, so it can run on any Java-compliant platform. Installing SoapUI is straightforward: you simply download and run the installer from the SoapUI web site.^[*] Alternatively, you can install it using WebStart from the SoapUI web site.

SoapUI runs as a standalone Java client application with a rich user interface that looks and feels like a modern IDE. There are also plug-ins available for NetBeans, Eclipse, and IntelliJ.

To illustrate the features of SoapUI, we will use a locally deployed web service. This is a simple web service that provides access to an imaginary customer database. The Java version of this service implements the following interface:

public interface CustomerService {
    public Customer findById(String id);
    public Customer[] findByName(String name);
    public Customer[] findByHomeCity(String city);
}

A basic understanding of the domain model will make it easier to follow the SOAP examples later on. The UML class diagram for this sample application (generated by DOxygen—see Generating Source Code Documentation with Doxygen) is illustrated in Figure 17-1.

If you want to follow along at home, you will need to download and install Apache Axis 2,^[16] which has been used to write a simple web service of our own. For detailed instructions on how to install and use Axis2, check out the web site. Here is the condensed version (with the corresponding Linux commands just for a bit of extra precision):

Now that we have a local web service, we can do some testing.

Now let’s see SoapUI in action. SoapUI has an initiative IDE-like GUI, which lets you easily create and execute your tests (see Figure 17-2). On the left side of the screen, you will find an overview of your project displayed in a tree-like structure.

Figure 17-2. SoapUI has a rich and fairly intuitive user interface

In this section, we will go through the steps involved in setting up some simple unit tests for a web service. Start up SoapUI and create a new project using the “File→New WSDL project” menu (see Figure 17-3). The easiest way to get started is to provide a WSDL (Web Service Definition Language) file when you create the project. It is the cornerstone of any web service. In a real-world project, the WSDL may be defined from the outset as a contract between interoperating systems. Alternatively, it might be generated during the build process, based on the classes being deployed to the web service. Probably the easiest way to obtain a reliable, up-to-date WSDL file is to query the deployed web service itself. For example, the WSDL file for our locally deployed web service can be found at http://localhost:8080/axis2/services/CustomerService?wsdl.

Figure 17-3. Creating a new WSDL project

This screen also lets you choose to create sample requests for all of your endpoint operations. These sample SOAP requests are a good starting point for your tests. You can use them as a convenient template for your own requests, which comes in handy if you don’t remember the exact structure of a SOAP XML request off the top of your head.

Now just press OK, and let SoapUI create the new project for you.

Your new project will contain the list of operations available from this web service, as illustrated in Figure 17-2. From here, you can build SOAP requests for operations you want to test. Testing a web service using SoapUI basically involves creating SOAP requests for the operations to be tested and then organizing them into test cases and test suites. SoapUI is fairly flexible about how you create these requests; you can either create them directly within a test case, or you can create them from within an operation and then add them to a test case. Using this latter approach, you can create test requests, which can be reused in different test cases.

Let’s create our first SOAP request, which will test the findByName operation. The findByName operation returns a list of customers with a given surname. It takes a single parameter called, appropriately enough, name. For our first test, we are going to look for all the clients named “Smart.” Open the sample request (initially named “Request 1”). As illustrated in Figure 17-2, this request contains a full SOAP request with the operation parameters replaced by question marks:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
 xmlns:xsd="http://com.wakaleo.jpt.customerws/xsd">
   <soap:Header/>
   <soap:Body>
      <xsd:findByName>
         <xsd:name>?</xsd:name>
      </xsd:findByName>
   </soap:Body>
</soap:Envelope>

So, all we need to do to tailor our request is fill in the gaps. Just replace the question mark in the <xsd:name> parameter with “Smart.” The full SOAP request for this query looks like this:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
 xmlns:xsd="http://com.wakaleo.jpt.customerws/xsd">
   <soap:Header/>
   <soap:Body>
      <xsd:findByName>
         <xsd:name>Smart</xsd:name>
      </xsd:findByName>
   </soap:Body>
</soap:Envelope>

Now, rename the request to something more appropriate (using the “Rename” option in the contextual menu) and run the request using the green arrow button in the request editor (see Figure 17-4). SoapUI will execute the request and display the SOAP response. Note that you are testing against a live server here: SoapUI does not verify the syntax or contents of your request before sending it off. So, you do need a fairly good understanding of SOAP and XML at this level, as well as a good idea of what your query is supposed to look like.

Figure 17-4. Executing a SOAP request in SoapUI

The next step involves creating a test suite. SoapUI, like many software testing products, lets you organize your individual test cases into test suites. Right-click on project node, select New TestSuite, and enter an appropriate name. This Test Suite is simply a container designed to regroup a set of related test cases. As in unit tests, SoapUI test cases are where all the real testing gets done. So, without further ado, let’s look at how to create a SoapUI test case.

Right-click on the test suite you created earlier, and select New Test Case. Name the test case something easy to remember, like “FindByName.” Your new test case should appear nested in your test suite in the SoapUI project pane.

A test case is made up of a series of test steps, which are executed one after the other. A Test Step can come in a number of forms: sending a SOAP request, pausing for a certain period of time, reading data from an external source, running a Groovy script, and many more. For our simple unit test, we need to add a Request step, which sends a SOAP request to the web service and allows you to verify the response.

Figure 17-5. Inserting a new request in a test case

There are several ways to do this. You can insert a new request directly into the test steps by selecting the Test Steps node and choosing “New Step→Test Request” in the contextual menu. SoapUI will let you choose the operation you want to test, and create an empty SOAP message for this operation for you to fill in. This can be useful if you want to insert a one-off request that you do not intend to use elsewhere.

Alternatively, you can add an existing request to your test case, which is what we will be doing here. Go to the request we created earlier on, and select “Add to TestCase” in the contextual menu (see Figure 17-5). SoapUI will propose a list of test cases where you can place your request (see Figure 17-6). Select the test case we created above.

Figure 17-6. Selecting a test case

A test case should not simply execute a piece of code; it should also verify the results. SoapUI lets you add different types of assertions to your test case to verify that the operation was successful. When you add a request to a test case, SoapUI proposes to add some basic assertions automatically (see Figure 17-7). These include optional tests that check that your request complies to the WSDL you specified, no server errors occurred, and the response is correctly formed.

Figure 17-7. Adding a request to an existing test case

When you are done, SoapUI will create a new SOAP request step within your test case. You can display and edit the SOAP request in the same way as we saw previously, with one addition: you can insert additional assertions (see Figure 17-8). Note that you are actually editing the original request that you created in the first operation, not a copy, so any changes you make will affect all the test cases that use this request.

Figure 17-8. Adding an assertion to a test

In addition to the basic assertions we saw earlier, SoapUI supports a number of other assertions, including “Contains,” “Not Contains,” and “XPath Match.” You can use these to verify that the response contains the data that you expect. For example, we might want to make sure that every SOAP response in the findName query above at least contains the word “Smart” somewhere in its contents. We could do this with a Simple Contains Assertion, as shown in Figure 17-9.

Figure 17-9. Adding an assertion to a test

For more sophisticated tests, you can use XPath Assertions to narrow down your assertion to a particular XML element or attribute.

In real-world situations, a typical test case may involve more than one request. For example, a unit test might involve simply invoking a web service once and making sure that the response contains the data you expect, or it could involve calling the web service several times with different data. Functional testing might involve invoking a series of requests to simulate a full business transaction.

Once you have a working SOAP request, it is easy to create other similar requests for the same operation using the “Clone Request” menu item. You can also clone Test Cases and Test Suites, so you can create a sizable battery of tests in very little time. As you would expect, you can either run test cases individually, or alternatively run all the test cases in a test suite simply by executing the test suite (see Figure 17-10). When you run a test suite, you can also choose to run the test cases sequentially (the default option) or in parallel.

Figure 17-10. Running a TestSuite

This is a fairly trivial example. With a little practice you can build much more sophisticated test cases in SoapUI using properties, conditional expressions, or even the Groovy scripting language.

SoapUI comes with some convenient features for web service load-testing. Although these load-testing features are not as rich and flexible as those of tools like JMeter (see Chapter 16), they are sufficient to let you set up simple but effective load tests for your web services in a minimum of time. Load tests can be used to measure the performance of your web services under different loads, including assertions to ensure that performance always remains within acceptable limits.

In SoapUI, you create load tests for a given test case. We could use the test case we built in the previous chapter. SoapUI makes a clear distinction between load tests and functional tests, even when they are in the same test case. However, it is often useful to distinguish between functional tests and load tests, as you tend to run them at different times, and the test steps you run in a load test may not be the same as those of functional tests. So, here we will set up a test suite just for load testing. Create a new test suite and add a test case containing the requests that you want to include in your load test. In Figure 17-11, the test case contains several requests, each separated by a one second delay.

Next, you need to create the actual load test. Select the “Load Tests” node in your new test case and choose “New LoadTest” in the contextual menu (see Figure 17-11). This will create a new load test ready to go.

Figure 17-11. Creating a new load test

The load test window (see Figure 17-12) gives you a central interface where you manage all load testing activities for a given test case. The interface is fairly intuitive. Like elsewhere in SoapUI, the green arrow starts the tests and the red cross stops them. So far so good. Running the load tests like this, using the default configuration, can yield some interesting initial results.

The Threads field lets you define the number of threads that will be run simultaneously, and therefore the number of users that are being simulated. The number of users that can be effectively simulated depends largely on your hardware configuration. You can also define a startup delay between each in the options window, which allows you to gradually build up user load. The total length of the load test is determined by the Limit field, in terms of either elapsed time (seconds) or the number of times the test case is executed in each thread (runs).

You can define load tests using several different strategies. In the Simple strategy, the test case will be invoked at (more or less) regular intervals in each thread. You can use the Test Delay and Random fields to define how long each thread should wait between calls, and how random this delay should be.

The Variance strategy lets you study a varying number of users over time. The number of active threads will decrease and increase periodically, simulating periods of more or less intensive load.

The Burst strategy simulates bursts of intensive use (the Burst Duration), followed by periods of inactivity (the Burst Delay).

The Threads strategy lets you simulate a linearly increasing number of users over time.

Figure 17-12. Configuring a load test

When you execute the load test, SoapUI will display (and record) statistics about the web service calls: the shortest, longest and average time taken, the number of transactions per second, and so on. This data can also be displayed in graphical form. SoapUI supports two main types of graph:

In Figure 17-13, for example, you can see a statistical graph showing the number of errors increasing sharply as the thread count increases.

Figure 17-13. A statistical history graph of Transactions Per Second (TPS)

Assertions are another useful thing you can add to your load test (see Figure 17-14). Assertions let you make sure that the web service performs . You can define assertions in terms of the maximum number of permissible errors, the minimum number of transactions per second, the maximum total time required to run the load test, and more. Load-test assertions can be used alongside functional test assertions to perform sophisticated integration tests and to monitor your live web services.

SoapUI can also be run from the command line, which can be useful for automated testing and web service monitoring. The testrunner script (testrunner.bat for Windows and testrunner.sh for Unix) will run all the functional tests in a given SoapUI project file:

$ $SOAPUI_HOME/bin/testrunner.sh -r CustomerService-soapui-project.xml
================================
=
= SOAPUI_HOME = /usr/local/tools/soapui-1.7.1/
=
================================
soapUI 1.7.1 TestCase Runner
Configuring log4j from [jar:file:/usr/local/tools/soapui-1.7.1/bin/soapui-1.7.1.jar!
/soapui-log4j.xml]
22:41:25,431 INFO  [SoapUITestCaseRunner] setting projectFile to 
[test/CustomerService-soapui-project.xml]
22:41:25,432 INFO  [SoapUI] Missing folder [/home/john/projects/jpt-sample-code
/customer-cs/ext] for external libraries
22:41:25,998 WARN  [SoapUI] Failed to load settings [soapui-settings.xml (No such file 
or directory)], creating new 
22:41:26,098 INFO  [WsdlProject] Loaded project from [/home/john/projects/jpt-
sample-code/customer-cs/test/CustomerService-soapui-project.xml]
22:41:26,441 INFO  [SoapUITestCaseRunner] Running soapUI tests in project 
[CustomerService]
22:41:26,444 INFO  [SoapUITestCaseRunner] Running soapUI suite [FindByName TestSuite], 
runType = SEQUENTIAL
22:41:26,451 INFO  [SoapUITestCaseRunner] Running soapUI testcase 
[Get Smart TestCase]
22:41:26,451 INFO  [SoapUITestCaseRunner] running step [findByName - 
Get Smart]
...
22:41:29,958 INFO  [SoapUITestCaseRunner] Finished running soapUI testcase 
[FindByName], 
time taken: 2157ms, status: FINISHED
22:41:29,958 INFO  [SoapUITestCaseRunner] soapUI suite [Load TestSuite] finished 
in 2190ms

SoapUI 1.7.1 TestCaseRunner Summary
-----------------------------
Time Taken: 3516ms
Total TestSuites: 2
Total TestCases: 4 (0 failed)
Total TestSteps: 8
Total Request Assertions: 5
Total Failed Assertions: 0
Total Exported Results: 0

Figure 17-14. Assertions in a load test

The -r option used here displays a summary report at the end of the tests. You can also narrow down the scope of your tests using the -s (run only a specified test suite) or -c (run only a specified test case) options. At the time of this writing, this worked best if your test suite and test case names didn’t have any spaces:

$ $SOAPUI_HOME/bin/testrunner.sh -r -sFindByName CustomerService-soapui-project.xml

In the same manner, you can use the loadtestrunner script to execute the load tests defined in a given SoapUI project:

$ $SOAPUI_HOME/bin/testrunner.sh CustomerService-soapui-project.xml -r
================================
=
= SOAPUI_HOME = /usr/local/tools/soapui-1.7.1/
=
================================
soapUI 1.7.1 LoadTest Runner
...
22:46:57,451 INFO  [SoapUILoadTestRunner] LoadTest [LoadTest 1] progress: 1.00565, 5
22:46:58,456 INFO  [SoapUILoadTestRunner] LoadTest [LoadTest 1] progress: 1.0224, 5
22:46:59,461 INFO  [SoapUILoadTestRunner] LoadTest [LoadTest 1] progress: 1.03915, 2
22:47:00,465 INFO  [SoapUILoadTestRunner] LoadTest [LoadTest 1] finished with status 
FINISHED
22:47:00,465 INFO  [SoapUILoadTestRunner] Exporting log and statistics for LoadTest 
[LoadTest 1]
22:47:00,469 INFO  [SoapUILoadTestRunner] Exported 2 log items to [LoadTest 1-log.txt]
22:47:00,469 INFO  [SoapUILoadTestRunner] Exported 0 error results
22:47:00,471 INFO  [SoapUILoadTestRunner] Exported 6 statistics to [LoadTest 
1-statistics.txt]
22:47:00,471 INFO  [SoapUILoadTestRunner] soapUI suite [Load TestSuite] finished 
in 63363ms

In addition to the options mentioned above, you can also use the -l option to narrow your tests down to a particular load test.

Integrating SoapUI tests into an Ant build isn’t too difficult, although you do need to invoke the SoapUI scripts at the command line, which makes portability an issue. Indeed, to run the script correctly, you need an <exec> tag for each of your target operating systems.The -j command-line option tells SoapUI to generate Javadoc-style XML report data, so you can invoke <junitreport> to generate the results in HTML form.

<property environment="env"/>
<property name="AXIS2_HOME" value="${env.AXIS2_HOME}"/>
<property name="SOAPUI_HOME" value="${env.SOAPUI_HOME}"/>
<property name="soapui.home" value="${SOAPUI_HOME}" />

<property name="testrunner.sh" location="${soapui.home}/bin/testrunner.sh"/>
<property name="testrunner.bat" location="${soapui.home}/bin/testrunner.bat"/>
...
<target name="soapui-report">
  <mkdir dir="reports" />
  <exec executable="${testrunner.sh}" os="Linux" failonerror="true">
    <arg line="-j -freports CustomerService-soapui-project.xml"/>
  </exec>
  <exec executable="cmd.exe" os="Windows 2000" failonerror="true">
    <arg line="/c ${testrunner.bat} -j -freports CustomerService-soapui-
     project.xml"/>
  </exec>
  ...
  <junitreport todir="reports">
    <fileset dir="reports">
      <include name="TEST-*.xml"/>
    </fileset>
    <report format="frames" todir="reports/html"/>
  </junitreport>
</target>

Note that we are using the SOAPUI_HOME environment variable to find the SoapUI scripts. Indeed, this build script requires the AXIS2_HOME and SOAPUI_HOME environment variables to be correctly defined.

The soapui-report target will produce a JUnit-style report illustrated in Figure 17-15.

Figure 17-15. JUnit reports generated by SoapUI

SoapUI provides a Maven 2 plug-in, which is well documented on the SoapUI web site. The plug-in is not in the standard repositories, so you need to add the following to your plug-in repository list:

<pluginRepositories>
   <pluginRepository>
      <id>eviwarePluginRepository</id>
      <url>http://www.eviware.com/repository/maven2/</url>
   </pluginRepository>
</pluginRepositories>

Next, add the SoapUI plug-in to the <build> section of your pom.xml file:

    <build>
        <plugins>
           ...
           <plugin>
               <groupId>eviware</groupId>
               <artifactId>maven-soapui-plugin</artifactId>
               <version>1.7</version>
               <configuration>
                   <projectFile>CustomerService-soapui-project.xml
                   </projectFile>
                   <host>127.0.0.1:8080</host>
               </configuration>
               <executions>
                   <execution>
                       <phase>integration-test</phase>
                       <goals>
                           <goal>test</goal>
                           <goal>loadtest</goal>
                       </goals>
                   </execution>
               </executions>
           </plugin>
        </plugins>    
    </build>

This will execute both SoapUI functional and load tests during the integration test phase.

You can also run SoapUI functional tests directly using the test goal:

$ mvn eviware:maven-soapui-plugin:test

Load tests can be run using the loadtest goal:

$ mvn eviware:maven-soapui-plugin:loadtest

It is often useful to integrate web service functional and load tests into your Continuous Build process. Frequent functional tests serve the same purpose as periodic builds and unit tests; they let you detect errors as early as possible. Frequent load tests let you detect any performance problems that might have slipped into your application.

You can use SoapUI fairly easily to do this sort of continuous web service testing. Because SoapUI needs to work against a real server, you need to make sure you have the latest build of your application always running on a development server somewhere. You would typically build this into your standard Continuous Integration process, and reserve a deployment environment for this purpose. Then it is simply a matter of integrating the SoapUI tests into the build process.

If you are using Ant, just set the failonerror attribute to true when you invoke the SoapUI scripts. This will force the build to fail and notify the team if the web service tests fail. This is illustrated here:

    <property name="soapui.home" value="/usr/local/tools/soapui-1.7.1" />
    <property name="testrunner.sh" location="${soapui.home}/bin/
     testrunner.sh"/>
    <property name="testrunner.bat" location="${soapui.home}/bin/
     testrunner.bat"/>
    <property name="loadtestrunner.sh" location="${soapui.home}/bin/
     loadtestrunner.sh"/>
    <property name="loadtestrunner.bat" location="${soapui.home}/bin/
     loadtestrunner.bat"/>
    ...
    <target name="soapui-functional-tests">
        <echo>os.name = ${os.name}</echo>
        <mkdir dir="reports" />
        <exec executable="${testrunner.sh}" os="Linux" failonerror="true">
            <arg line="CustomerService-soapui-project.xml"/>
        </exec>        
    </target>

    <target name="soapui-load-tests">
        <echo>os.name = ${os.name}</echo>
        <mkdir dir="reports" />
        <exec executable="${testrunner.sh}" os="Linux" failonerror="true">
            <arg line="CustomerService-soapui-project.xml"/>
        </exec>        
    </target>

Now you can just set up an Ant-based project in your favorite Continuous Integration tool and invoke these two targets. If the SoapUI tests fail, everyone will be automatically notified. Figure 17-16 illustrates how this could be done using Continuum.

Figure 17-16. Continuous testing using SoapUI tests in an Ant file

For a Maven project, the process is even simpler. Set up the Maven project as described in Running SoapUI from Maven and then create a CI project that invokes the integration-test goal (see Figure 17-17).

Figure 17-17. Continuous Testing using SoapUI tests in a Maven file

This chapter gave a brief overview of how SoapUI can help you to test your web services. It is a powerful tool, with a fairly low learning curve, which can be a valuable asset in a project involving web services.