Testing the WebAuction Application

When developing and deploying a WebLogic Server application, you want to make sure that the application performs as expected. It should present a user interface that functions as expected. When many requests come in simultaneously, your deployment should scale to meet the needs of your users.

The testing process assures that the application will perform as expected when it is deployed. In most organizations, this process is called quality assurance (QA). The QA group designs, implements, and runs tests for each new release to ensure that the application continues to meet defined quality standards.

Every organization will have different requirements and different styles for testing their WebLogic Server applications. There is no universal testing methodology that addresses the needs of every environment. However, it is important you understand that QA is a key component of the Web application development process. It is unlikely that the procedures used in this section will apply exactly to your deployment. However, you should model the thought processes described in the following sections as you define your own QA process, suitable to your environment.

In this section, we discuss two types of application testing:

• Functional testing, which ensures that the application functions as expected

• Stress and performance testing, which ensures that the application performs as expected under heavy load and real-world conditions

We cover the methodology for implementing both of these types of testing, and illustrate the testing process through the use of third-party tools.

Functional Testing

Functional testing ensures that the application performs as expected. For example, if someone enters a bid, then that bid should appear in the auction system. You implement functional testing in order to isolate any bugs in the application. You should design your tests to be repeatable, so that they can be applied to new versions of your application as they are developed.

Functional Testing Methodology

There are three major steps in the functional testing of WebLogic Server Web applications:

Infrastructure Requirements for Functional Testing

In many organizations, individuals are designated to specialize in QA and testing for applications. These individuals are in charge of developing a test plan, and implementing and executing the test. They're responsible for assuring that the product meets the quality requirements defined by the organization.

Very large organizations might have dozens or even hundreds of QA specialists. In very small organizations, the developers of the product themselves often perform both QA and development roles for the product. Depending on your situation, you should balance your organization's quality requirements with the resources available to meet those requirements.

Most functional testing can be done in a scaled-down manner. Because functional testing is meant to verify that subsets of the application's functionality perform as expected, QA specialists often can perform testing on a single PC, as was the case with the WebAuction functional testing described in this chapter.

Functional Testing Example with the WebAuction Application

In the majority of cases, it makes sense to use third-party, specialized Web testing tools on your applications. Many WebLogic Server developers attempt to write custom test frameworks for their applications. Most discover that off-the-shelf testing tools provide superior functionality and improved efficiency over home-grown tools.

This section shows how to apply functional testing methodology on a single feature of the WebAuction application, and test it using a third-party tool.

Define Scope of Testing

In a real-world testing situation, you would specify the complete scope of testing all the features of the WebAuction application. In this case, we will limit the scope, for the sake of the example, to a single feature.

Here's the feature: WebAuction should reject a user name that's already registered.

Design Tests

Design a test that attempts to create two users with the same user name and verifies that the second attempt is rejected.

Implement and Execute Tests

To implement this test, use the Astra QuickTest product available from Mercury Interactive (http://www.mercuryinteractive.com/). A free seven-day evaluation version of this product is available for download. Several other Web application–testing tools perform in much the same way as Astra QuickTest.

The QuickTest product enables you to record user interactions in a Web application, directly from the Web browser. We can record the actions required to create a new user.

First, start the QuickTest product and choose the option for Record. You will be prompted to enter your Web browser preference. Your Web browser will start at the URL you specify (http://127.0.0.1/auction/).

Start the WebAuction application and click the link to create a new account. After this is successful, log out of the application and attempt to create another new user with the identical user name and password (see Figure 14-34).

To finish the test, instruct QuickTest to verify that we did indeed receive notice that the user name we just attempted already exists. To do this, add in a checkpoint that verifies the message “user name already exists” is displayed in the final page of the recording.

To add the checkpoint, click on the final page, which will likely be WebAuctionNew_4 in the left-hand navigation panel of QuickTest. Use the menu bar to instruct QuickTest to insert a new text checkpoint (see Figure 14-36).

To execute this test, you should first empty your database of users. You can do this by dropping and then re-adding the database tables. Then, click the Run button at the top of the QuickTest application. You should see a simulated browser run through your entire application, mimicking the steps that you recorded earlier.

Finally, you should see the results of your tests displayed in a report window (see Figure 14-37).

In this example, we have successfully tested that the application performs as expected when you attempt to create a user whose name already exists. In a real-world situation, you would extend this process to all of the major features of your application.

Stress and Performance Testing

In the previous section, functional testing was used to assure that the WebAuction application performed as expected. It also is important to make sure that your applications perform as expected in high-stress situations. As many users connect to your site simultaneously, your deployment must scale to handle the increased demand. To verify that your application will scale to meet the needs, stress and performance testing attempts to replicate the load that will be placed on your application when it is put into production.

Stress and Performance Testing Methodology

There are four major steps in stress and performance testing with WebLogic Server Web applications:

In a real-world application, your interaction model could be much more complicated. In the case of the WebAuction application, the interaction model is actually more complex than that previously described. Nonetheless, it is beneficial to make the interaction model for testing as simple as possible, so long as it is not radically different than what will occur during deployment.

For example, the preceding model grouped interactions involving only database reads, and those involving no database access, into a single group. If you tested the capacity of your system to browse items up for auction only, with no testing of the non-database pages, you can be assured that the results will be no worse than testing main.jsp by itself. You can then discover what the minimum capacity is of your application.

Infrastructure Requirements for Performance and Stress Testing

The infrastructure requirements for performance and stress testing are much greater than for functional testing. Because you are attempting to simulate how the application will perform under very heavy load, you must create an operating environment that represents your real deployment environment. In most cases, this involves substantial network and hardware infrastructure.

To test performance and stress on the WebAuction application, we used a combination of four 4-CPU Windows NT 4.0 Server machines with 1GB of memory each. Each machine contained four network interface cards (NIC), which connected it to a 32 Port 100Mb Ethernet switch. Standard hardware, including SCSI disks and Intel Pentium II 500Mhz Xeon processors, powered each machine.

In terms of software, an Oracle database instance was installed on one of the machines. This instance was the standard untuned Oracle installation. Two of the machines acted as clients to a third machine, which ran the WebLogic Server deployment (see Figure 14-38).

The client machines each ran a load-testing tool available for free from Microsoft, which is called the Microsoft Web Application Stress Tool (WAST). It is a fully functional Web application load-generation tool, which includes the capability to record user interactions through a Web browser, much like Astra QuickTest. In addition, it enables you to simulate many different users, each with a different login and password.

The Microsoft WAST can be downloaded from the Microsoft Web site at http://homer.rte.microsoft.com.

Stress and Performance Testing Example with the WebAuction Application

To demonstrate stress and performance testing methodology with the WebAuction application, we follow the steps defined previously.

We will implement, execute, and review the results for each of these tests.

Simulating Browsing

In each client machine, use the recording feature available in the Microsoft WAST to create a script that represents a single user browsing a single category. In this case, every simulated user would make a request for the items available in the books category in the WebAuction.

This test includes visiting a single URL repeatedly to display the items in the books category: http://bench1:7001/auction/browseitems?cat=books.

We ran 60-second simulations in which the client machines put heavy load on the server machine. We initially began with 8 client threads each opening 25 sockets to the server. It became apparent that this was not enough load to stress the server CPUs sufficiently. Thus began an iterative process to attempt to locate bottlenecks in the application deployment (see Table 14.1):

After the second run, it became apparent that we were making full utilization of the CPU for the WebLogic Server server instance by looking at the Windows NT processor monitor (see Figure 14-39).

Next, we tested to see if memory utilization was a limiting factor. To test this, we hypothesized that the memory heap size might have been too low. We ran the same test again, but with a greater memory size. We increased the memory to 256MB in the WebLogic Server heap, which is a parameter set in the command line starting WebLogic Server:

java -hotspot -Xms64m –Xmx256m -classpath

You can edit this value by changing the start scripts used for your WebLogic Server installation. These are found in the eawlserver6.0config<servername> directory and are the scripts that are called from the Windows Start menu (see Table 14.2).

Because the CPU utilization and the throughput did not change, we can conclude that the application is limited by the CPU in this case and changing the memory size will not likely affect performance greatly. In more complex applications, it is likely that larger heap sizes will be required. The WebAuction application is relatively simple and requires less memory. As you performance test your own applications, modify the heap size to determine how changes affect performance. You can use this data to figure out the appropriate heap sizes for optimum performance.

Next, we modified the number of WebLogic Server execute threads inside of WebLogic Server to see if there was contention around connections to the database. Contention is a phenomenon that occurs when more than one thread or process of execution attempts to access a resource at the same time. In the case of database connections, disk access, or memory access, only a single Java entity can access that resource at a time. If multiple threads attempt access simultaneously, one will block (sit idle) while waiting for that resource to become available.

Understanding ExecuteThreads

In order to understand the tuning of execute threads, it is helpful to understand the internal architecture of WebLogic Server. At the heart of WebLogic Server is a set of queues, called ExecuteQueues. Each of these queues contains all of the work currently slated for WebLogic Server. This includes work such as creating a new database connection, executing a servlet on behalf of a user, and so forth. Work enters the WebLogic Server system through one of many ways. It can arrive through a request to the servlet engine over HTTP, and a request to the T3/IIOP subsystem for RMI access to objects like EJB, among others.

A pool of Java threads is maintained that does all of the work inside of WebLogic Server and allows for parallel execution of work tasks. These Java threads are called execute threads and their number is determined by the size of the ExecuteThread count in the WebLogic Server configuration. As work arrives and is placed into the execute queue, ExecuteThreads become free and perform the work tasks. When a response to a client is required, the response is handed off to other special threads whose job is to deal with the input and output to the server. The number of those threads is not regularly changed.

Figure 14-40 graphically depicts this process.

Changing the size of the ExecuteThread pool enables you to modify the amount of parallelism possible in your WebLogic Server deployment. But, as is mentioned in the WebLogic Server tuning guide, greater parallelism has a greater cost in terms of thread context switching and memory usage. For this reason, WebLogic Server permits a maximum of 400 execute threads, though virtually all applications will require much less than this.

In most deployments, the default value for the count of ExecuteThreads is sufficient though this parameter should be manipulated slightly during your load-testing process to see if changes can enhance performance of your particular application.

You want your application to avoid contention for resources as much as possible. In WebLogic Server, execute threads are the Java threads that do work on behalf of clients. When work enters WebLogic Server in the form of a request from a user, that work is put on a queue maintained by WebLogic Server. This queue is different than the JMS queues discussed previously: It is not visible to applications or developers.

WebLogic Server maintains a pool of threads, the size of which is configurable by the administrator through the WebLogic Server console. WebLogic Server removes work from the work queue, and execute threads perform those work items. When those work items result in contention for resources, threads will sit idle and the CPU utilization will go down. If you notice that your CPU utilization is very low under heavy load, increasing the execute thread count through the WebLogic Server console is an option. You can modify this value in your console (on a per-server basis) under the Configuration Tuning tab in your individual server (see Figure 14-41).

To determine if increasing execute threads in WebLogic Server would help ease contention (and maximize CPU utilization at 100 percent), another test was run with 30 execute threads (see Table 14.3).

Notice that the size of the WebLogic Server database connection pool to the Oracle instance was also increased. As stated in the WebLogic Server tuning guide, the number of database connections that you have should never be greater than the number of execute threads. Preferably, the number of database connections should be about equal to the number of execute threads.

In Run No. 4, the server utilization and throughput of the application actually went down relative to Run No. 3. It is likely that the addition of new threads and connections to the database resulted in increased context switching inside of WebLogic Server. Context switching occurs anytime the Java virtual machine (JVM) must switch between two threads of execution. During this switching time, work cannot be done, resulting in lower performance.

At this point, we made use of an application profiler tool called OptimizeIt, which is one of the many Java profiling tools available today. These tools enable you to monitor the execution of a Java application and take snapshots of the performance of the application at any given point. You can see the internal workings of every thread operating inside of the JVM to determine where your application bottlenecks might lie. For example, you might see that every thread inside of WebLogic Server is blocked as it waits on a resource, such as a database connection. Or, you might see that all of the application code that you have built on top of WebLogic Server is blocked and waiting for access to a single resource, such as a static variable or a connection to a legacy system.

In the case of the WebAuction application, using the Java profiler validated that our architecture had no issues surrounding contention or lack of resources.

If you recall, the fourth step in application performance and stress testing is to review and analyze the results. It was at this point we noticed that the WebAuction application repeatedly reads the same data.

For this type of read-mostly application, we should make use of one helpful feature of WebLogic Server: caching tags for JSP. Therefore, we improved the browseitems.jsp page to specify that the output from the JSP page that includes the data items should be cached for 10 minutes, using the following code:

<wl:cache key="application.cat" timeout="10m" >

// JSP and HTML print out all of the items in the current cate-
gory

</wl:cache>

More information on how to use the WebLogic Server caching tags can be found in the WebLogic Server documentation.

With the caching tags added to the application, the same tests were executed again resulting in a substantial performance increase of over 10 percent (see Table 14.4).

At 342 pages per second, the WebAuction application could theoretically provide nearly 30 million page views in a single day:

342 pages/s * 60s/minute * 60 minutes/hour * 24 hours/1 day =
29,548,800 pages/day

Bandwidth

What about bandwidth? A request for the browseitems.jsp page retrieves all of the following HTML and images:

2,797 browseitems.jsp
2,435 built_bea_web.gif
  317 footer.jsp
  152 header.jsp
1,402 header2.jsp
1,293 logo.gif
1,993 lwl.gif
3,427 title.gif
-------
13,816 bytes = 13KB per page

At 342 pages per second, each at 13 KB, this constitutes 4442 KB per second. Or, about 35Mbps, which is enough to saturate most high-end Internet connections. Chapter 15, “Capacity Planning for the WebLogic Server,” covers how server output relates to Internet connection speeds.

In addition, another test run, not included here, compared the results of our performance testing efforts when multiple network interfaces were used. In that case, even though connections were spread across two different network connections, the throughput rate of pages per second did not change noticeably.

Simulating New Users

The Microsoft WAST has limited capabilities to simulate user behavior that is dependent upon the user name. The tool enables you to create thousands of virtual users, with each one interacting differently with this site. To simulate new users, we created 20,000 virtual users inside of WAST on each of the two client machines. Unfortunately, the WAST had difficulty generating large amounts of load and is only able to be single-threaded for virtual user tests.

We ran a single test with new user creations using the same server configuration previously determined optimal (see Table 14.5).

At this point, we would have attempted to continue increasing load upon the server if the tool had provided sufficient functionality. In this case, we deemed 26 new users per second to be an acceptable throughput. This would result in the capability for the configuration to add about 2.25 million users to the site per day.

Note also that the database CPU utilization became a factor in doing updates to the user table. Previously, the database utilization had been negligible for read operations because of the data caching functionality in WebLogic Server. For applications that are heavily oriented toward updating the database, make sure to pay attention to the database usage by your application. In many cases, database updates can cause the most substantial bottleneck for the application.

Simulating a Complete User Interaction

The goal of the final test was to simulate a complete and realistic user interaction with the application. To create this final test, a data read was added to the new user creation test completed in Run No. 9. In this test, a user first visits the main page for the site. The virtual user then creates a new account and enters an item for bid. The virtual user then browses the items before logging out (see Table 14.6).

This test represented a realistic user case and also stressed multiple parts of the application inside a single user account creation and logout from the system. With this data, it is very likely that the WebAuction application will perform extremely well in production situations from a performance standpoint.