Recent years have seen an increasing awareness of the importance of development best practices such as unit testing and test-driven development. However, unit tests are not all there is to testing. High-quality software needs to perform well under stress, using system resources such as memory and processor time efficiently. Performance bottlenecks may need to be identified and removed and memory leaks detected and eliminated. Profiling and performance testing play a crucial part in this side of application development. And software profiling is an area in which it is virtually impossible to work effectively without a good toolset.

Most profiling tools, both in the open source and commercial domains, need to be run as standalone applications. You start them up when you detect a memory leak or performance issue in your application and run them against your application. However, when you are writing or debugging an application, there is a lot to be said for being able to run a profiler directly from within your development environment. This enables you to integrate performance testing and profiling directly into your day-to-day development environment using the tool with which you are familiar. In Eclipse, you can do just this with the Eclipse Test & Performance Tools Platform, or TPTP.

The Eclipse IDE proposes a rich set of optional plug-ins designed to provide a coherent, integrated palette of extensions for the Eclipse development environment. This includes the convenient, although optional, profiling tool TPTP. TPTP provides a comprehensive suite of open source performance testing and profiling tools, including integrated application monitoring, testing, tracing, and profiling functionalities, as well as static code analysis tools. And in the Eclipse tradition, TPTP is more than simply a set of plug-ins; it is a platform that can be used to write test and performance tools integrated into the Eclipse development environment.

The Test & Performance Tools Platform contains an extensive set of profiling tools for Java applications. It is actually composed of four distinct but related components:

Profiling an application typically involves observing how the application copes under stress. A common way of doing this is to run a set of load tests on a deployed application and use profiling tools to record the application’s behavior. You can then study the results to investigate any performance issues. This is often done at the end of the project, once the application is almost ready for production.

TPTP is well suited to this type of task. A typical use case would be to run load tests using a load-testing tool such as JMeter (see Chapter 16), and record and analyze the performance statistics using the TPTP tools.

However, this is not the only way you can profile an application with TPTP. As a rule, the earlier you test, the less problems you have later on. With TPTP, you can profile your code in a wide range of contexts, including JUnit test cases, Java applications, and web applications. And it is well integrated into the Eclipse IDE., so there is no reason not to start preliminary performance tests and profiling early on.

TPTP lets you test several aspects of your application’s behavior, including memory usage (how many objects are being created, and how big they are), execution statistics (where did the application spend the most of it’s time), and test coverage (how much of the code was actually executed during the tests). Each of these can provide invaluable information about your application’s performance.

The sort of testing we are talking about here is not optimization as such. Optimization involves fine-tuning application performance using techniques such as caching. It is a highly technical activity, and it is best done at the very end of the project.

This type of preliminary performance testing and profiling discussed here simply involves making sure that the application performs correctly from the start, and that there are no coding errors or poor coding practices that will penalize performance later on. Indeed, fixing memory leaks and avoiding unnecessary object creation is not optimization: it is debugging and, as such, should be done as early as possible.

In this chapter, we will look at how you can use TPTP to guarantee high-quality and high-performance code, even during unit and integration testing.

The easiest way to install TPTP on an existing Eclipse installation is to use the Remote Update site (see Figure 19-1). Open the Remote Update window (Help→Software Updates→ Find and Install), and select the Discovery Site for your version of Eclipse. For the Europa edition, for example, this is called the “Europa Discovery Site.” Here, Eclipse will propose the set of plug-ins. The TPTP tools are listed under “Testing and Performance.” The easiest option, albeit the most time-consuming, is just to install all the proposed plug-ins. Even if you don’t install the entire toolset, you will still need to install some other components needed by TPTP, such as “Charting and Reporting,” “Enabling Features,” and “Data Tool Performance.”

Figure 19-1. Installing TPTP from the Europa Discovery remote site

An important thing to know about TPTP is that, at the time of this writing (using Eclipse 3.3 Europa), the TPTP profiling tools do not support Java 6. TPTP relies on JVMPI (JVM profiling interface), which it uses to capture data about applications running in the JVM. Now JVMPI was dropped in Java 6 in favor of the more modern and flexible JVMTI (JVM Tool Interface). If you try to run TPTP using a Java 6 VM, you will obtain an error along the lines of “FATAL ERROR: JVMPI, an experimental interface, is no longer supported.” So if you are using Java 6, make sure that you are running Eclipse under Java 5 if you want to use TPTP.

You can run Eclipse using a different JVM using the vm command-line option. Here is how you might do this on a Windows machine:

D:	oolseclipseeclipse.exe -vm "C:Program FilesJavajdk1.5.0_10jreinjavaw.exe" 
-vmargs -Xmx512M

Or under Linux, you might do something like this:

$ /usr/local/eclipse/eclipse -vm  /usr/lib/jvm/java-1.5.0-sun/bin/java
 -vmargs -Xmx512M &

For the same reason, when you run your code from within Eclipse, you need to use a Java 5 JVM. If you have several JVMs configured in your Eclipse preferences, make sure you are compiling and executing your project in Java 5.

One of the best ways to check that performance is (and remains) up to scratch is to write comprehensive performance-oriented unit (or “integration,” if you prefer) tests for each of your use cases. In my experience, this is also one of the best ways of isolating and correcting performance issues. This involves writing simple performance-oriented unit tests for your key business functions. This approach has the additional advantage of progressively building a suite of regression tests for future development. Here, we go through the basics of profiling with TPTP, by looking at how to profile the behavior of an application using simple, performance-oriented unit tests.

As a rule, you should try to profile code that is as close as possible to the production code. Many people use mock objects to replace DAOs (Data Access Objects) for unit tests, and it can be a powerful technique to speed up the development lifecycle. If you use this type of approach, by all means, run your profiling with these tests: it can reveal useful information about memory usage and test coverage. However, the performance tests are of limited value, since performance in a database-related application is often dominated by database performance, so any serious performance testing should be done in this context. A good compromise is to run performance tests against an embedded Java database such as JavaDB/Derby or HSQLDB—this will give you an idea of how your application behaves against a real database, without incurring the overhead of network traffic or having to set up and maintain your own dedicated test database instance.

Throughout this chapter, we are going to test a simple web application that manages a database of model planes. In this web application, users can consult the list of known plane types, select a plane type, and then view the corresponding model planes.

For our first profiling exercise, we want to make sure that the application home page, which involves displaying the list of all available plane types, will not present any performance issues. This page will be heavily used, so it is important that it can support a high load. Let’s see how we would do this using JUnit and TPTP in Eclipse.

In the application, plane types are represented by the PlaneType class. The DAO class for plane types implements the following interface:

public interface PlaneTypeDAO {
    PlaneType findById(long id);
    List<PlaneType> findAll();
    public void save(PlaneType planeType);
    public void delete(PlaneType planeType);
}

To list all available plane types, we need to invoke the findAll() method. It is important that this method always performs efficiently, with a minimum of SQL and in a minimum of time. Because we are using JUnit here, we will use Mike Clark’s excellent performance unit testing library, JUnitPerf (see Chapter 28), to implement some simple performance tests on this method. The full unit test is shown here:

public class ModelPlaneDaoPerfTests extends TestCase {

    private PlaneTypeDAO dao;

    public ModelPlaneDaoPerfTests(String value) {
        super(value);
    }

    public void setUp() throws SQLException {
        ApplicationContext ctx = SpringUtilsTestConfig.getApplicationContext();
        dao = (PlaneTypeDAO) ctx.getBean("planeTypeDAO");    
    }
    
    public void testFindAll() {
        List<PlaneType> planes = dao.findAll();
        assertTrue(planes.size() > 0);
    }
    
    public static Test suite() {        
         TestSuite suite = new TestSuite();
         TestCase testCase = new ModelPlaneDaoPerfTests("testFindAll"); 
         suite.addTest(testCase);
         suite.addTest(new TimedTest(testCase, 1000));
         return suite;
    }
}

Our first unit test (testFindAll()) simply invokes the findAll() method, and checks that the results list is not empty. The setUp() method, executed before each test, obtains a DAO using a Spring application context. Behind the scenes, this instantiates the DAO, along with the appropriate JDBC data source and Hibernate session, using an embedded Java database. It also populates the test database with test data.

The performance testing is done in the suite() method, where we use the TimedTest decorator to ensure that this test case runs in less than a second. Note that, for fairness, we add the undecorated test case to the suite first. This ensures that the test database and application environment have been set up before we time the test.

You can run this test case easily in Eclipse in the usual way, using “Run As...JUnit Test.” If you have JUnit 4 libraries on your build path, don’t forget to run this class in Eclipse as a JUnit 3 test case. You can do this to make sure that the test runs correctly. Because profiling is a fairly time-consuming operation, this is usually a good idea. However, what we really want to do is profile the application’s behavior when the tests are running. To do this, we need to use the TPTP profiling tools.

The first thing that you need to do is to set up a profile to run this particular test. Select “Profile As→Profile…” in the main Eclipse menu (see Figure 19-2).

Figure 19-2. Profiling a JUnit test using the contextual menu

This opens a Wizard in which you can configure different sorts of testing profiles, shown in Figure 19-3. This is where you can set up configurations for profiling different sorts of applications such as web servers, Java applications, or JUnit tests. We are interested in the latter. Your unit test class should appear under the JUnit entry. If you’ve done it before, you can also create a JUnit test profile directly by selecting “Profile As...JUnit test” in the contextual menu.

Figure 19-3. Creating a new TPTP profile configuration

TPTP is quite flexible, and this screen uses many of the same configuration options that you find when running, testing or debugging code in the Eclipse IDE. In the “Test” tab, you can either profile unit test classes individually, or group them by project or package. The “Arguments” tab lets you specify runtime arguments, and the “Environment” tab lets you define environment variables. There are also many options that are specific to the profiling tools. In the “Destination” tab, you can specify an external file where profiling data is to be saved for future use. But the most useful is the “Monitor” tab (see Figure 19-4), where you specify which performance-related data you want to record and study.

Figure 19-4. Launch configuration properties for a TPTP profile

The “Monitor” tab lets you define the type of profiling data you want to record. Profiling with Java 5.0 and higher uses a quite different library to the one used in previous versions. The most useful options here are the following:

Double-clicking on “Basic Memory Analysis” or “Execution Time Analysis” enables you to specify some additional options.

You can either run the profiling tool directly from this window or, using the contextual menu positioned on the test class you want to profile, via the “Profile As” contextual menu entry. The profiling tool may take some time to run, depending on how big your test cases are. Once done, Eclipse will switch to the “Profiling and Logging” perspective from which you can display details of the results of each type of profiling (see Figure 19-5).

Figure 19-5. Visualizing TPTP memory analysis statistics

This perspective regroups a rich collection of views, which you can open using the contextual menu over the various profiling analysis results. The memory analysis view (see Studying Memory Use with the Basic Memory Analysis Results and Figure 19-5) provides information about the number of instances of each class currently in memory, including how many have been collected by the garbage collector, and how many are in use. This can help in identifying and isolating memory leaks. The execution time analysis (see Analyzing Execution Time) helps you study the application’s dynamic behavior.

In the following sections of this chapter, we will take a closer look at how to use these views.

The memory analysis provides useful information about which objects, and in what quantity, are created by the application. The “Memory Statistics” view (see Figure 19-6) displays the number of objects created by the application. The results can be organized by package (in the form of a tree view), or as a list of classes or instances.

Figure 19-6. The Memory Statistics view

This data can give you an idea of how many objects of each type are being created; unusually high numbers of created objects (especially high-level objects such as domain objects) should be treated with suspicion. This view displays information about:

In many cases, the dynamics of object creation are just as important as the raw numbers at a given point in time. As the application runs, you can update the view using the “Refresh Views” in the contextual menu. The small triangles (actually, deltas) in Figure 19-6 indicate which values have changed since the last refresh. This helps you identify which objects were created during application initialization and which ones are being created as the application runs. The “Show Delta Columns” button (the delta symbol in the view’s toolbar) adds an extra column for each field, showing how many instances were created since the last time the view was updated.

The “Show as percentage” button (the percentage symbol in the view’s toolbar) shows the number of instances and size of each class as percentage values. This is a handy way of checking whether any particular classes take up a suspiciously high proportion of memory space or have an unusually high number of instances compared to the other classes. Again, this can be a symptom of a memory leak.

Another useful tool for detecting memory leaks is the “Collect Object References” view. To obtain this data, you need to activate reference collecting. After you start the profiling, click on the monitor entry and select “Collect Object References” in the contextual menu (see Figure 19-7).

Figure 19-7. Activating reference collection

Then open the “Collect Object References” view via the contextual menu (“Open with→Object References”). You will obtain a list of classes with the number of references to each class. This can give some clues concerning possible memory leaks.

You can also force the JVM to run a garbage collection, by selecting “Run Garbage Collection” in the contextual menu. This is a good way to flush out memory leaks (see Detecting and Identifying Memory Leaks with the JDK Tools). Indeed, a good way to check for potential memory leaks is to run the garbage collector and then to look at the remaining live instances.

Execution time analysis provides useful information about the more dynamic aspects of the application, where the code spends the most time, and what is the flow between the objects.

The “Execution Statistics” view, shown in Figure 19-8, gives a good view of where your application is spending its time. The organization by package (“Package Level Information”) lets you drill down to the classes (“Class Level Information”) and methods (“Method Level Information”) that are taking the most time to execute. Alternatively, you can display a list of classes or methods, which is useful for sorting by execution time.

Figure 19-8. Execution statistics

Clicking on a method will open the “Method Invocation Details” view (Figure 19-9), which displays some finer details on the number of times that the method is being called, where it is being called from, and what other methods it invokes. You can also navigate through the methods listed in the view: clicking on a method will take you to the invocation details for that method. Although this view is not as well integrated into the source code views as some of the commercial tools (where it is possible to drill down into the source code itself), it can give some vital clues as to which methods may be performing badly.

Figure 19-9. The Method Invocation Details view

Another interesting view that gives a useful description of the dynamics of the application is the “UML Trace Interactions” view (see Figure 19-10). This view is an annotated UML interactions diagram that displays an exhaustive view of every interaction between every class during the course of the application execution. If you move the mouse over a particular interaction, Eclipse will display how long this interaction took. This information is enhanced by a colored bar in the lefthand margin, which gives an indication of the time spent in each phase: the redder the bar, the more time was spent in these interactions. In other words, this can give you a good high-level view of any application hotspots.

Figure 19-10. The UML Trace Interactions view

The “Coverage Statistics” view (see Figure 19-11) provides information on which methods were used (and, therefore, tested, at least to some extent) by the test cases you just ran. The coverage statistics are a nice feature, although they don’t provide the same level of detail as dedicated coverage tools such as Cobertura do (see Chapter 12), which provide line-precision coverage data as well as statistics on both line and branch coverage. Nevertheless, it does have the advantage of providing real-time coverage results, and, currently, only commercial code coverage tools such as Clover and jCoverage provide line-level coverage reporting and full IDE integration.

Figure 19-11. Coverage statistics view

When you are profiling an application, there may be classes that you don’t need or don’t want to include in your analysis. For example, if you are testing a web application, you may want to exclude classes belonging to the web server. You can set this up when you set up your profiling configuration (using the “Profile As...” menu). Double-click on the “Java Profiling” entry to modify the profiling options. The first screen lets you select and customize a filter set for your profiling session (see Figure 19-12). Filter sets let you exclude certain classes or packages from data collection, which speeds up the profiling process and makes it easier to see relevant data.

Figure 19-12. Refining results by applying a filter

TPTP comes with a few predefined filter sets that you can customize, or you can create a new one. A filter set is simply a list of rules that include or exclude classes or methods based on a regular expression. Although the most common use is to exclude entire packages, you can also exclude (or include) specific classes, or even a particular method in a particular class.

Filters are also used to refine and clarify the results of the profiling process.

Once you’ve started profiling and have displayed the profiling result views, you can narrow down the selection using the Filters menu (see Figure 19-13). You can choose from a few handy predefined filters such as “Highest 10 total size” (for Memory Statistics views) and “Highest 10 cumulative time” (for Execution Statistics views). Alternatively, you can create your own customized filter by selecting “Manage Filters….”

Figure 19-13. Refining results by applying a filter

These filters are a powerful tool. TPTP lets you define sophisticated filter rules (see Figure 19-12). It is easy to filter the displayed data by package, class, or method names, using wildcard-based expressions. A common use is to filter results down to a particular package or group of packages, which makes it easier to focus your optimization efforts on certain classes or isolate memory leaks. In addition, using the “Advanced” tab, you can build more elaborate rules using the other collected fields, such as execution time or number of instances. For example, you can set up a filter to display only the methods whose average time is greater than half a second.

TPTP can profile a wide range of tests and applications. If you are developing a web application in Eclipse, it is an easy matter to profile the application using the TPTP profiling features. This ties in well with a development environment in which an application is written, unit tested, and deployed for functional testing within the Eclipse workspace. Open the Server view, and right-click on the server you want to profile, and select “Profile” (see Figure 19-14).

Figure 19-14. Profiling a web application

This will open the “Profile on server” configuration screen (see Figure 19-15). Choose the Java Profiling Agent for the PID corresponding to your server (generally, there is only one), and configure the monitoring options as above. Finally, to collect data, you need to start the monitoring process manually. Go to the “Profiling Monitor” view and select “Start Monitoring” in the contextual menu. Once you’ve done this, you can profile your web application just as you would an ordinary Java application.

Figure 19-15. The Profile On Server configuration

The Eclipse Test & Performance Tools Platform is a valuable addition to the Eclipse IDE toolkit. The wide range of performance testing helps you to guarantee high-quality and high-performance code right from the first unit tests. TPTP is certainly not as developed as some of the commercial tools out there such as OptimizeIt and JProbe, which often have more sophisticated reporting and analysis functionalities and a more polished presentation. However, commercial profiling tools tend to be notoriously expensive, and it is often difficult justifying their use in all but the most dire of circumstances. Although it is still relatively young, TPTP is a powerful and capable product, and it can certainly provide valuable profiling data that many projects would otherwise have to do without.

It is worth noting that NetBeans also comes with an excellent, integrated profiling tool.