Few people would perceive the software development process as being similar to a production line. Developing software is a creative task and is different for each system. Nevertheless, the many and varied activities that make up a software development project are common to all projects and are frequently repeated by each member of the team on an almost daily basis.

A software engineer’s tasks include such common activities as pulling down the latest version of the application from source control; building, deploying, and testing new application functionality; and checking the defects log for errors raised against previous releases. These tasks and many more make up a typical developer’s day.

Similar recurring tasks exist for other project roles. The quality assurance team has a process for installing the latest releases from the development team and preparing a suitable environment for testing. The team may perform this task manually or rely upon a script for creating a clean environment for each test cycle.

For rapid development to take place, all of these activities must run smoothly and efficiently.

The software development environment and the processes and procedures that drive it must ensure all team members can work expediently and accurately. To achieve this optimal state for the project, teams should look to the practices of production engineers and perform their own time-and-motion studies on those everyday activities that are performed so frequently.

This shouldn’t be a matter of hiring production engineers with stopwatches to come in and time people as they work. Instead, efficiency is the responsibility of all team members, and everyone should seek ways to improve the process.

Feeding this information back into a company’s development practices is critical to maintaining a companywide adaptive foundation for rapid development, as passing these process improvements from project to project helps establish productive environments for future applications.

With the importance of time and motion in mind, let’s consider the intricacies of the build process.

Like software systems, build systems require careful design. Regardless of the type of software under development, the requirements for a build system tend to be common between projects. Here is a summary of some of the main requirements:

These requirements are common to most software projects. The development of enterprise-level software introduces a further set of requirements peculiar to the J2EE platform.

The build process for a conventional Java-based application often builds only a single target, thereby making for a simple build environment. Unfortunately, a J2EE application is not so straightforward; it involves numerous intricate steps to generate multiple targets.

Unlike its J2SE counterpart, a J2EE application comprises a set of components, which collectively make up the whole application. Each component can have vastly different build requirements. EJB components require specialized compilation tasks for the generation of stub and skeleton implementations. Code generation tools such as XDoclet are becoming increasingly popular for the effort they save, but add additional steps and complexity to the build process.

In addition to these specialized tasks, J2EE components require packaging. EJB components are wrapped in JAR files, and Web applications are packed in a Web archive (WAR) file. Finally, all components can be bundled in an enterprise resource file (EAR), the format recommended for J2EE deployments by application servers.

To summarize, a J2EE build typically includes the following tasks, which are additional to traditional Java builds:

All of these tasks devour time. Packaging, which requires moving files around the file system into a structure where they can be bundled into a format ready for deployment, is a particularly time-consuming process. Likewise, the process of deploying components to the server and placing them in an executable state is also potentially time consuming.

Shortening the length of time spent building and deploying is a case of reducing the amount of work the build system has to undertake. This gives rise to the concept of a build system capable of performing minimal builds and minimal deployments. We consider each concept in turn.

Hot deployment is a concept embraced by most application server vendors and refers to the ability of the server to deploy an updated J2EE application into a live environment without having to shut down either the server or the running application.

For a production system, hot deployment has obvious benefits for system availability. For development teams, however, we require a slightly different form of hot deployment that is perhaps better called automatic deployment.

This concept has the application server continually polling for new files. On detecting a new file, the application server immediately loads the change and integrates it into the current application.

In a fast-paced development environment, this functionality is a major timesaver for the software engineer, as no manual steps are needed for the application server to apply the software changes other than deploying the changes to the server.

Automatic deployment is a proprietary feature and is not covered by the J2EE specification, which only stipulates the deployment of EAR, WAR, JAR, and RAR files. Application servers, such as WebLogic from BEA, have gone beyond the J2EE specification and provide support for the deployment of applications in an expanded, or exploded, format rather than an archive file. This approach, which BEA recommends for deployment to its server, provides support for the minimal deployment approach.

WebLogic server supports automatic deployment if the server is running in development mode. This is achieved by setting the system property -Dweblogic.ProductionModeEnabled to false.

You must check the details of your particular application server to determine what it provides in terms of minimal and hot deployment capabilities.

Having covered some of the main requirements for a build system, we can now look at producing the build process itself. For this, we need to adopt a suitable build tool.

To appreciate the impact of build dependencies, let’s consider the example build.xml in Listing 12-1. In this example, the build file instructs Ant as to the build order of each of the targets.

<project name="ant-build" default="compile">

  <target name="compile"
          description="Compile all Java source">
    <javac srcDir="."/>
  </target>

  <target name="clean"
          description="Removes all class files">
    <delete>
      <fileset dir="." includes="*.class"/>
    </delete>
  </target>

  <target name="build"
          depends="clean, compile"
          description="Rebuilds all source"/>

</project>

The example build file in Listing 12-1 contains three build targets: compile, clean, and build. The compile target is set as the default for the project and is run whenever Ant is executed unless an alternative target, such as ant build, is explicitly specified.

Running the build file for the first time with the default compile target results in the compilation of all Java source in the base directory. For this example, assume we have a single HelloWorld.java file.

Running the build file a second time is distinctly quicker, as Ant determines all class files are up-to-date and no compilation is necessary. How is this possible given the build file provided no dependency information between HelloWorld.java and HelloWorld.class?

The secret lies in the <javac> task, used to compile the Java source. This task has built-in dependency rules and knows to associate *.java and *.class files. Therefore, for Java compilations, Ant supports the minimal build approach we require for fast builds. Unfortunately, Ant is not as knowledgeable about other file types. This is a problem, especially for J2EE builds that rely on many custom build steps.

Listing 12-2 revises the previous example to illustrate the problem.

<project name="ant-build" default="compile">

  <target name="generate"
          description="Long running build task">
    <ejbdoclet>
      .
      .
      .
    </ejbdoclet>
  </target>

  <target name="compile"
          depends="generate"
          description="Compile all Java source">
    <javac srcDir="."/>
  </target>

  <target name="clean"
          description="Removes all class files">
    <delete>
      <fileset dir="." includes="*.class"/>
    </delete>
  </target>

  <target name="build"
          depends="clean, compile"
          description="Rebuilds all source"/>

</project>

A new target, <generate>, has been added to the build. This target invokes a code generator, for example, XDoclet, which processes those files containing annotations.

Before the <compile> target can begin, the <generate> target must have completed. This dependency between the two targets is expressed using the special Ant attribute depends. With this dependency defined, Ant always runs the <generate> target ahead of <compile>.

The relationship specified between the two targets is procedural: Ant does not check file timestamps to determine if the <generate> target must be run.

Assume our example project comprised 10,000 source files, with 5,000 of them marked up with XDoclet-style annotations or attributes. Modifying an annotated file requires running XDoclet to pick up the change. However, if a file without annotations is changed, then we can safely skip the <generate> task and run only <compile>. Because the <compile> target uses the <javac> task, only the affected source file is compiled.

Ideally, we want our code generator, be it XDoclet or some other tool, to have the same functionality as the <javac> task. If only one of our 5,000 annotated files is modified, then the code generator should process only a single file.

With this approach, the steps of the build process are as follows:

As it stands now, the <generate> target will pick up all 5,000 files, thereby resulting in the need for <javac> to undertake a significant recompilation effort.

This is a problem. Our build process does not support the concept of minimal builds, and our time-and-motion expert is far from pleased.

Using a Make tool, we could have defined build rules to instruct Make to run the code generator against modified files only. Unfortunately, Ant does not support this form of deterministic build process directly. More importantly, neither does the XDoclet Ant task.

Nevertheless, the authors of Ant recognized the importance of build dependencies and added support for this feature as a set of core tasks. We still have work to do if we are to make use of this functionality, as it is not default behavior. To understand how build dependencies can be enforced, let’s leave XDoclet and consider another example.

The build.xml file shown in Listing 12-3 demonstrates the use of the <uptodate> task to define a conditional build dependency between two targets. In this example, the dependency defines the need to package Java binaries into a single JAR file.

<project name="depend-example" default="make">

<!-- Build file with dependency defined between
     package and compile targets -->

<property name="src.dir" location="src"/>
<property name="bin.dir" location="bin"/>
<property name="dist.dir" location="dist"/>

<target name="compile"
        description="Compiles all Java source">
  <javac srcdir="${src.dir}"
          destdir="${bin.dir}"/>

  <!-- Check if files have been updated -->
  <uptodate property="package.notRequired"
            targetfile="${dist.dir}/app.jar">
    <srcfiles dir="${bin.dir}" includes="**/*.class"/>
  </uptodate>
</target>

<target name="package"
        depends="compile"
        unless="package.notRequired"
        description="Produces JAR file">

  <jar destfile="${dist.dir}/app.jar"
        basedir="${bin.dir}"/>
</target>

<target name="clean"
        description="Removes all class files">
  <delete>
    <fileset dir="${bin.dir}"/>
    <fileset dir="${dist.dir}"/>
  </delete>
</target>

<target name="make"
        depends="package"
        description="Incremental build"/>

<target name="build"
        depends="clean, make"
        description="Rebuilds all source"/>

</project>

The build process shown in Listing 12-3 involves two steps. First, the compile target compiles all code in the src directory with <javac>, directing all output to the bin directory. Second, the package target collects the contents of the bin directory into a single JAR file, placing the archive in the dist directory ready for deployment.

Two top-level build targets are responsible for performing these steps:

Packaging large numbers of files into JAR files is a time-consuming process, so it is of benefit to perform this task only when needed. We make the running of the package target conditional with the use of the unless attribute.

The unless attribute instructs Ant to skip the execution of the target if the associated property has been set to any value. Conversely, the if attribute of <target> instructs Ant to run the target if the property is set.

With properties and the if and unless attributes of the <target> task, we can control at build time which targets are run. From the example in Listing 12-3, the package target references the package.notRequired property to determine if it should generate a JAR file. This property is set as the final act of the compile target upon which the package target is dependent.

The task <uptodate> is used to set the package.notRequired property in the example. This conditional task sets the property identified with the property attribute if the target file is more up-to-date than the source. In this case, the <uptodate> task checks to see if any class files have been generated since the JAR file was last produced. If any class files prove to be more recent than the JAR file, then the package.notRequired property is left unset and the package target is allowed to execute.

The conditional support provided by Ant makes it is possible to create sophisticated build scripts that support incremental builds. However, adding conditional behavior to the build process can significantly increase its complexity. The example in Listing 12-3 covers a very basic case. A typical J2EE build involves many more targets, and adding dependency information therefore adds significant complexity to the build process.

To prevent additional build logic making build files unduly complex, good design dictates the build process be broken down into discrete blocks, or modules. With this approach, modules of the system can be built in isolation, and dependencies can be more easily defined between build targets.

The source directory contains all program artifacts involved in the construction of the application. This is not limited to Java source but encompasses such artifacts as build scripts, property files, deployment descriptors, database scripts, and XML schemas. Figure 12-4 shows the organization of the source directory.

Figure 12-4. Source directory structure.

The project is structured around the creation of a single EAR file, with each module of the distribution being maintained as a subproject.

From the directory structure shown in Figure 12-4, we have several modules:

Each module that makes up the J2EE distribution possesses its own build.xml file. These are self-contained build files, capable of building each module in isolation. The build files are also aware of dependencies that may exist between the different modules. A Web application will likely call into a business layer and so may have a dependency on the EJB module. This should all be managed by each build file.

All build files should be consistent between modules, with each build file exposing the same set of targets. A single controlling build file resides at the top of the project directory structure. This is a delegating Ant build file, again exposing the same targets as the build files of each subproject.

Calls to targets in the project build file are cascaded down to the equivalent targets in each of the subprojects. Although each build file has the same set of targets, the work they perform is in context with the type of module they are responsible for building. On a Web module, the target package produces a WAR file, while on the top-level project build file, the package target generates an EAR file.

The build order of the subprojects is important and should commence with the low-level modules and work upwards. Building from bottom-to-top and ensuring hierarchical dependencies are in place provides a build system that is easy to maintain and consequently produces consistent results.

This directory holds all external libraries. The build scripts reference this directory structure during compilation. When assembling a distribution, libraries required at runtime are copied from this directory as needed.

All output from the source directory is directed to the build directory. Keeping source files separate from built files is important for good housekeeping. A clean separation of the two directories means the build directory is disposable and can simply be deleted at any time. This makes the writing of the clean target a trivial task.

The structure of the build directory should mimic that of the source for consistent navigation of all built files. It is considered good practice to direct all output from the build process to the build directory, including output from code generators such as deployment descriptors and remote interfaces. The simple rule to follow is to place all output that will be created each time a full build is run under the build directory, regardless of whether the output is source or binary.

Figure 12-5 illustrates the structure of the build directory based on the project structure defined for the source in Figure 12-4.

Figure 12-5. The build directory structure.

Having built all the necessary parts of the project, the next step is to package all components ready for deployment to the application server. For this task, we make use of the distribution directory.

The distribution directory is convenient for packaging and deployment purposes. It is possible to undertake all of this work within the build directory, but having a separate directory for all packaged files simplifies the process of generating formal releases or passing new versions of the system onto a formal test environment.

Figure 12-6 depicts an example of a distribution directory. Again this directory structure adopts the module structure seen in both the source and build directories.

Figure 12-6. The distribution directory.

Having the distribution directory fall under the main project directory is optional. For development purposes, you may wish to generate the packaged files directly into the directory structure of the application server to take advantage of its hot deployment capabilities. In some cases, the application server may call for all modules to be deployed in an uncompressed format, bypassing the need for extensive packaging.

The project structure outlined is a guideline only. The requirements of your project dictate the workings of your build process, and you should design a project structure according to your specific needs.

Implementing a basic Ant task involves the following steps:

Listing 12-5 shows an implementation of an Ant task in Jython. The task has a single attribute called message. Invoking the task from an Ant build file prints out the message held in the attribute and then prints further information on the owning target’s dependency information.

You can use this example as a template for creating your own Jython Ant tasks.

from org.apache.tools.ant import Task
from org.apache.tools.ant import Target

class RapidTask(Task):

  # Overrides the Task.execute() method
  #
  def execute(self):
    "@sig public void execute()"

    # Print out properties of task
    #
  self.log('Task Name: ' + self.taskName)
  self.log('Description: ' + self.description)

  # Print out the message property
  #
  self.log('Message: ' + self.message)

  # Get the owning target and list dependencies
  #
  target = self.owningTarget

  self.log('Target name: ' + target.name)

  for dependency in target.dependencies:
    self.log('	Depends: ' + dependency)

# Ant attribute setter method
#
def setMessage(self, message):
  "@sig public void setMessage(java.lang.String str)"
  self.message = message

The Jython class shown in Listing 12-5 has a few differences from a normal Jython class, as it must be complied if Ant is to use it. Compiling Jython to standard Java bytecode involves moving from the type-less world of Jython to the strongly typed world of Java. This presents a few problems when defining methods on Java classes as Jython methods have no signature.

To get around this problem, it is necessary to embed a string in the __doc__ namespace of each method that defines the Jython method’s Java signature. Here are the embedded method signatures for both the setMessage() and execute() methods on the RapidTask class:

@sig public void setMessage(java.lang.String str)
@sig public void execute()

The @sig preamble at the start of the string tells the jythonc compiler the Java method signature to generate. This is all jythonc needs to compile the class to bytecode.

The Jython class is compiled from the command line:

jythonc -a -c -d -j rapidTask.jar RapidTask.py

The parameters -a, -c, and, -d instruct jythonc to include the entire core Jython libraries in the build. The –j rapidTask.jar specifies to place the compiled classes into the named JAR file.

With the new task successfully compiled, it is ready for use from within a build file.

The new task is included in the build script using the <taskdef>, which defines the name of the new task and the class to load. Listing 12-6 shows an example of an extract from a small test build file for the new Ant task.

<target name="test"
      depends="clean, package"
      description="Access the Jython Ant task">

  <!-- Declare the new Jython task -->
  <taskdef name="Rapid"
            classname="RapidTask">
    <classpath>
      <pathelement location="${task.jar}"/>
    </classpath>
  </taskdef>

  <!-- Set a property on the task -->
  <Rapid description="Example task"
          message="My Jython Task" />

</target>

Invoking the <test> target generates the following output from the Jython Ant task.

test:
    [Rapid] Task Name: Rapid
    [Rapid] Description: Example task
    [Rapid] Message: My Jython Task
    [Rapid] Target name: test
    [Rapid]     Depends: clean
    [Rapid]     Depends: package

BUILD SUCCESSFUL

The Jython task uses methods on the Task class to obtain information about its environment, such as the name of the task, the name of the owning target, and a list of all dependencies. It also prints out the message attribute we added as part of the task.

That’s all there is to building an Ant task with Jython. Mixing Ant and Jython gives us the structure and control of Ant coupled with the rapid scripting capabilities of Jython—the perfect combination for constructing build processes.

Ant is a tool for building software. The folks at Apache chose Ant as the basis for Maven, a tool for performing project management, build, and deployment tasks. Maven provides a formal framework for developing software artifacts and strictly enforces dependencies between build targets.

For more information on Maven, and to download the latest version of the software, see http://maven.apache.org.

An important part of the software development environment not covered in this chapter is the mechanism for the integration of software produced by team members. Bringing together newly developed pieces of functionality can often result in a broken build process, a major source of wasted time on any project.

A solution to the software integration problem is to adopt a policy of continuous integration. This idea involves integrating software changes frequently, possibly several times a day, and is the brainchild of Martin Fowler and Matthew Foemmel. The pair wrote their ideas up in a paper that can be viewed at http://www.martinfowler.com/articles/continuousIntegration.html.

The concept of continuous integration is backed by the open source product CruiseControl, a framework that integrates with your build system to perform regular builds and reports on build status. The CruiseControl software is obtainable from http://cruisecontrol.sourceforge.net.

Finally, for anyone who wishes to experiment with make files, a copy of GNU Make is available from http://www.gnu.org/software/make.