Java supports overloaded methods by coupling the method with the descriptor, created based on the parameters the method takes. That way, internally print(int i) and print(char ch) are stored as two separate methods. Name mangling follows a convention mandated by the JVM specification, and because the bytecode stores the mangled names, you can get a glimpse of it here.

The fields and method descriptors are encoded based on their types. Table 17.1 shows the Java declared type and the corresponding field descriptor type that is used in the bytecode.

Table 17.2 shows some examples of Java declarations and their descriptors in the bytecode.

Method descriptors are created using the following format:

([<param1>[...<paramN>]])<return>

where

For example, a method that is declared as

Integer getIntProperty(String propertyName, int defaultValue)

would have the method descriptor

(Ljava.lang.String;I)Ljava.lang.Integer;

Certain special methods have predefined names. Static initializers are named <clinit>, and instance initializers and constructors are named <init>.

Each Java class is defined by a binary stream, typically stored in a class file, consisting of 8-bit bytes. The stream content is described by a pseudo structure given in the JVM specification and quoted here in Listing 17.3. Although this might look like too much information, the structures presented in this and the following sections will help in understanding the generation and instrumentation of bytecode later.

ClassFile {
    u4 magic;
    u2 minor_version;
    u2 major_version;
    u2 constant_pool_count;
    cp_info constant_pool[constant_pool_count-1];
    u2 access_flags;
    u2 this_class;
    u2 super_class;
    u2 interfaces_count;
    u2 interfaces[interfaces_count];
    u2 fields_count;
    field_info fields[fields_count];
    u2 methods_count;
    method_info methods[methods_count];
    u2 attributes_count;
    attribute_info attributes[attributes_count];
}

For clarity, the JVM specification defines pseudo-types u1, u2, and u4 representing unsigned 1-, 2-, and 4-byte types, respectively. Table 17.3 lists each field of the ClassFile structure and its meaning.

The constant pool deserves a little more attention because it is used frequently by other structures. Any text string found in a Java class, regardless of its nature, is stored in the same pool of constants. This includes the class name, names of fields and methods, names of classes and methods the class invokes, and literal strings used inside the Java code. Anytime a name or string needs to be used, it is referred to by an index into the constant pool. The constant pool is an array of cp_info structures, the general format of which is shown in Listing 17.4.

cp_info {
    u1 tag;
    u1 info[];
}

The actual items stored in the pool follow the structure that corresponds to the tag. For example, a string is defined using a CONSTANT_String structure and a reference to a field using CONSTANT_Fieldref. The list of structures and their contents can be found in the JVM specification.

The ClassFile structure uses three other structures: field_info, method_info, and attribute_info. field_info is similar to method_info, so we'll show only the method_info structure in Listing 17.5.

method_info {
    u2 access_flags;
    u2 name_index;
    u2 descriptor_index;
    u2 attributes_count;
    attribute_info attributes[attributes_count];
}

The meanings of the method_info fields are given in Table 17.4.

The attributes are used in the ClassFile, field_info, method_info, and Code_attribute structures to provide additional information that depends on the structure type. For example, class attributes include the source filename and debugging information, whereas method attributes include the bytecode and exceptions. Listing 17.6 shows the structure of attribute_info, and Table 17.5 lists its fields.

attribute_info {
    u2 attribute_name_index;
    u4 attribute_length;
    u1 info[attribute_length];
}

The compilers and post processors are allowed to define and name new attributes, as long as they do not affect the semantics of the class. For instance, AOP implementations can use bytecode attributes to store the aspects defined for a class.

When a compiler compiles Java source into bytecode, it performs extensive checks on syntax, keyword, operator usage, and other possible errors. This ensures that the generated bytecode is valid and safe to run. As the class is loaded into a JVM, a simplified subset of verifications is performed to ensure that the class file has the correct format and has not been tampered with. For instance, the bytecode verifier checks that the first 4 bytes contain the magic number and the attributes are of the proper length. It checks that the final classes are not subclassed and that the fields and methods have correct references into the constants pool; it also performs a number of other checks.

The home page for BCEL is located at http://jakarta.apache.org/bcel, where you can download the binary distribution, source code, and manual. The library provides an object-oriented API to work with the structures and fields that compose a class. It can be used to read an existing class file and represent it with a hierarchy of objects; transform the class representation by adding fields, methods, and binary code; and programmatically generate new classes from scratch. The class representation can be saved to a file or passed to the JVM as an array of bytes to support instrumentation and generation on-the-fly. BCEL even comes with a class loader that can be used to dynamically instrument or create classes at runtime. The class diagram of BCEL's main classes is shown in Figure 17.2.

Figure 17.2. Class diagram of BCEL's main classes.

Table 17.6 provides brief descriptions of the main classes we will use. The detailed information is available from BCEL JavaDoc.

As you can see, most of the classes are a direct mapping to the terms and data structures defined in the JVM specification.

Instrumenting is inserting new bytecode or augmenting the existing bytecode of a class. Products that produce runtime performance metrics of executing Java applications rely on instrumentation to collect the data. To get some practical experience, let's develop a framework that produces a log of method invocations at runtime. Omniscient Debugger, covered in Chapter 9, “Cracking Code with Unorthodox Debuggers,” uses a similar technique to record the program execution so it can be viewed later. Recording the method invocations at runtime provides the benefit of having a detailed log of the code, executed by the JVM.

To test the implementation, we'll use a class called SimpleClass defined in package covertjava.bytecode, with a main method that is shown in Listing 17.7.

public static void main(String[] args) {
    int i = 0;
    i = i + 1;
    System.out.println(i);
}

To keep the example simple, we are not going to write the entire invocation logging framework. Instead, we'll limit the implementation to the InvocationRegistry class with a static method, as shown in Listing 17.8.

public static void methodInvoked(String methodName) {
    System.out.println("*** method invoked " + methodName);
}

methodInvoked()is the entry point into the method logging framework, and it is used to log a method invocation. For each thread, it can store a call stack of methods, which can be saved or printed at the end of the application run. For now, the implementation just prints the method name to indicate that the framework was called for that method.

With the foundation laid, we can embark on implementing the class that will do the method bytecode instrumentation. We'll call it MethodInstrumentor and have its main() method take in the name of the class and the methods we want to instrument from the command line. When executed, MethodInstrumentor will load the given class, instrument the methods whose names match the given regular expression pattern by adding a call to InvocationRegistry.methodInvoked(), and then save the class under a new name. Running the new version of the class should log its method invocations in the Registry. MethodInstrumentor is located in the covertjava.bytecode package, and we are going to use a top-down approach to develop it. The main() method of MethodInstumentor is shown in Listing 17.9.

public static void main(String[] args) throws IOException {
    if (args.length != 2) {
        System.out.println("Syntax: MethodInstrumentor " +
                           "<full class name> <method name pattern>");
        System.exit(1);
    }
    JavaClass cls = Repository.lookupClass(args[0]);
    MethodInstrumentor instrumentor = new MethodInstrumentor();
    instrumentor.instrumentWithInvocationRegistry(cls, args[1]);
    cls.dump("new_" + cls.getClassName() + ".class");
}

After checking the command-line syntax, the MethodInstrumentor attempts to load the given class using BCEL's Repository class. The Repository uses the application class path to locate and load the class, which is just one of many alternatives to loading a class with BCEL. For some inexplicable reason, BCEL returns null on error conditions instead of throwing an exception, but for the sake of code clarity we won't check for it. After the class is loaded, an instance of MethodInstrumentor is created and its instrumentWithInvocationRegistry() method is called to perform the transformations. When finished, the class is saved to a file with a new name. Let's look at the implementation of instrumentWithInvocationRegistry shown in Listing 17.10.

public void instrumentWithInvocationRegistry(JavaClass cls,
                                             String methodPattern) {
    ConstantPoolGen constants = new ConstantPoolGen(cls.getConstantPool());
    Method[] methods = cls.getMethods();

    for (int i = 0; i < methods.length; i++) {
        // Instrument all methods that match the given criteria
        if (Pattern.matches(methodPattern, methods[i].getName())) {
            methods[i] = instrumentMethod(cls, constants, methods[i]);
        }
    }
    cls.setMethods(methods);
    cls.setConstantPool(constants.getFinalConstantPool());
}

Because we are going to be adding invocation of a method from a different class, we must refer to it by name. Recall that all names are stored in the constants pool, which means we'll have to add new constants to the existing pool. To add new elements to structures in BCEL, we must rely on the generator classes, which have a suffix Gen in their names. The code creates an instance of ConstantPoolGen that is initially populated with constants from the existing pool; then it iterates all the methods, harnessing the power of regular expressions to test which methods must be instrumented. When all the methods are processed, the class is updated with the new methods and the new pool of constants. The actual job of instrumenting is done in instrumentMethod(), as shown in Listing 17.11.

public Method instrumentMethod(JavaClass cls, ConstantPoolGen constants,
                               Method oldMethod) {
    System.out.println("Instrumenting method " + oldMethod.getName());
    MethodGen method = new MethodGen(oldMethod, cls.getClassName(), constants);
    InstructionFactory factory = new InstructionFactory(constants);
    InstructionList instructions = new InstructionList();

    // Append two instructions representing a method call
    instructions.append(new PUSH(constants, method.getName()));
    Instruction invoke = factory.createInvoke(
            "covertjava.bytecode.InvocationRegistry",
            "methodInvoked",
            Type.VOID,
            new Type[] {new ObjectType("java.lang.String")},
            Constants.INVOKESTATIC
            );
    instructions.append(invoke);

    method.getInstructionList().insert(instructions);
    instructions.dispose();
    return method.getMethod();
}

As you can see, instrumentMethod() programmatically creates bytecode instructions that correspond to a method call. The easiest way to select the correct JVM instructions and their parameters is to write the code in Java first, compile it, and then use something like the jClassLib viewer to see how it is translated to the bytecode. Then the corresponding bytecode can be constructed using BCEL objects.

The first thing instrumentMethod() does is instantiate a MethodGen object that is used to store the new bytecode. Then a factory to create and a list in which to store the instructions are created. If you have paid attention to this chapter and played with the jClassLib Bytecode Viewer, you might recall that a Java method call is represented by several bytecode instructions. First, the method parameters must be pushed onto the operands stack, and then the invokevirtual instruction is issued to transfer the control to the method (refer to Listing 17.2 for an example of method call bytecode). This is precisely what we have to insert into the method code before its existing bytecode. If we were working with the bytecode directly, we'd have to insert two constants into the constants pool: covertjava.bytecode.InvocationRegistry for the class name and methodInvoked for the method name. Luckily, BCEL does this for us because we are using the high-level classes such as InstructionFactory and PUSH, which automatically add constants to the pool. After the instructions are created, they are appended to the instruction list. When the code generation part is finished, the list is inserted into the generated method instructions and the method structure is returned.

To test that the instrumentation works, compile the classes and run MethodInstrumentor on SimpleClass.class using the following command line:

java covertjava.bytecode.MethodInstrumentor covertjava.bytecode.SimpleClass .*

A new class file called new_covertjava.bytecode.SimpleClass.class should be created in the current directory. Copy this class to the classes directory, overriding the existing SimpleClass.class file; then run the SimpleClass main() method. If all works well, you should see the following on the console:

C:ProjectsCovertJavaclasses>java covertjava.bytecode.SimpleClass
*** method invoked main
1

As you can see, the instrumented class starts by calling InvocationRegistry, which outputs the first line; then it executes its own body, which outputs 1.

Our second task is to learn how to generate a new class programmatically. As was mentioned earlier, this comes in handy for middleware products and frameworks that want to avoid source code generation. In our example, we'll create a generator of a value object that contains all the fields of the given class but no methods. The value object is a common design pattern used in distributed applications to pass data across the network. Admittedly, our generator will produce a very crude version of the value objects, but we'll make it a little interesting by ensuring that it generates only the fields whose values are meant to be retained.

Once again, we will use SimpleClass as a guinea pig in our experiment. SimpleClass defines five fields, as shown in Listing 17.12.

public int number;
protected String name;
private Thread myThread;
static String className;
transient String transientName;

We will write a ClassGenerator class in package covertjava.bytecode that takes two command-line parameters—a fully qualified class name and a regular expression pattern for field names to copy. The main() method of ClassGenerator is shown in Listing 17.13.

public static void main(String[] args) throws IOException {
    if (args.length != 2) {
        System.out.println("Syntax: ClassGenerator " +
                           "<full class name> <field name pattern>");
        System.exit(1);
    }
    JavaClass sourceClass = Repository.lookupClass(args[0]);
    ClassGenerator generator = new ClassGenerator();
    JavaClass valueClass = generator.generateValueObject(sourceClass, args[1]);
    valueClass.dump(valueClass.getClassName() + ".class");
}

Just as in MethodInstrumentor, the implementation checks the command-line syntax, loads the class, and then calls the generateValueObject() method that is shown in Listing 17.14.

public JavaClass generateValueObject(
    JavaClass sourceClass,
    String fieldPattern)
{
    String newName = sourceClass.getClassName() + "Value";
    ClassGen classGen = new ClassGen(
            newName,
            "java.lang.Object",
            newName,
            Constants.ACC_PUBLIC | Constants.ACC_SUPER,
            new String[] { "java.io.Serializable" });
    Field[] fields = sourceClass.getFields();
    for (int i = 0; i < fields.length; i++) {
        if (Pattern.matches(fieldPattern, fields[i].getName())) {
            int skipFlags = Constants.ACC_STATIC | Constants.ACC_TRANSIENT;
            if ((fields[i].getAccessFlags() & skipFlags) == 0) {
                fields[i].setAccessFlags(Constants.ACC_PUBLIC);
                addField(classGen, fields[i]);
            }
        }
    }
    return classGen.getJavaClass();
}

The implementation first creates an instance of ClassGen to represent the class being generated. The class has the same name as the parameter class, but with a Value suffix. It extends java.lang.Object and implements java.io.Serializable. Next, the implementation iterates the fields of the parameter class looking for names that match the given criteria. Using a bitmask, the implementation filters out the static and transient fields and copies the qualifying fields to the class being generated. The access modifier of the generated field is set as public for simplicity. After the generation is complete, the class representation is returned to the caller, which persists it to disk.

Running ClassGenerator on SimpleClass produces a file called covertjava.bytecode.SimpleClassValue.class in the current directory. Listing 17.15 shows the decompiled version of the class.

package covertjava.bytecode;
import java.io.Serializable;

public class SimpleClassValue
    implements Serializable
{
    public int number;
    public String name;
    public Thread myThread;
}

Voilá! All the appropriate fields of SimpleClass have been generated for SimpleClassValue.

A new open source project that is gaining momentum is the ASM bytecode manipulation library, hosted at http://asm.objectweb.org/. It is designed to achieve the same goals as the BCEL library but claims a significantly better performance because of a different implementation approach. BCEL creates a complete object tree representing a binary class file, down to the individual bytecode instructions. Therefore, it can potentially have hundreds of objects created for one class file, which can lead to performance degradation. Although having an object for every class file attribute is convenient, this approach can become costly for runtime bytecode manipulation if thousands of classes are instrumented.

ASM uses a visitor design pattern to avoid instantiating objects when not required. A class analyzer provided by the framework invokes a user-defined visitor class passing method and field data as parameters. For most of the parameters, the visitor implementation simply passes them to the next visitor, keeping the data in binary form. For those fields or methods that need to be changed, the visitor implementation obtains object representation from the framework and then manipulates the object. This way, most of the bytecode remains in binary form and the performance overhead is minimal.

If having minimal performance overhead of instrumentation is important, ASM is a better choice than BCEL. If clarity and simplicity of implementation are of a higher priority, I recommend BCEL.