Chapter 15

Serialization

Interoperation between a client and server can be separated into two distinct parts: how the data is transmitted, and what is transmitted. The previous chapter gave one method for how to transmit and receive data, and this chapter explains different methods for representing your data over that transmission.

The methods here are not specific to network communication. Object serialization is simply a way of “exporting” Java objects from the JVM: The serialized data can be written to disk or another I/O interface rather than to the network.

For convenience, most of the listings in this chapter read and write serialized data to and from the filesystem, using FileInputStreams and FileOutputStreams. These can be substituted for any InputStream and OutputStream, such as the ones provided by HttpServletRequest and HttpServletResponse if you wanted to use the data in an HTTP server.

Reading and Writing Java Objects

The standard Java libraries provide a pair of classes for writing and reading Java objects: ObjectOutputStream and ObjectInputStream. These are part of the InputStream and OutputStream family of classes.

For writing objects, ObjectOutputStreams are constructed using the Decorator Pattern, and require another OutputStream to physically write the serialized data. The ObjectOutputStream class has methods for all the primitive types, as well as a method for reference types, for writing these values to the stream. Listing 15-1 shows the use of the ObjectOutputStream.

Listing 15-1: Writing Java types to a file

public class Pair implements Serializable {
    private final int number;
    private final String name;

    public Pair(int number, String name) {
        this.number = number;
        this.name = name;
    }

    public int getNumber() {
        return number;
    }

    public String getName() {
        return name;
    }
}

@Test
public void writeData() throws IOException {
    final FileOutputStream fos = new FileOutputStream("/tmp/file");
    final ObjectOutputStream oos = new ObjectOutputStream(fos);

    oos.writeInt(101);
    oos.writeBoolean(false);
    oos.writeUTF("Writing a string");

    final Pair pair = new Pair(42, "Forty two");
    oos.writeObject(pair);

    oos.flush();
    oos.close();
    fos.close();
}

If you take a look at the data written to the file /tmp/file, you will see that it is binary data. This has been written using a form that the JVM understands and is able to read back, too, as Listing 15-2 shows.

Listing 15-2: Reading from an ObjectInputStream

@Test
public void readData() throws IOException, ClassNotFoundException {
    final FileInputStream fis = new FileInputStream("/tmp/file");
    final ObjectInputStream ois = new ObjectInputStream(fis);

    final int number = ois.readInt();
    final boolean bool = ois.readBoolean();
    final String string = ois.readUTF();
    final Pair pair = (Pair) ois.readObject();

    assertEquals(101, number);
    assertFalse(bool);
    assertEquals("Writing a string", string);
    assertEquals(42, pair.getNumber());
    assertEquals("Forty two", pair.getName());
}

It is important to read the data in the same order in which you wrote it! If you, for instance, try to call readObject when the next type in the stream is an int, you will get an exception.

There are two important things of note for the small Pair class created for this example. Notice that the readData test can possibly throw a ClassNotFoundException. If you are reading this file in a different JVM than the one in which it was written, and the Pair class is not on the classpath, the JVM will not be able to construct the object.

Also, the Pair class must implement the Serializable interface. This is a marker interface, which signals to the JVM that this class can be serialized.

If you have a Serializable object, but have fields that you do not want to be serialized when writing the data to a stream, you can apply the transient modifier to the field declaration. When a transient field is deserialized, that field will be null. Listing 15-3 shows an example.

Listing 15-3: Using transient fields

public class User implements Serializable {
    private String username;
    private transient String password;

    public User(String username, String password) {
        this.username = username;
        this.password = password;
    }

    public String getUsername() {
        return username;
    }

    public String getPassword() {
        return password;
    }
}

@Test
public void transientField() 
        throws IOException, ClassNotFoundException {
    final User user = new User("Noel", "secret321");

    final FileOutputStream fos = new FileOutputStream("/tmp/user");
    final ObjectOutputStream oos = new ObjectOutputStream(fos);

    oos.writeObject(user);

    oos.flush();
    oos.close();
    fos.close();

    final FileInputStream fis = new FileInputStream("/tmp/user");
    final ObjectInputStream ois = new ObjectInputStream(fis);

    final User deserialized = (User) ois.readObject();

    assertEquals("Noel", deserialized.getUsername());
    assertNull(deserialized.getPassword());
}

The main use cases for creating transient fields are when you are holding a private field as a cache, so you can simply regenerate the cache after deserializing, or when data can be reconstructed after it has been deserialized, such as a field created from other fields:

private String firstName = "Bruce";
private String lastName = "Springsteen";
private String fullName = String.format("%s %s", firstName, lastName);

As Listing 15-3 shows, you can hold secure or sensitive data in a transient field, and be confident that it will not be written to an ObjectOutputStream.

Using XML

Rather than delegating the representation of objects to the JVM and the mechanics of ObjectInputStream and ObjectOutputStream, you could define the format yourself.

One huge advantage of doing this is that once the object has been serialized, it can be read by any process that understands your format, whether that is another JVM or perhaps a different setup altogether.

A popular notation for defining domain objects is XML. It allows for a well-structured representation of data, and most languages have well-established libraries for parsing and writing XML.

A meta-language, called XML Schema Definition (XSD), enables you to define a domain object using XML markup, which is used to create XML documents. Java has a library called JAXB, which understands XSD notation and can create Java objects from these documents. This library is often used within other libraries to serialize and deserialize objects, for use in SOAP applications, or similar.

Listing 15-4 shows an XSD for representing players on a sports team.

Listing 15-4: A sample XML Schema document

<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="Team">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="Name" type="xs:string"/>
        <xs:element name="DateFounded" type="xs:date"/>
          <xs:element name="Players">
            <xs:complexType>
              <xs:sequence minOccurs="2" maxOccurs="11">
                <xs:element name="Player">
                  <xs:complexType>
                    <xs:sequence>
                      <xs:element name="Name" type="xs:string"/>
                      <xs:element name="Position" type="xs:string"/>
                    </xs:sequence>
                  </xs:complexType>
                </xs:element>
              </xs:sequence>
            </xs:complexType>
          </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

This schema allows the creation of Team types, and among the attributes of that type, it has a collection of Player types. This allows for the creation of sample XML documents such as the one in Listing 15-5.

Listing 15-5: A Team XML document

<?xml version="1.0" encoding="utf-8"?>
<Team xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="team.xsd">
    <Name>AllStars</Name>
    <DateFounded>2001-09-15</DateFounded>
    <Players>
        <Player>
            <Name>Adam Smith</Name>
            <Position>Goal</Position>
        </Player>
        <Player>
            <Name>Barry Jenkins</Name>
            <Position>Defence</Position>
        </Player>
    </Players>
</Team>

By including the schema location in the Team definition, any software you use that understands XSD can validate this document. This includes parsers such as JAXB, which you will see next, or even IDEs with smart XML and XSD support. These can provide smart auto-completion to help you generate the document.

JAXB is run through a separate compiler, which parses XSD content and creates Java source files. The xjc command is distributed as part of the JDK. If you run

xjc -p com.wiley.acinginterview.chapter15.generated /path/to/team.xsd

you should see the following output:

parsing a schema...
compiling a schema...
com/wiley/acinginterview/chapter15/generated/ObjectFactory.java
com/wiley/acinginterview/chapter15/generated/Team.java

If you inspect the Team.java file, you can see that it is a regular Java class. This class is a representation of the team.xsd created in Listing 15-5, and can be used as any Java object, such as that shown in Listing 15-6.

Listing 15-6: Using JAXB-generated Java objects

@Test
public void useJaxbObject() {
    final Team team = new Team();
    team.setName("Superstars");

    final Team.Players players = new Team.Players();

    final Team.Players.Player p1 = new Team.Players.Player();
    p1.setName("Lucy Jones");
    p1.setPosition("Striker");

    final Team.Players.Player p2 = new Team.Players.Player();
    p2.setName("Becky Simpson");
    p2.setPosition("Midfield");
    players.getPlayer().add(p1);
    players.getPlayer().add(p2);

    team.setPlayers(players);

    final String position = team
            .getPlayers()
            .getPlayer()
            .get(0)
            .getPosition();
    
    assertEquals("Striker", position);
}

The point of interest with this generated object is that the rich Player and Players objects are static inner classes of the Team object, because these were created as nested complex types in the original XSD. These could have been created as top-level objects if you wanted to use them elsewhere, as shown in Listing 15-7.

Listing 15-7: Top-level XSD types

<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <xs:element name="Player">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="Name" type="xs:string"/>
                <xs:element name="Position" type="xs:string"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>

    <xs:element name="Players">
        <xs:complexType>
            <xs:sequence minOccurs="2" maxOccurs="11">
                <xs:element ref="Player"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>
...
</xs:schema>

You can see here that generated types are included with the ref attribute, rather than type.

This then creates Java objects, which can be used as shown in Listing 15-8.

Listing 15-8: Using top-level XSD types

@Test
public void useTopLevelJaxbObjects() {
    final Team team = new Team();
    team.setName("Superstars");

    final Players players = new Players();

    final Player p1 = new Player();
    p1.setName("Lucy Jones");
    p1.setPosition("Striker");

    final Player p2 = new Player();
    p2.setName("Becky Simpson");
    p2.setPosition("Midfield");
    players.getPlayer().add(p1);
    players.getPlayer().add(p2);

    team.setPlayers(players);

    final String position = team
            .getPlayers()
            .getPlayer()
            .get(0)
            .getPosition();
    
    assertEquals("Striker", position);
}

Clearly, this is much easier to read and work with.

One point that often surprises new users is that you do not need to create, or set, a collection type, such as the Players in this example. Rather, you call get to get the collection instance, and then you can add or remove from that collection.

Once the JAXB bindings have been created using xjc, parsing an XML document that conforms to an XSD is relatively simple, as shown in Listing 15-9.

Listing 15-9: Reading XML into a Java object

@Test
public void readXml() throws JAXBException {
        final JAXBContext context = 
                JAXBContext.newInstance(
                        "com.wiley.javainterviewsexposed.chapter15.generated2");
    final Unmarshaller unmarshaller = context.createUnmarshaller();
        final Team team = (Team) unmarshaller
                .unmarshal(getClass()
                        .getResourceAsStream("/src/main/xsd/teamSample.xml"));

    assertEquals(2, team.getPlayers().getPlayer().size());
}

Although this XML document was provided as a resource on the classpath, it could easily have come from any source, such as a messaging queue or an HTTP request.

Similarly, writing Java objects as XML is also simple, as Listing 15-10 demonstrates.

Listing 15-10: Using JAXB to serialize a Java object

@Test
public void writeXml() throws JAXBException, FileNotFoundException {
    final ObjectFactory factory = new ObjectFactory();
    final Team team = factory.createTeam();
    final Players players = factory.createPlayers();
    final Player player = factory.createPlayer();
    player.setName("Simon Smith");
    player.setPosition("Substitute");
    players.getPlayer().add(player);

    team.setName("Megastars");
    team.setPlayers(players);

        final JAXBContext context = 
                JAXBContext.newInstance(
                        "com.wiley.javainterviewsexposed.chapter15.generated2");
    final Marshaller marshaller = context.createMarshaller();
    marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
    marshaller.marshal(team, new FileOutputStream("/tmp/team.xml"));
}

Listing 15-10 produces the following XML:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Team>
    <Name>Megastars</Name>
    <Players>
        <Player>
            <Name>Simon Smith</Name>
            <Position>Substitute</Position>
        </Player>
    </Players>
</Team>

It is important that the Marshaller was created from a specific JAXBContext, which was configured to the package where xjc created the Java objects. If you use a different package for your automatically generated objects, the Marshaller will not be able to parse your Java objects into XML.

It is possible to have your Java sources compiled as part of your build. Listing 15-11 shows the plugin tag for the Maven POM for the examples used in this section.

Listing 15-11: Automatically generating JAXB objects

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>jaxb2-maven-plugin</artifactId>
    <version>1.5</version>
    <executions>
        <execution>
            <id>xjc</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>xjc</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
    <packageName>
        com.wiley.javainterviewsexposed.chapter15.generated3
    </packageName>
    </configuration>
</plugin>

This assumes that your XSD files are contained in src/main/xsd. This plug-in will fire with the generate-sources phase of the build, meaning that any XSD objects will be re-created before the main source of the project is compiled.

Having your domain objects generated by the build is an extremely useful feature, because any breaking changes to the model will also break your build, and the tests will not likely work, either. Fixing your code following a breaking change gives you confidence that your serialization and deserialization routines are working properly.

JSON

JavaScript Object Notation, or JSON, is another serialization approach, similar to using XML with XSD. It uses a human-readable approach, which can be parsed and processed by a number of languages. Indeed, JSON originated as part of JavaScript, and has been adopted by libraries for other languages as a lightweight, less verbose approach over XML.

Listing 15-12 illustrates the team model in JSON, previously used in earlier examples in this chapter.

Listing 15-12: A JSON example

{
    "name": "AllStars",
    "dateFounded": "2001-09-15",
    "players": [
        {
            "name": "Adam Smith",
            "position": "Goal"
        },
        {
            "name": "Barry Jenkins",
            "position": "Defence"
        }
    ]
}

For JSON, there is a meta-language called JSON Schema, but it is not widely used. A JSON document is loosely defined, but is still well structured. There are very few types: A string is text surrounded by quotes, there is no differentiation between floating point and integral numbers, and you can use the boolean values true and false. A list is a comma-separated list, surrounded by square brackets, like the list of players in Listing 15-12. An object is a collection of key-value pairs surrounded by curly braces. The keys are always strings; the value is any type. Any type can be null.

An implementation of JSON parsing and processing is contained in a library called Jackson. It upholds JSON’s simplicity by being straightforward to use. Listing 15-13 contains sample domain objects for the Team and Players, and Listing 15-14 illustrates how to parse the JSON into these objects.

Listing 15-13: Sample domain objects for Team and Players

public class Team {

    private String name;
    private String dateFounded;
    private List<Player> players;

... // include getters and setters
}

public class Player {

    private String name;
    private String position;
... // include getters and setters
}

Listing 15-14: Parsing JSON into rich domain objects

@Test
public void readJson() throws IOException {
    final ObjectMapper mapper = new ObjectMapper();
    final String json =
            "/com/wiley/javainterviewsexposed/chapter15/team.json";
    final Team team = mapper.readValue(
            getClass().getResourceAsStream(json),
            Team.class);

    assertEquals(2, team.getPlayers().size());
}

This code makes heavy use of reflection to make sure the JSON data will fit into the given domain object, so make sure you are thorough with your testing if you use this approach.

The Jackson library can also be used to create JSON from a domain object, which works exactly as you would expect, as shown in Listing 15-15.

Listing 15-15: Writing domain objects to JSON

@Test
public void writeJson() throws IOException {
    final Player p1 = new Player();
    p1.setName("Louise Mills");
    p1.setPosition("Coach");

    final Player p2 = new Player();
    p2.setName("Liam Turner");
    p2.setPosition("Attack");

    final Team team = new Team();
    team.setPlayers(Arrays.asList(p1, p2));

    final ObjectMapper mapper = new ObjectMapper();
    mapper.writeValue(new File("/tmp/newteam"), team);

}

This code produces the following JSON:

{
    "name": null,
    "dateFounded": null,
    "players": [
        {
            "name": "Louise Mills",
            "position": "Coach"
        },
        {
            "name": "Liam Turner",
            "position": "Attack"
        }
    ]
}

Note that name and dateFounded are null, because these were not set on the Team object.

Summary

Creating Java applications and servers that can communicate with other services and devices is often one of the main talking points in interviews. Being able to create stable and robust services is extremely important; however, defining how the services talk to each other can have a major influence on application performance, as well as architectural choices made early in a project.

Many development teams, particularly ones that work heavily in cross-platform communication, such as between a mobile device and server, are turning to JSON for their communications over the wire. It is much less verbose than XML, which is useful when taking volumes of data into account. But it is also less strict because there is no document definition as there is with XSD, so be aware of this trade-off: You will need to be sure that your document is correct before parsing or sending to a recipient.

Several serialization technologies not covered here communicate using a binary protocol rather than the plaintext approaches of XML and JSON. However, unlike Java’s own serialization techniques, the binary protocols such as Google’s Protocol Buffers or Apache Thrift are designed to be cross-platform, and work over a multitude of languages and devices.

The next chapter covers the Spring Framework, including a section on creating web services using Spring MVC. The techniques used in this chapter can be applied to Spring MVC in defining exactly what is transmitted when querying or receiving responses from a web server.