The InputStream class has an abstract method:

public abstract int read() throws IOException

This method reads one byte and returns the byte read, or –1 if it encounters the end of the input source. The designer of a concrete input stream class overrides this method in order to provide useful functionality. For example, in the FileInputStream class, this method reads one byte from a file. The InputStream class also has non-abstract methods to read an array of bytes or to skip a number of bytes. These methods call the abstract read method, so that subclasses only need to override one method.

Similarly, the OutputStream class defines the abstract method

public abstract void write(int b) throws IOException

which writes one byte to an output file.

Both the read and write methods can block a thread until the byte is actually read or written. This means if the byte cannot immediately be read from or written to (usually because of a busy network connection), Java suspends the thread containing this call. This gives other threads the chance to do useful work while the method is waiting for the stream to again become available. (We discuss threads in Chapter 2.)

The available method lets you check the number of bytes that are currently available for reading. This means a fragment like the following is unlikely to ever block:

int bytesAvailable = System.in.available(); 
if (bytesAvailable > 0) 
{  byte [] data = new byte [bytesAvailable]; 
   System.in.read(data); 
}

When you have finished reading or writing to a stream, close it, using the appropriately named close method, because streams use operating system resources that are in limited supply. If an application opens many streams without closing them, system resources may become depleted. Closing an output stream also flushes the buffer used for the output stream: any characters that were temporarily placed in a buffer so that they could be delivered as a larger packet are sent off. In particular, if you do not close a file, the last packet of bytes may never be delivered. You can also manually flush the output with the flush method.

Even if a stream class provides concrete methods to work with the raw read and write functions, Java programmers seldom use them. This is because you rarely need to read and write streams of bytes. The data that you are interested in probably contain numbers, strings, and objects.

Java gives you many stream classes derived from the basic InputStream and OutputStream classes that let you work with data in the forms that you usually use rather than at the low, byte level.

java.io.InputStream

• abstract int read()

  reads a byte of data and returns the byte read. The read method returns a –1 at the end of the stream.

• int read(byte b[])

  reads into an array of bytes and returns the number of bytes read. As before, the read method returns a –1 at the end of the stream.

• int read(byte b[], int off, int len)

  reads into an array of bytes. The read method returns the actual number of bytes read, or –1 at the end of the stream.

  Parameters:	b	the array into which the data is read
  	off	the offset into b where the first bytes should be placed
  	len	the maximum number of bytes to read

• long skip(long n)

  skips n bytes in the input stream. It returns the actual number of bytes skipped (which may be less than n if the end of the stream was encountered).

• int available()

  returns the number of bytes available without blocking. (Recall that blocking means that the current thread loses its turn.)

• void close()

  closes the input stream.

• void mark(int readlimit)

  The mark method puts a marker at the current position in the input stream. (Not all streams support this feature.) If more than readlimit bytes have been read from the input stream, then the stream is allowed to forget the marker.

• void reset()

  returns to the last marker. Subsequent calls to read reread the bytes.

• boolean markSupported()

  returns true if the stream supports marking.

java.io.OutputStream

• public abstract void write(int b)

  writes a byte of data.

• public void write(byte b[])

  writes all bytes in the array b.

• public void write(byte b[], int off, int len)

  Parameters:	b	the array from which to write the data
  	off	the offset into b to the first byte that will be written
  	len	the number of bytes to write

• public void close()

  closes the output stream.

• public void flush()

  flushes the output stream, that is, sends any buffered data to its destination.

FileInputStream and FileOutputStream give you input and output streams attached to a disk file. You give the name or full pathname of the file in the constructor. For example,

FileInputStream fin = new FileInputStream("employee.dat");

looks in the current directory for a file named "employee.dat". You can also use a File object:

File f = new File("employee.dat") 
FileInputStream fin = new FileInputStream(f);

Like the abstract InputStream and OutputStream classes, these classes only support reading and writing on the byte level. That is, we can only read bytes and byte arrays from the object in.

byte b = fin.read();

As we will see in the next section, if we just had a DataInputStream, then we could read numeric types:

DataInputStream din = . . .; 
double s = din.readDouble();

But just as the FileInputStream has no methods to read numeric types, the DataInputStream has no method to get data from a file.

Java uses a clever mechanism to separate two kinds of responsibilities. Some streams (such as the FileInputStream and the input stream returned by the openStream method of the URL class) can retrieve bytes from files and other more exotic locations. Other streams (such as the DataInputStream and the PrintWriter ) can assemble bytes into more useful data types. The Java programmer has to combine the two into what are often called filtered streams by feeding an existing stream to the constructor of another stream. For example, to be able to read numbers from a file, first create a FileInputStream and then pass it to the constructor of a DataInputStream.

FileInputStream fin = new FileInputStream("employee.dat"); 
DataInputStream din = new DataInputStream(fin); 
double s = din.readDouble();

The data input stream does not correspond to a new disk file. It accesses the data from the file attached to the file input stream, but it has a more capable interface.

If you look at Figure 1-1 again, you can see the classes FilterInputStream and FilterOutputStream. You combine their child classes into a new filtered stream to construct the streams you want. For example, by default, streams are not buffered. That is, every call to read contacts the operating system to ask it to dole out yet another byte. If you want buffering and data input, you need to use the following rather monstrous sequence of constructors:

DataInputStream din = new DataInputStream 
   (new BufferedInputStream 
      (new FileInputStream("employee.dat")));

Notice that we put the DataInputStream last in the chain of constructors because we want to use the DataInputStream methods, and we want them to use the buffered read method.

Sometimes you need to keep track of the intermediate streams when chaining them together. For example, when reading input, you often need to peek at the next byte to see if it is the value that you expect. Java provides the PushbackInputStream for this purpose.

PushbackInputStream pbin = new PushbackInputStream 
   (new BufferedInputStream 
      (new FileInputStream("employee.dat")));

Now you can speculatively read the next byte

int b = pbin.read();

and throw it back if it wasn’t what you wanted.

if (b != '<') pbin.unread(b);

But reading and unreading are the only methods that apply to the pushback input stream. If you want to look ahead and also read numbers, then you need both a pushback input stream and a data input stream reference.

DataInputStream din = new DataInputStream 
   (pbin = new PushbackInputStream 
      (new BufferedInputStream 
      (new FileInputStream("employee.dat"))));

Of course, in the stream libraries of other programming languages, niceties such as buffering and lookahead are automatically taken care of, so it is a bit of a hassle in Java that one has to resort to stream filters in these cases. But you can also mix and match filter classes to construct truly useful sequences of streams. For example, you can read numbers from a compressed ZIP file by using the following sequence of streams (see Figure 1-3).

Figure 1-3. A sequence of filtered streams

ZipInputStream zin = new ZipInputStream(new 
FileInputStream("employee.zip")); 
DataInputStream din = new DataInputStream(zin);

(See the section on “ZIP file streams” later on in this chapter for more on Java’s ability to handle ZIP files.)

All in all, apart from the rather monstrous constructors that are needed to layer streams, the ability to mix and match streams is a very useful feature of Java!

java.io.FileInputStream

• FileInputStream(String name)

  creates a new file input stream using the file whose pathname is specified by the name string.

• FileInputStream(File f)

  creates a new file input stream using the information encapsulated in the File object. (The File class is described at the end of this chapter.)

java.io.BufferedInputStream

• BufferedInputStream(InputStream in)

  creates a new buffered stream with a default buffer size. A buffered input stream reads characters from a stream without causing a device access every time. When the buffer is empty, a new block of data is read into the buffer.

• BufferedInputStream(InputStream in, int n)

  creates a new buffered stream with a user-defined buffer size.

java.io.BufferedOutputStream

• BufferedOutputStream(OutputStream out)

  creates a new buffered stream with a default buffer size. A buffered output stream collects characters to be written without causing a device access every time. When the buffer fills up, the data is written.

• BufferedOutputStream(OutputStream out, int n)

  creates a new buffered stream with a user-defined buffer size.

java.io.PushbackInputStream

• PushbackInputStream(InputStream in)

  constructs a stream with one-character lookahead.

• PushbackInputStream(InputStream in, int size)

  constructs a stream with a push-back buffer of specified size.

• void unread(int ch)

  pushes back a character, which is retrieved again by the next call to read. You can push back only one character at a time.

Parameters:

ch

the character to be read again

You often need to write the result of a computation or read one back. The data streams support methods for reading back all of the basic Java types. To write a number, character, Boolean value, or string, use one of the following methods:

For example, writeInt writes an integer as a 4-byte binary quantity, and writeDouble writes a double as an 8-byte binary quantity. The resulting output is not humanly readable—see the section on the PrintWriter class later in this chapter for text output of numbers.

NOTE

There are two different methods of storing integers and floating-point numbers in memory, depending on the platform you are using. Suppose, for example, you are working with a 4-byte quantity, like an int or a float. This can be stored in such a way that the first of the 4 bytes in memory holds the most significant byte (MSB) of the value, the so-called big-endian method, or it can hold the least significant byte (LSB) first, which is called, naturally enough, the little-endian method. For example, the SPARC uses big-endian; the Pentium, little-endian. This can lead to problems. For example, when saving a file using C or C++, the data is saved exactly as the processor stores it. That makes it challenging to move even the simplest data files from one platform to another. In Java, all values are written in the big-endian fashion, regardless of the processor. That makes Java data files platform independent.

The writeUTF method writes string data using Unicode Text Format (UTF). UTF format is as follows. A 7-bit ASCII value (that is, a 16-bit Unicode character with the top 9 bits zero) is written as one byte:

0a6a5a4a3a2a1a0

A 16-bit Unicode character with the top 5 bits zero is written as a 2-byte sequence:

110a10a9a8a7a6   10a5a4a3a2a1a0

(The writeUTF method actually writes only the 11 lowest bits.)

All other Unicode characters are written as 3-byte sequences:

1110a15a14a13a12   10a11a10a9a8a7a6   10a5a4a3a2a1a0

This is a useful format for text consisting mostly of ASCII characters, because ASCII characters still take only a single byte. On the other hand, it is not a good format for Asiatic languages, for which you are better off directly writing sequences of double-byte Unicode characters. Use the writeChars method for that purpose.

Note that the top bits of a UTF byte determine the nature of the byte in the encoding scheme.

To read the data back in, use the following methods:

NOTE

The binary data format is compact and platform independent. Except for the UTF strings, it is also suited to random access. The major drawback is that binary files are not readable by humans.

java.io.DataInput

• boolean readBoolean()

  reads in a Boolean value.

• byte readByte()

  reads an 8-bit byte.

• char readChar()

  reads a 16-bit Unicode character.

• double readDouble()

  reads a 64-bit double.

• float readFloat()

  reads a 32-bit float.

• void readFully(byte b[])

  reads bytes, blocks until all bytes are read.

  Parameters:

  b

  the buffer into which the data is read

• void readFully(byte b[], int off, int len)

  reads bytes, blocking until all bytes are read.

  Parameters:	b	the buffer into which the data is read
  	off	the start offset of the data
  	len	the maximum number of bytes read

• int readInt()

  reads a 32-bit integer.

• String readLine()

  reads in a line that has been terminated by a , , , or EOF. Returns a string containing all bytes in the line converted to Unicode characters.

• long readLong()

  reads a 64-bit long integer.

• short readShort()

  reads a 16-bit short integer.

• String readUTF()

  reads a string of characters in UTF format.

• int skipBytes(int n)

  skips bytes, blocks until all bytes are skipped.

Parameters:

n

the number of bytes to be skipped

java.io.DataOutput

• void writeBoolean(boolean b)

  writes a Boolean value.

• void writeByte(byte b)

  writes an 8-bit byte.

• void writeChar(char c)

  writes a 16-bit Unicode character.

• void writeChars(string s)

  writes a string as a sequence of characters.

• void writeDouble(double d)

  writes a 64-bit double.

• void writeFloat(float f)

  writes a 32-bit float.

• void writeInt(int i)

  writes a 32-bit integer.

• void writeLong(long l)

  writes a 64-bit long integer.

• void writeShort(short s)

  writes a 16-bit short integer.

• void writeUTF(String s)

  writes a string of characters in UTF format.

The RandomAccessFile stream class lets you find or write data anywhere in a file. Disk files are random access, but streams of data from a network are not. You open a random-access file either for reading only or for both reading and writing. You specify the option by using the string "r" (for read access) or "rw" (for read/write access) as the second argument in the constructor.

RandomAccessFile in = new RandomAccessFile("employee.dat", "r"); 
RandomAccessFile inOut 
   = new RandomAccessFile("employee.dat", "rw");

A random-access file also has a file pointer setting that comes with it. The file pointer always indicates the position of the next record that will be read or written. The seek method sets the file pointer to an arbitrary byte position within the file. The argument to seek is a long integer between zero and the length of the file in bytes.

The getFilePointer method returns the current position of the file pointer.

To read from a random-access file, you use the same methods—such as readInt and readUTF —as for DataInputStream objects. That is no accident. These methods are actually defined in the DataInput interface that both DataInputStream and RandomAccessFile implement.

Similarly, to write a random-access file, you use the same writeInt and writeUTF methods as in the DataOutputStream class. These methods are defined in the DataOutput interface that is common to both classes.

The advantage of this setup is that you can write methods whose argument types are the DataInput and DataOutput interfaces.

class Employee 
{  . . . 
   read(DataInput in) { . . . } 
   write(DataOutput out) { . . . } 
}

Note that the read method can handle either a DataInputStream or a RandomAccessFile object because both of these classes implement the DataInput interface. The same is true for the write method.

java.io.RandomAccessFile

• RandomAccessFile(String name, String mode)

  Parameters:	name	system-dependent file name
  	mode	"r" for reading only, or "rw" for reading and writing

• RandomAccessFile(File file, String mode)

  Parameters:	file	a File object encapsulating a system-dependent file name. (The File class is described at the end of this chapter.)
  	mode	"r" for reading only, or "rw" for reading and writing

• long getFilePointer()

  returns the current location of the file pointer.

• void seek(long pos)

  sets the file pointer to pos bytes from the beginning of the file.

• public long length()

  returns the length of the file in bytes.

InputStreamReader in = new InputStreamReader(System.in);

InputStreamReader(InputStream, String)

InputStreamReader in = new InputStreamReader(new 
   FileInputStream("kremlin.dat"), "8859_5");

FileWriter out = new FileWriter("output.txt");

OutputStreamWriter out = new OutputStreamReader(new 
   FileOutputStream("output.txt"));

Writing text output

For text output, you want to use a PrintWriter. A print writer can print strings and numbers in text format. Just as a DataOutputStream has useful output methods but no destination, a PrintWriter must be combined with a destination writer.

PrintWriter out = new PrintWriter(new FileWriter("employee.txt"));

You can also combine a print writer with a destination (output) stream.

PrintWriter out = new PrintWriter(new 
   FileOutputStream("employee.txt"));

The PrintWriter(OutputStream) constructor automatically adds an OutputStreamWriter to convert Unicode characters to bytes in the stream.

To write to a print writer, you use the same print and println methods that you used with System.out. You can use these methods to print numbers (int, short, long, float, double ), characters, Boolean values, strings, and objects.

NOTE

Java veterans probably wonder whatever happened to the PrintStream class and to System.out. In Java1.0, the PrintStream class simply truncated all Unicode characters to ASCII characters by dropping the top byte. Conversely, the readLine method of the DataInputStream turned ASCII to Unicode by setting the top byte to 0. Clearly, that was not a clean or portable approach, and it was fixed with the introduction of readers and writers in Java 1.1. For compatibility with existing code, System.in, System.out, and System.err are still streams, not readers and writers. But now the PrintStream class internally converts Unicode characters to the default host encoding in the same way as the PrintWriter. And all constructors for PrintStream are now deprecated—simply use PrintWriter instead. That means that new Java code has exactly two objects of type PrintStream, namely, System.out and System.err. These act exactly like print writers when you use the print and println methods, but unlike print writers, you can also send raw bytes to them with the write(int) and write(byte[]) methods.

Table 1-1. Character encodings

8859_1	ISO Latin-1
8859_2	ISO Latin-2
8859_3	ISO Latin-3
8859_5	ISO Latin/Cyrillic
8859_6	ISO Latin/Arabic
8859_7	ISO Latin/Greek
8859_8	ISO Latin/Hebrew
8859_9	ISO Latin-5
Cp1250	Windows Eastern Europe / Latin-2
Cp1251	Windows Cyrillic
Cp1252	Windows Western Europe / Latin-1
Cp1253	Windows Greek
Cp1254	Windows Turkish
Cp1255	Windows Hebrew
Cp1256	Windows Arabic
Cp1257	Windows Baltic
Cp1258	Windows Vietnamese
Cp437 PC	Original
Cp737	PC Greek
Cp775	PC Baltic
Cp850	PC Latin-1
Cp852	PC Latin-2
Cp855	PC Cyrillic
Cp857	PC Turkish
Cp860	PC Portuguese
Cp861	PC Icelandic
Cp862	PC Hebrew
Cp863	PC Canadian French
Cp864	PC Arabic
Cp865	PC Nordic
Cp866	PC Russian
Cp869	PC Modern Greek
Cp874	Windows Thai
EUCJIS	Japanese EUC
JIS	JIS
MacArabic	Macintosh Arabic
MacCentralEurope	Macintosh Latin-2
MacCroatian	Macintosh Croatian
MacCyrillic	Macintosh Cyrillic
MacDingbat	Macintosh Dingbat
MacGreek	Macintosh Greek
MacHebrew	Macintosh Hebrew
MacIceland	Macintosh Icelandic
MacRoman	Macintosh Roman
MacRomania	Macintosh Romania
MacSymbol	Macintosh Symbol
MacThai	Macintosh Thai
MacTurkish	Macintosh Turkish
MacUkraine	Macintosh Ukraine
SJIS	PC and Windows Japanese
UTF8	Standard UTF-8

For example, consider this code:

String name = "Harry Hacker"; 
double salary = 75000; 
out.print(name); 
out.print(' '), 
out.println(salary);

This writes the characters

Harry Hacker 75000

to the stream out. The characters are then converted to bytes and end up in the file employee.txt.

As you know, the println method always prints a line terminator. This is the string obtained by the call System.getProperty("line.separator"), such as " " (Unix), " " (DOS) or " " (Macintosh). If the writer is set to auto flush mode, then all characters in the buffer are sent to their destination whenever println is called. (Print writers are always buffered.) By default, auto flushing is not enabled. You can enable or disable auto flushing by using the PrintWriter(Writer, boolean) constructor and passing the appropriate Boolean as the second argument.

PrintWriter out = new PrintWriter(new 
   FileWriter("employee.txt"), true); // auto flush

The print methods don’t throw exceptions. You can call the checkError method to see if something went wrong with the stream.

NOTE

You cannot write raw bytes to a PrintWriter. Print writers are designed for text output only.

java.io.PrintStream

• void print(Object obj)

  prints an object by printing the string resulting from toString.

  Parameters:

  obj

  the object to be printed

• void print(String s)

  prints a Unicode string.

• void println(String s)

  prints a string followed by a line terminator. Flushes the stream if the stream is in autoflush mode.

• void print(char s[])

  prints an array of Unicode characters.

• void print(char c)

  prints a Unicode character.

• void print(int i)

  prints an integer in text format.

• void print(long l)

  prints a long integer in text format.

• void print(float f)

  prints a floating-point number in text format.

• void print(double d)

  prints a double-precision floating-point number in text format.

• void print(boolean b)

  prints a Boolean value in text format.

• boolean checkError()

  returns true if a formatting or output error occurred. Once the stream has encountered an error, it is tainted and all calls to checkError return true.

java.io.PrintWriter

• PrintWriter(Writer out)

  Creates a new PrintWriter, without automatic line flushing.

  Parameters:

  out

  a character-output writer

• PrintWriter(Writer out, boolean autoFlush)

  Creates a new PrintWriter.

  Parameters:	out	a character-output writer
  	autoFlush	if true, the println() methods will flush the output buffer

• PrintWriter(OutputStream out)

  Creates a new PrintWriter, without automatic line flushing, from an existing OutputStream by automatically creating the necessary intermediate OutputStreamWriter.

  Parameters:

  out

  an output stream

• PrintWriter(OutputStream out, boolean autoFlush)

  Also creates a new PrintWriter from an existing OutputStream but allows you determine whether the writer autoflushes or not.

  Parameters:	out	an output stream
  	autoFlush	if true, the println() methods will flush the output buffer

BufferedReader in = new BufferedReader(new 
   FileReader("employee.txt"));

String s; 
while ((s = in.readLine()) != null) 
{do something with s; 
}

BufferedReader in2 = new BufferedReader(new 
   InputStreamReader(System.in)); 
BufferedReader in3 = new BufferedReader(new 
   InputStreamReader(url.openStream()));

String s = in.readLine(); 
double x = new Double(s).doubleValue();

In this section, you will learn how to store an array of Employee records in the time-honored delimited format. This means that each record is stored in a separate line. Instance fields are separated from each other by delimiters. We use a vertical bar (| ) as our delimiter. (Acolon (: ) is another popular choice. Part of the fun is that everyone uses a different delimiter.) Naturally, we punt on the issue of what might happen if a | actually occurred in one of the strings we save.

NOTE

Especially on Unix systems, an amazing number of files are stored in exactly this format. We have seen entire employee databases with thousands of records in this format, queried with nothing more than the Unix awk, sort, and join utilities. (In the PC world, where excellent database programs are available at low cost, this kind of ad hoc storage is much less common.)

Here is a sample set of records:

Harry Hacker|35500|1989|10|1 
Carl Cracker|75000|1987|12|15 
Tony Tester|38000|1990|3|15

Writing records is simple. Since we write to a text file, we use the PrintWriter class. We simply write all fields, followed by either a | or, for the last field, a . Finally, in keeping with the idea that we want the class to be responsible for responding to messages, we add a method, writeData, to our Employee class.

public void writeData(PrintWriter os) throws IOException 
{  Format.print(os, "%s|", name); 
   Format.print(os, "%.14g|", salary); 
   Format.print(os, "%d|", hireDay.getYear()); 
   Format.print(os, "%d|", hireDay.getMonth()); 
   Format.print(os, "%d
", hireDay.getDay()); 
}

To read records, we read in a line at a time and separate the fields. This is the topic of the next section, in which we use a utility class supplied with Java to make our job easier.

When reading a line of input, we get a single long string. We want to split it into individual strings. This means finding the | delimiters and then separating out the individual pieces, that is, the sequence of characters up to the next delimiter. (These are usually called tokens.) The StringTokenizer class in java.util is designed for exactly this purpose. It gives you an easy way to break up a large string that contains delimited text. The idea is that a string tokenizer object attaches to a string. When you construct the tokenizer object, you specify which characters are the delimiters. For example, we need to use

StringTokenizer t = new StringTokenizer(line, "|");

You can specify multiple delimiters in the string. For example, to set up a string tokenizer that would let you search for any delimiter in the set

" 	

"

use the following:

StringTokenizer t = new StringTokenizer(line, " 	

");

(Notice that this means that any white space marks off the tokens.)

NOTE

These four delimiters are used as the defaults if you construct a string tokenizer like this:

StringTokenizer t = new StringTokenizer(line);

Once you have constructed a string tokenizer, you can use its methods to quickly extract the tokens from the string. The nextToken method returns the next unread token. The hasMoreTokens method returns true if more tokens are available.

NOTE

In our case, we know how many tokens we have in every line of input. In general, you have to be a bit more careful: call hasMoreTokens before calling nextToken because the nextToken method throws an exception when no more tokens are available.

java.util.StringTokenizer

• StringTokenizer(String str, String delim)

  Parameters:	str	the input string from which tokens are read
  	delim	a string containing delimiter characters (any character in this string is a delimiter)

• StringTokenizer(String str)

  constructs a string tokenizer with the default delimiter set " ".

• boolean hasMoreTokens()

  returns true if more tokens exist.

• String nextToken()

  returns the next token; throws a NoSuchElementException if there are no more tokens.

• String nextToken(String delim)

  returns the next token, after switching to the new delimiter set. The new delimiter set is subsequently used.

• int countTokens()

  returns the number of tokens still in the string.

Reading in an Employee record is simple. We simply read in a line of input with the readLine method of the BufferedReader class. Here is the code needed to read one record in a string.

BufferedReader in 
   = new BufferedReader(new FileReader("employee.dat")); 
. . . 
String line = in.readLine();

Next, we need to extract the individual tokens. When we do this, we end up with strings, so we need to convert them into numbers when appropriate. To do this, we turn to the atoi and atof methods from the Format class in our corejava package.

Just as with the writeData method, we add a readData method of the Employee class. When you call

e.readData(in);

this method overwrites the previous contents of e. Note that the method may throw an IOException if the readLine method throws that exception. There is nothing this method can do if an IOException occurs, so we just let it propagate up the chain.

Here is the code for this method:

public void readData(BufferedReader in) throws IOException 
{  String line = in.readLine(); 
   if (line == null) return; 
   StringTokenizer t = new StringTokenizer(line, "|"); 
   name = t.nextToken(); 
   salary = Format.atof(t.nextToken()); 
   int y = Format.atoi(t.nextToken()); 
   int m = Format.atoi(t.nextToken()); 
   int d = Format.atoi(t.nextToken()); 
   hireDay = new Day(y, m, d); 
}

Finally, in the code for a program that tests these methods, the static method

void writeData(Employee[] e, PrintWriter out)

first writes the length of the array, then writes each record. The static method

readData(Employee[] BufferedReader in)

first reads in the length of the array, then reads in each record, as illustrated in Example 1-2.

import java.io.*; 
import java.util.*; 
import corejava.*; 

public class DataFileTest 
{  static void writeData(Employee[] e, PrintWriter os) 
      throws IOException 
   {  Format.print(os, "%d
", e.length); 
      int i; 
      for (i = 0; i < e.length; i++) 
         e[i].writeData(os); 
   } 

   static Employee[] readData(BufferedReader is) 
      throws IOException 
   {  int n = Format.atoi(is.readLine()); 
      Employee[] e = new Employee[n]; 
      int i; 
      for (i = 0; i < n; i++) 
      {  e[i] = new Employee(); 
         e[i].readData(is); 
      } 
      return e; 
   } 


   public static void main(String[] args) 
   {  Employee[] staff = new Employee[3]; 

      staff[0] = new Employee("Harry Hacker", 35500, 
         new Day(1989,10,1)); 
      staff[1] = new Employee("Carl Cracker", 75000, 
         new Day(1987,12,15)); 
      staff[2] = new Employee("Tony Tester", 38000, 
         new Day(1990,3,15)); 
      int i; 
      for (i = 0; i < staff.length; i++) 
         staff[i].raiseSalary(5.25); 
      try 
      {  PrintWriter os = new PrintWriter(new 
            FileWriter("employee.dat")); 
         writeData(staff, os); 
         os.close(); 
      } 
      catch(IOException e) 
      {  System.out.print("Error: " + e); 
         System.exit(1); 
      } 

      try 
      {  BufferedReader is = new BufferedReader(new 
            FileReader("employee.dat")); 
         Employee[] in = readData(is); 
         for (i = 0; i < in.length; i++) in[i].print(); 
         is.close(); 
      } 
      catch(IOException e) 
      {  System.out.print("Error: " + e); 
         System.exit(1); 
      } 
   } 
} 

class Employee 
{  public Employee(String n, double s, Day d) 
   {  name = n; 
      salary = s; 
      hireDay = d; 
   } 
   public Employee() {} 
   public void print() 
   {  System.out.println(name + " " + salary 
         + " " + hireYear()); 
   } 
   public void raiseSalary(double byPercent) 
   {  salary *= 1 + byPercent / 100; 
   } 
   public int hireYear() 
   {  return hireDay.getYear(); 
   } 
   public void writeData(PrintWriter os) throws IOException 
   {  Format.print(os, "%s|", name); 
      Format.print(os, "%.14g|", salary); 
      Format.print(os, "%d|", hireDay.getYear()); 
      Format.print(os, "%d|", hireDay.getMonth()); 
      Format.print(os, "%d
", hireDay.getDay()); 
   } 
   public void readData(BufferedReader is) throws IOException 
   {  String s = is.readLine(); 
      StringTokenizer t = new StringTokenizer(s, "|"); 
      name = t.nextToken(); 
      salary = Format.atof(t.nextToken()); 
      int y = Format.atoi(t.nextToken()); 
      int m = Format.atoi(t.nextToken()); 
      int d = Format.atoi(t.nextToken()); 
      hireDay = new Day(y, m, d); 
   } 

   private String name; 
   private double salary; 
   private Day hireDay; 
}

If you have a large number of employees, the storage technique used in the preceding section suffers from one limitation: it is not possible to read a record in the middle of the file without first reading all records that come before it. In this section, we will make all records the same length. This lets us implement a random-access method of reading back the information—we can get at any record in the same amount of time.

We will store the numbers in the instance fields in our classes in a binary format. This is done using the writeInt and writeDouble methods of the DataOutput interface. (This is the common interface of the DataOutputStream and the RandomAccessFile classes.)

However, since the size of each record must remain constant, we need to make all the strings the same size when we save them. The variable-size UTF format does not do this, and the rest of the Java library provides no convenient means for accomplishing this. We need to write a bit of code to implement two helper methods. We will call them writeFixedString and readFixedString. These methods read and write Unicode strings that always have the same length.

The writeFixedString method takes the parameter size. Then, it writes the specified number of characters, starting at the beginning of the string. (If there are too few characters, it pads the string using characters whose ASCII/Unicode values are zero.) Here is the code for the writeFixedString method:

static void writeFixedString 
   (String s, int size, DataOutput out) 
   throws IOException 
{  int i; 
   for (i = 0; i < size; i++) 
   {  char ch = 0; 
      if (i < s.length()) ch = s.charAt(i); 
      out.writeChar(ch); 
   } 
}

The readFixedString method reads characters from the input stream until it has consumed size characters, or until it encounters a character with Unicode 0. Then, it should skip past the remaining zero characters in the input field.

For added efficiency, this method uses the StringBuffer class to read in a string. A StringBuffer is an auxiliary class that lets you preallocate a memory block of a given length. In our case, we know that the string is, at most, size bytes long. We make a string buffer in which we reserve size characters. Then we append the characters as we read them in.

NOTE

This is more efficient than reading in characters and appending them to an existing string. Every time you append characters to a string, Java needs to find new memory to hold the larger string: this is time consuming. Appending even more characters means the string needs to be relocated again and again. Using the StringBuffer class avoids this problem.

Once the string buffer holds the desired string, we need to convert it to an actual String object. This is done with the String(StringBuffer b) constructor. This constructor does not copy the characters in the string buffer. Instead, it freezes the buffer contents. If you later call a method that makes a modification to the StringBuffer object, the buffer object first gets a new copy of the characters and then modifies those.

static String readFixedString(int size, DataInput in) 
   throws IOException 
{  StringBuffer b = new StringBuffer(size); 
   int i = 0; 
   boolean more = true; 
   while (more && i < size) 
   {  char ch = in.readChar(); 
      i++; 
      if (ch == 0) more = false; 
      else b.append(ch); 
   } 
   in.skipBytes(2 * (size - i)); 
   return b.toString(); 
}

NOTE

These two functions are packaged inside the DataIO helper class.

To write a fixed-size record, we simply write all fields in binary.

public void writeData(DataOutput out) throws IOException 
{  DataIO.writeFixedString(name, NAME_SIZE, out); 
   out.writeDouble(salary); 
   out.writeInt(hireDay.getYear()); 
   out.writeInt(hireDay.getMonth()); 
   out.writeInt(hireDay.getDay()); 
}

Reading the data back is just as simple.

public void readData(DataInput in) throws IOException 
{  name = DataIO.readFixedString(NAME_SIZE, in); 
   salary = in.readDouble(); 
   int y = in.readInt(); 
   int m = in.readInt(); 
   int d = in.readInt(); 
   hireDay = new Day(y, m, d); 
}

In our example, each employee record is 100 bytes long because we specified that the name field would always be written using 40 characters. This gives us a breakdown as indicated in the following:

As an example, suppose we want to position the file pointer to the third record. We can use the following version of the seek method:

long int n = 3; 
int RECORD_SIZE = 100; 
in.seek((n - 1) * RECORD_SIZE);

To determine the total number of bytes in a file, use the length method. The total number of records is the length divided by the size of each record.

long int nbytes = in.length(); // length in bytes 
int nrecords = (int)(nbytes / RECORD_SIZE);

The test program shown in Example 1-3 writes three records into a data file and then reads them from the file in reverse order. To do this efficiently requires random access—we need to get at the third record first.

import java.io.*; 
import corejava.*; 

public class RandomFileTest 
{  public static void main(String[] args) 
   {  Employee[] staff = new Employee[3]; 

      staff[0] = new Employee("Harry Hacker", 35000, 
         new Day(1989,10,1)); 
      staff[1] = new Employee("Carl Cracker", 75000, 
         new Day(1987,12,15)); 
      staff[2] = new Employee("Tony Tester", 38000, 
         new Day(1990,3,15)); 
      int i; 
      try 
      {  DataOutputStream out = new DataOutputStream(new 
            FileOutputStream("employee.dat")); 
         for (i = 0; i < staff.length; i++) 
            staff[i].writeData(out); 
         out.close(); 
      } 
      catch(IOException e) 
      {  System.out.print("Error: " + e); 
         System.exit(1); 
      } 

      try 
      {  RandomAccessFile in 
            = new RandomAccessFile("employee.dat", "r"); 
         int n = (int)(in.length() / Employee.RECORD_SIZE); 
         Employee[] newStaff = new Employee[n]; 

         for (i = n - 1; i >= 0; i--) 
         {  newStaff[i] = new Employee(); 
            in.seek(i * Employee.RECORD_SIZE); 
            newStaff[i].readData(in); 
         } 
         for (i = 0; i < newStaff.length; i++) 
            newStaff[i].print(); 
      } 
      catch(IOException e) 
      {  System.out.print("Error: " + e); 
         System.exit(1); 
      } 

   } 
} 
class Employee 
{  public Employee(String n, double s, Day d) 
   {  name = n; 
      salary = s; 
      hireDay = d; 
   } 
   public Employee() {} 
   public void print() 
   {  System.out.println(name + " " + salary 
         + " " + hireYear()); 
   } 
   public void raiseSalary(double byPercent) 
   {  salary *= 1 + byPercent / 100; 
   } 
   public int hireYear() 
   {  return hireDay.getYear(); 
   } 
   public void writeData(DataOutput out) throws IOException 
   {  DataIO.writeFixedString(name, NAME_SIZE, out); 
      out.writeDouble(salary); 
      out.writeInt(hireDay.getYear()); 
      out.writeInt(hireDay.getMonth()); 
      out.writeInt(hireDay.getDay()); 
   } 

   public void readData(DataInput in) throws IOException 
   {  name = DataIO.readFixedString(NAME_SIZE, in); 
      salary = in.readDouble(); 
      int y = in.readInt(); 
      int m = in.readInt(); 
      int d = in.readInt(); 
      hireDay = new Day(y, m, d); 
   } 

   public static final int NAME_SIZE = 40; 
   public static final int RECORD_SIZE 
      = 2 * NAME_SIZE + 8 + 4 + 4 + 4; 

   private String name; 
   private double salary; 
   private Day hireDay; 
} 

class DataIO 
{  public static String readFixedString(int size, 
      DataInput in) throws IOException 
   {  StringBuffer b = new StringBuffer(size); 
      int i = 0; 
      boolean more = true; 
      while (more && i < size) 
      {  char ch = in.readChar(); 
         i++; 
         if (ch == 0) more = false; 
         else b.append(ch); 
      } 
      in.skipBytes(2 * (size - i)); 
      return b.toString(); 
   } 

   public static void writeFixedString(String s, int size, 
      DataOutput out) throws IOException 
   {  int i; 
      for (i = 0; i < size; i++) 
      {  char ch = 0; 
         if (i < s.length()) ch = s.charAt(i); 
         out.writeChar(ch); 
      } 
   } 
}

java.lang.StringBuffer

• StringBuffer()

  constructs an empty string buffer.

• StringBuffer(int length)

  constructs an empty string buffer with the initial capacity length.

• StringBuffer(String str)

  constructs a string buffer with the initial contents str.

• int length()

  returns the number of characters of the buffer.

• int capacity()

  returns the current capacity, that is, the number of characters that can be contained in the buffer before it must be relocated.

• void ensureCapacity(int m)

  enlarges the buffer if the capacity is fewer than m characters.

• void setLength(int n)

  If n is less than the current length, characters at the end of the string are discarded. If n is larger than the current length, the buffer is padded with '' characters.

• char charAt(int i)

  returns the i ’th character (i is between 0 and length()-1 ); throws a StringIndexOutOfBoundsException if the index is invalid.

• void getChars(int from, int to, char a[], int offset) copies characters from the string buffer into an array.

  Parameters	from	the first character to copy
  	to	the first character not to copy
  	a	the array to copy into
  	offset	the first position in a to copy into

• void setCharAt(int i, char ch)

  sets the i ’th character to ch.

• StringBuffer append(String str)

  appends a string to the end of this buffer (the buffer may be relocated as a result); returns this.

• StringBuffer append(char c)

  appends a character to the end of this buffer (the buffer may be relocated as a result); returns this.

• StringBuffer insert(int offset, String str)

  inserts a string at position offset into this buffer (the buffer may be relocated as a result); returns this.

• StringBuffer insert(int offset, char c)

  inserts a character at position offset into this buffer (the buffer may be relocated as a result); returns this.

• String toString()

  returns a string pointing to the same data as the buffer contents. (No copy is made.)

java.lang.String

• String(StringBuffer buffer)

  makes a string pointing to the same data as the buffer contents. (No copy is made.)

To save object data, you first need to open an ObjectOutputStream object:

ObjectOutputStream out = new ObjectOutputStream(new 
   FileOutputStream("employee.dat"));

Now, to save an object, you simply use the writeObject method of the ObjectOutputStream class as in the following fragment:

Employee harry = new Employee("Harry Hacker", 
   35000, new Day(1989, 10, 1)); 
Manager carl = new Manager("Carl Cracker", 
   75000, new Day(1987, 12, 15)); 
out.writeObject(harry); 
out.writeObject(carl);

To read the objects back in, first get an ObjectInputStream object:

ObjectInputStream in = new ObjectInputStream(new 
   FileInputStream("employee.dat"));

Then, retrieve the objects in the same order in which they were written, using the readObject method.

Employee e1 = (Employee)in.readObject(); 
Employee e2 = (Employee)in.readObject();

When reading back objects, you must carefully keep track of the number of objects that were saved, their order, and their types. Each call to readObject reads in another object of the type Object. You, therefore, will need to cast it to its correct type.

If you don’t need the exact type, or you don’t remember it, then you can cast it to any superclass or even leave it as type Object. For example, e2 is an Employee object variable even though it actually refers to a Manager object. If you need to dynamically query the type of the object, you can use the getClass method that we described in Chapter 5 of Volume 1.

You can only write and read objects, not numbers. To write and read numbers, you use methods such as writeInt/readInt or writeDouble/readDouble. (The object stream classes implement the DataInput /DataOutput interfaces.) Of course, numbers inside objects (such as the salary field of an Employee object) are saved and restored automatically. (Recall that, in Java, strings and arrays are objects and can, therefore, be restored with the writeObject/readObject methods.)

There is, however, one change you need to make to any class that you want to save and restore in an object stream. The class must implement the Serializable interface:

class Employee implements Serializable { . . .}

The Serializable interface has no methods, so you don’t need to change your classes in any way. In this regard, it is similar to the Cloneable interface that we also discussed in Chapter 5 of Volume 1. However, to make a class cloneable, you still had to override the clone method of the Object class. To make a class serializable, you do not need to do anything else. Why aren’t all classes serializable by default? We will discuss this in the section “Security.”

Example 1-4 is a test program that writes an array containing two employees and one manager to disk and then restores it. Once the information is restored, we give each employee a 100% raise, not because we are feeling generous, but because you can then easily distinguish employee and manager objects by their different raiseSalary actions. This should convince you that we did restore the correct type.

import java.io.*; 
import corejava.*; 

class ObjectFileTest 
{  public static void main(String[] args) 
   {  try 
      {  Employee[] staff = new Employee[3]; 

         staff[0] = new Employee("Harry Hacker", 35000, 
            new Day(1989,10,1)); 
         staff[1] = new Manager("Carl Cracker", 75000, 
            new Day(1987,12,15)); 
         staff[2] = new Employee("Tony Tester", 38000, 
            new Day(1990,3,15)); 

         ObjectOutputStream out = new ObjectOutputStream(new 
            FileOutputStream("test1.dat")); 
         out.writeObject(staff); 
         out.close(); 

         ObjectInputStream in =  new 
            ObjectInputStream(new FileInputStream("test1.dat")); 
         Employee[] newStaff = (Employee[])in.readObject(); 

         int i; 
         for (i = 0; i < newStaff.length; i++) 
            newStaff[i].raiseSalary(100); 
         for (i = 0; i < newStaff.length; i++) 
            newStaff[i].print(); 
      } 
      catch(Exception e) 
      {  System.out.print("Error: " + e); 
         System.exit(1); 
      } 
   } 
} 

class Employee implements Serializable 
{  public Employee(String n, double s, Day d) 
   {  name = n; 
      salary = s; 
      hireDay = d; 
   } 

   public Employee() {} 
   public void print() 
   {  System.out.println(name + " " + salary 
         + " " + hireYear()); 
   } 

   public void raiseSalary(double byPercent) 
   {  salary *= 1 + byPercent / 100; 
   } 

   public int hireYear() 
   {  return hireDay.getYear(); 
   } 

   private String name; 
   private double salary; 
   private Day hireDay; 
} 

class Manager extends Employee 
{  public Manager(String n, double s, Day d) 
   {  super(n, s, d); 
      secretaryName = ""; 
   } 

   public Manager() {} 

   public void raiseSalary(double byPercent) 
   {  // add 1/2% bonus for every year of service 
      Day today = new Day(); 
      double bonus = 0.5 * (today.getYear() - hireYear()); 
      super.raiseSalary(byPercent + bonus); 
   } 

   public void setSecretaryName(String n) 
   {  secretaryName = n; 
   } 

   public String getSecretaryName() 
   {  return secretaryName; 
   } 

   private String secretaryName; 
}

java.io.ObjectOutputStream

• ObjectOutputStream(OutputStream out)

  creates an ObjectOutputStream so that you can write to the specified OutputStream.

• void writeObject(Object obj)

  writes the specified object to the ObjectOutputStream. The class of the object, the signature of the class, and the values of any field not marked as transient are written, as well as the non-static fields of the class and all of its supertypes.

• ObjectInputStream(InputStream is)

  creates an ObjectInputStream to read back object information from the specified InputStream.

• Object readObject()

  reads an object from the ObjectInputStream. In particular, this reads back the class of the object, the signature of the class, and the values of the non-transient and non-static fields of the class and all of its superclasses. It does deserializing to allow multiple object references to be recovered.

Object serialization saves object data in a particular file format. Of course, you can use the writeObject/readObject methods without having to know the exact sequence of bytes that represents objects in a file. Nonetheless, we found studying the data format to be extremely helpful for gaining insight into the object streaming process. We did this by looking at hex dumps of various saved object files. However, the details are somewhat technical, so feel free to skip this section if you are not interested in the implementation.

Every file begins with the 2-byte “magic number”

followed by the version number of the object serialization format, which is currently

(We will be using hexadecimal numbers throughout this section to denote bytes.) Then it contains a sequence of objects, in the order that they were saved.

String objects are saved as

For example, the string “Harry” is saved as

The Unicode characters of the string are saved in UTF format.

When saving an object, the class of that object must be saved as well. The class description contains

Java gets the fingerprint by:

SHA is a very fast algorithm that gives a “fingerprint” to a larger block of information. This fingerprint is always a 20-byte data packet, regardless of the size of the original data. It is created by a clever sequence of bit operations on the data that makes it essentially 100% certain that the fingerprint will change if the information is altered in any way. SHA is a U.S. standard, recommended by the National Institute for Science and Technology (NIST). (For more details on SHA, see, for example, Network and Internetwork Security, by William Stallings [Prentice-Hall].) However, Java only uses the first 8 bytes of the SHA code as a class fingerprint. It is still very likely that the class fingerprint will change if the data fields or methods change in any way.

Java can then check the class fingerprint in order to protect us from the following scenario: An object is saved to a disk file. Later, the designer of the class makes a change, for example, by removing a data field. Then, the old disk file is read in again. Now the data layout on the disk no longer matches the data layout in memory. If the data were read back in its old form, it could corrupt memory. Java takes great care to make such memory corruption close to impossible. Hence, it checks, using the fingerprint, that the class definition has not changed when restoring an object. It does this by comparing the fingerprint on disk with the fingerprint of the current class.

NOTE

Technically, as long as the data layout of a class has not changed, it ought to be safe to read objects back in. But Java is conservative and checks that the methods have not changed either. (After all, the methods describe the meaning of the stored data.) Of course, in practice, classes do evolve and it may be necessary for a program to read in older versions of objects. We will discuss this in the section “Versioning Objects.”

Here is how a class identifier is stored:

The flag byte is composed of three bit masks, defined in

java.io.ObjectStreamConstants: 

  static final byte SC_WRITE_METHOD = 1; 
     // class has writeObject method that writes additional data 
  static final byte SC_SERIALIZABLE = 2; 
     // class implements Serializable interface 
  static final byte SC_EXTERNALIZABLE = 4; 
     // class implements Externalizable interface

We will discuss the Externalizable interface later in this chapter; for now, all our example classes will implement the Serializable interface and have a flag value of 02.

Each data field descriptor has the format

The type code is one of the following:

When the type code is L, the field name is followed by the field type. Class and field name strings do not start with the string code 74, but field types do. Field types use a slightly different encoding of their names, namely, the format used by native methods. (See Chapter 10 for native methods.)

For example, the day field of the Day class is encoded as

I 00 03 d a y

Here is the complete class descriptor of the Day class:

72 
00 0C c o r e j a v a . D a y 
16 9A C1 B6 6E 7E C0 13 
02 
00 03 
I 00 03 d a y 
I 00 05 m o n t h 
I 00 04 y e a r 
78 
70

These descriptors are fairly long. If the same class descriptor is needed again in the file, then an abbreviated form is used:

The serial number refers to the previous explicit class descriptor. We will discuss the numbering scheme later.

An object is stored as

For example, here is how a Day object is stored:

As you can see, the data file contains enough information to restore the Day object.

Arrays are saved in the following format:

The array class name in the class descriptor is in the same format as that used by native methods (which is slightly different from the class name used by class names in other class descriptors). In this format, class names start with an L and end with a semicolon.

For example, here is an array of two Day objects.

75

   72

      00 0F

      [ L c o r e j a v a / D a y ;

      FE . . . 36 02

      00 00

      78

      70

   00 00 00 02

   73

      72 . . . 70

      00 00 00 01

      00 00 00 0A

      00 00 07 C5

   73

      71 00 7E 00 02

      00 00 00 0F

      00 00 00 0C

      00 00 07 C3

Of course, studying these codes can be about as exciting as reading the average phone book. But it is still instructive to know that the object stream contains a detailed description of all the objects that it contains, with sufficient detail to be able to reconstruct both objects and arrays of objects.

We now know how to save objects that contain numbers, strings, or other simple objects (like the Day object in the Employee class). However, there is one important situation that we still need to consider. What happens when one object is shared by several objects as part of its state?

To illustrate the problem, let us make a slight modification to the Manager class. Rather than storing the name of the secretary, save a reference to a secretary object, which is an object of type Employee. (It would make sense to derive a class Secretary from Employee for this purpose, but we will not do that here.)

class Manager extends Employee 
{  // previous code remains the same 
   private Employee secretary; 
}

This is a better approach to designing a realistic Manager class than simply using the name of the secretary—the Employee record for the secretary can now be accessed without having to search the staff array.

Having done this, you must keep in mind that the Manager object now contains a reference to the Employee object that describes the secretary, not a separate copy of the object.

In particular, two managers can share the same secretary, as is the case in Figure 1-5 and the following code:

Figure 1-5. Two managers can share a mutual employee

harry = new Employee("Harry Hacker", . . .); 
Manager carl = new Manager("Carl Cracker", . . .); 
carl.setSecretary(harry); 
Manager tony = new Manager("Tony Tester, . . .); 
tony.setSecretary(harry);

Now suppose we write the employee data to disk. What we don’t want is that the Manager saves its information according to the following logic:

Then, the data for harry would be saved three times. When reloaded, the objects would have the configuration shown in Figure 1-6.

Figure 1-6. Here, Harry is saved three times

This is not what we want. Suppose the secretary gets a raise. We would not want to hunt for all other copies of that object and apply the raise as well. We want to save and restore only one copy of the secretary. To do this, we must copy and restore the original references to the objects. In other words, we want the object layout on disk to be exactly like the object layout in memory. This is called persistence in object-oriented circles.

Of course, we cannot save and restore the memory addresses for the secretary objects. When an object is reloaded, it will likely occupy a completely different memory address than it originally did.

Instead, Java uses a serialization approach. Hence, the name object serialization for this new mechanism. Remember:

When reading back the objects, we simply reverse the procedure. For each object that we load, we note its sequence number and remember where we put it in memory. When we encounter the tag “same as previously saved object with serial number x”, we look up where we put the object with serial number x and set the object reference to that memory address.

Note that the objects need not be saved in any particular order. Figure 1-8 shows what happens when a manager occurs first in the staff array.

Figure 1-8. Objects saved in random order

All of this sounds confusing, and it is. Fortunately, when using object streams, it is also completely automatic. Object streams assign the serial numbers and keep track of duplicate objects. The exact numbering scheme is slightly different from that used in the figures—see the next section.

NOTE

In this chapter, we use serialization to save a collection of objects to a disk file and retrieve it exactly as we stored it. Another very important application is the transmittal of a collection of objects across a network connection to another computer. Just as raw memory addresses are meaningless in a file, they are also meaningless when communicating with a different processor. Since serialization replaces memory addresses with serial numbers, it permits the transport of object collections from one machine to another. We will study that use of serialization in Chapter 5.

Example 1-5 is a program that saves and reloads a network of employee and manager objects (some of which share the same employee as a secretary). Note that the secretary object is unique after reloading—when staff[0] gets a raise, that is reflected in the secretary fields of the managers.

import java.io.*; 
import java.util.*; 
import corejava.*; 

class ObjectRefTest 
{  public static void main(String[] args) 
   {  try 
      {
         Employee[] staff = new Employee[3]; 

         Employee harry = new Employee("Harry Hacker", 35000, 
            new Day(1989,10,1)); 
         staff[0] = harry; 
         staff[1] = new Manager("Carl Cracker", 75000, 
            new Day(1987,12,15), harry); 
         staff[2] = new Manager("Tony Tester", 38000, 
            new Day(1990,3,15), harry); 

         ObjectOutputStream out = new ObjectOutputStream(new 
            FileOutputStream("test2.dat")); 
         out.writeObject(staff); 
         out.close(); 

         ObjectInputStream in =  new 
            ObjectInputStream(new FileInputStream("test2.dat")); 
         Employee[] newStaff = (Employee[])in.readObject(); 

         for (int i = 0; i < newStaff.length; i++) 
            newStaff[i].raiseSalary(100); 
         for (int i = 0; i < newStaff.length; i++) 
            newStaff[i].print(); 
      } 
      catch(Exception e) 
      {  e.printStackTrace(); 
         System.exit(1); 
      } 
   } 
} 

class Employee implements Serializable 
{  public Employee(String n, double s, Day d) 
   {  name = n; 
      salary = s; 
      hireDay = d; 
   } 
   public Employee() {} 

   public void raiseSalary(double byPercent) 
   {  salary *= 1 + byPercent / 100; 
   } 

   public int hireYear() 
   {  return hireDay.getYear(); 
   } 

   public void print() 
   {  System.out.println(name + " " + salary 
         + " " + hireYear()); 
   } 

   private String name; 
   private double salary; 
   private Day hireDay; 
} 

class Manager extends Employee 
{  public Manager(String n, double s, Day d, Employee e) 
   {  super(n, s, d); 
      secretary = e; 
   } 

   public Manager() {} 

   public void raiseSalary(double byPercent) 
   {  // add 1/2% bonus for every year of service 
      Day today = new Day(); 
      double bonus = 0.5 * (today.getYear() - hireYear()); 
      super.raiseSalary(byPercent + bonus); 
   } 

   public void print() 
   {  super.print(); 
      System.out.print("Secretary: "); 
      if (secretary != null) secretary.print(); 
   } 

   private Employee secretary; 
}

This section continues the discussion of the output format of object streams. If you skipped the discussion before, you should skip this section as well.

All objects (including arrays and strings) and all class descriptors are given serial numbers as they are saved in the output file. This process is referred to as serialization since every saved object is assigned a serial number. (The count starts at 00 7E 00 00.

We already saw that a full class descriptor for any given class only occurs once. Subsequent descriptors refer to it. For example, in our previous example, the second reference to the Day class in the array of days was coded as

71 00 7E 00 02

The same mechanism is used for objects. If a reference to a previously saved object is written, it is saved in exactly the same way, that is, 71 followed by the serial number. It is always clear from the context whether the particular serial reference denotes a class descriptor or an object.

Finally, a null reference is stored as

70

Here is the commented output of the ObjectRefTest program of the preceding section. If you like, run the program, look at a hex dump of its data file test2.dat, and compare it with the commented listing. The important lines towards the end of the output (in bold) show the reference to a previously saved object.

AC ED 00 05

75

   72

      00 0B

      [ L E m p l o y e e ;

      FC BF 36 11 C5 91 11 C7 02

      00 00

      78

      70

      00 00 00 03

   73

      72

         00 08

         E m p l o y e e

         3E BB 06 E1 38 0F 90 C9 02

         00 03

         D 00 06 salary

         L 00 07 hireDay

           74 00 0E Lcorejava/Day;

         L 00 04 name

           74 00 12 Ljava/lang/String;

         78

         70

      40 E1 17 00 00 00 00 00

   73

         72

            00 0C

            c o r e j a v a . D a y

            16 9A C1 B6 6E 7E C0 13 02

            00 03

            I 00 03 day

            I 00 05 month

            I 00 04 year

            78

            70

         00 00 00 01

         00 00 00 0A

         00 00 07 C5

      74

         00 0C

         H a r r y   H a c k e r

   73

      72

         00 07

         M a n a g e r

         B1 C5 48 6B 95 EE BE C2 02

         00 01

         L 00 09 secretary

         74 00 0A Employee;

         78

         71 00 7E 00 02

      40 F2 4F 80 00 00 00 00

   73

         71 00 7E 00 06 

         00 00 00 0F

         00 00 00 0C

         00 00 07 C3

      74

         00 0C

         C a r l   C r a c k e r

      71 00 7E 00 05 

   73

      71 00 7E 00 09 

      40 E2 8E 00 00 00 00 00

   73

         71 00 7E 00 06

         00 00 00 0F

         00 00 00 03

         00 00 07 C6

      74

         00 0B

         T o n y   T e s t e r

      71 00 7E 00 05 

It is usually not important to know the exact file format (unless you are trying to create an evil effect by modifying the data—see the next section). What you should remember is this:

Even if you only glanced at the file format description of the preceding section, it should become obvious that a knowledgeable hacker can exploit this information and modify an object file so that invalid objects will be read in when you go to reload the file.

Consider, for example, the Day class in the corejava package. That class has been carefully designed so that all of its constructors check that the day, month, and year fields never represent an invalid date. For example, if you try to build a new Day(1996, 2, 31), no object is created and an IllegalArgumentException is thrown instead.

However, this safety guarantee can be subverted through serialization. When a Day object is read in from an object stream, it is possible—either through a device error or through malice—that the stream contains an invalid date. There is nothing that the serialization mechanism can do in this case—it has no understanding of the constraints that define a legal date.

For that reason, Java’s serialization mechanism provides a way for individual classes to add validation or any other desired action instead of the default behavior. A serializable class can define methods with the signature

private void readObject(ObjectInputStream in) 
   throws IOException, ClassNotFoundException; 
private void writeObject(ObjectOutputStream out) 
   throws IOException;

Then, the data fields are no longer automatically serialized, and these methods are called instead.

For example, let us add validation to the Day class. We don’t need to change the writing of Day objects, so we won’t implement the writeObject method.

In the readObject method, we first need to read the object state that was written by the default write method, by calling the defaultReadObject method. This is a special method of the ObjectInputStream class that can only be called from within a readObject method of a serializable class.

class Day 
{  . . . 
   private void readObject(ObjectInputStream in) 
      throws IOException, ClassNotFoundException 
   {  in.defaultReadObject(); 
      if (!isValid()) throw new IOException("Invalid date"); 
   } 
}

If the day, month, and year fields do not represent a valid date (for example, because someone modified the data file), then we throw an exception.

NOTE

Another way of protecting serialized data from tampering is authentication. As we will see in Chapter 8, a stream can save a message digest (such as the SHA fingerprint) to detect any corruption of the stream data.

Classes can also write additional information to the output stream by defining a writeObject method that first calls defaultWriteObject and then writes other data. Of course, the readObject method must then read the saved data—otherwise, the stream state will be out of synch with the object. Also, the writeObject and readObject can completely bypass the default storage of the object data by simply not calling the defaultWriteObject and defaultReadObject methods.

In any case, the readObject and writeObject methods only need to save and load their data fields. They should not concern themselves with superclass data or any other class information.

Rather than letting the serialization mechanism save and restore object data, a class can define its own mechanism. To do this, a class must implement the Externalizable interface. This in turn requires it to define two methods:

public void readExternal(ObjectInputStream in) 
  throws IOException, ClassNotFoundException; 
public void writeExternal(ObjectOutputStream out) 
  throws IOException;

Unlike the readObject and writeObject methods that were described in the preceding section, these methods will be fully responsible for saving and restoring the entire object, including the superclass data. The serialization mechanism merely records the class of the object in the stream.

CAUTION

Unlike the readObject and writeObject methods, which are private and can only be called by the serialization mechanism, the readExternal and writeExternal methods are public. In particular, readExternal potentially permits modification of the state of an object.

Finally, there are certain data members that should never be serialized, for example, integer values that store file handles or handles of windows that are only meaningful to native methods. Such information is guaranteed to be useless when you reload an object at a later time or transport it to a different machine. In fact, improper values for such fields can actually cause native methods to crash. Java has an easy mechanism to prevent such fields from ever being serialized. Mark them with the keyword transient. Transient fields are always skipped when objects are serialized.

Beyond the possibility of data corruption, there is another potentially worrisome security aspect to serialization. Any code that can access a reference to a serializable object can:

and thereby know the values of all the data fields in the objects, even the private ones. After all, the serialization mechanism automatically saves all private data. Fortunately, this knowledge cannot be used to modify data. The readObject method does not overwrite an existing object but always creates a new object. Nevertheless, if you need to keep certain information safe from inspection via the serialization mechanism, you should take one of the following three steps:

In the past sections, we showed you how to save relatively small collections of objects via an object stream. But those were just demonstration programs. With object streams, it helps to think big. Suppose you write a program that lets the user produce a document. This document contains paragraphs of text, tables, graphs, and so on. You can stream out the document object with a single call to writeObject, and the paragraph, table and graph objects are automatically streamed out as well. One user of your program can then give the output file to another user who also has a copy of your program, and that program loads the entire document with a single call to readObject.

This is very useful, but your program will inevitably change, and you will release a version 1.1. Can version 1.1 read the old files? Can the users who still use 1.0 read the files that the new version is now producing? Clearly, it would be desirable if object files could cope with the evolution of classes.

At first glance it seems that this would not be possible. When a class definition changes in any way, then its SHA fingerprint also changes and you know that Java will refuse to read in objects with different fingerprints. However, a class can indicate that it is compatible with an earlier version of itself. To do this, one must first obtain the fingerprint of the earlier version of the class. You use the standalone serialver program that is part of the JDK to obtain this number. For example, running

serialver corejava.Day

prints out

corejava.Day:    static final long serialVersionUID = 
   1628827204529864723L;

If you start the serialver program with the -show option, then it brings up a graphical dialog box (see Figure 1-9).

Figure 1-9. The graphical version of the serialver program

All later versions of the class must define the serialVersionUID constant to the same fingerprint as the original.

class Day // version 1.1 
{  . . . 
   static final long serialVersionUID = 1628827204529864723L; 
}

When a class has a static data member named serialVersionUID, it will not compute the fingerprint manually but instead will use that value.

Once that static data member has been placed inside a class, the serialization system is now willing to read in different versions of objects of that class.

If only the methods of the class change, then there is no problem with reading the object new data. However, if data fields change, then you may have problems. For example, the old file object may have more or fewer data fields than the one in the program, or the types of the data fields may be different. In that case, Java makes an effort to convert the stream object to the current version of the class.

Java compares the data fields of the current version of the class with the data fields of the version in the stream. Of course, Java considers only the non-transient and non-static data fields. If two fields have matching names but different types, then Java makes no effort to convert one type to the other—the objects are incompatible. If the object in the stream has data fields that are not present in the current version, then Java ignores the data in the stream. If the current version has data fields that are not present in the streamed object, the added fields are set to their default (null for objects, zero for numbers).

Here is an example. Suppose we have saved a number of employee records on disk, using the original version (1.0) of the class. Now we change the Employee class to version 2.0 by adding a data field called department. Figure 1-10 shows what happens when a 1.0 object is read into a program that uses 2.0 objects. The department field is set to null. Figure 1-11 shows the opposite scenario: a program using 1.0 objects reads a 2.0 object. The additional department field is ignored.

Figure 1-10. Reading an object with fewer data fields

Figure 1-11. Reading an object with more data fields

Is this process safe? It depends. Dropping a data field seems harmless—the recipient still has all the data that it knew how to manipulate. Setting a data field to null may not be so safe. Many classes work hard to initialize all data fields in all constructors to non-null values, so that the methods don’t have to be prepared to handle null data. It is up to the class designer to implement additional code in the readObject method to fix version incompatibilities or to make sure the methods are robust enough to handle null data.

import java.io.*; 
import corejava.*; 

public class SerialCloneTest 
{  public static void main(String[] args) 
   {  Employee harry = new Employee("Harry Hacker", 35000, 
         new Day(1989,10,1)); 
      Employee harry2 = (Employee)harry.clone(); 
      harry.raiseSalary(100); 
      harry.print(); 
      harry2.print(); 
   } 
} 

class SerialCloneable implements Cloneable, Serializable 
{  public Object clone() 
   {  try 
      {  ByteArrayOutputStream bout = new 
            ByteArrayOutputStream(); 
         ObjectOutputStream out = new ObjectOutputStream(bout); 
         out.writeObject(this); 
         out.close(); 
         ByteArrayInputStream bin = new 
            ByteArrayInputStream(bout.toByteArray()); 
         ObjectInputStream in = new ObjectInputStream(bin); 
         Object ret = in.readObject(); 
         in.close(); 
         return ret; 
      }  catch(Exception e) 
      {  return null; 
      } 
   } 
} 

class Employee extends SerialCloneable 
{  public Employee(String n, double s, Day d) 
   {  name = n; 
      salary = s; 
      hireDay = d; 
   } 
   public Employee() {} 

   public void print() 
   {  System.out.println(name + " " + salary + " " + 
         hireYear()); 
   } 

   public void raiseSalary(double byPercent) 
   {  salary *= 1 + byPercent / 100; 
   } 

   public int hireYear() 
   {  return hireDay.getYear(); 
   } 

   private String name; 
   private double salary; 
   private Day hireDay; 
}