The general form of a format specifier for general, character, or numeric conversions is

%[argument_index][flags][width][.precision]conversion

where everything except the % and the conversion indicator are optional.

If there is an error in the format string, a mismatch between the conversion and the other formatting requests, or a mismatch between the conversion and the argument type supplied, then an IllegalFormatException of some form will be thrown. These exceptions are described later.

The argument index is an optional indicator of which argument this format specifier should be applied to. This is very useful for referring to the same argument multiple times within the same format string. The argument index can take two forms: a number indicating which argument it applies to, followed by a $ character; or the < character meaning “the same argument as the previous format specifier.” If the argument index is not present, then each unmarked format specifier is numbered sequentially, and it is that argument to which the format specifier is applied. This numbering ignores the presence of any indexed format specifiers. For example, given

System.out.printf("%3$d %d %2$d %<d %d %n", 1, 2, 3);

the output is

3 1 2 2 2

The first specifier explicitly applies to argument three; the second is the first non-indexed specifier so it applies to argument one; the third specifier explicitly applies to argument two; the fourth specifier applies to whatever the third did, so that is again two; the fifth specifier is the second non-indexed specifier so it also applies to argument two.

If any format specifier indicates an argument that does not exist, then you get a MissingFormatArgumentException. Examples of this kind of error are using %3$ when there are only two arguments, or using %< as the first format specifier.

The meaning of flags, width, and precision is similar for the different conversions but the details differ depending on the exact kind of conversion, as discussed in the following sections.

When you consider which argument types are expected for each conversion, remember that anything that applies to a primitive type also applies to its wrapper. For brevity we don't mention the wrapper classes explicitly.

The integer conversions apply to arguments of type byte, short, int, and long. The different conversions are

The width value specifies the minimum number of characters to output—including additional characters that the flags may cause to be included. No precision value may be given. The flags applicable to each conversion are shown in the following table. An entry with a × means the flag is not allowed.

For example, to print a zero-padded, hexadecimal value with a radix indicator and a minimum width of 10, you could use

System.out.printf("%0#10x %n", val);

which for val equal to 32 prints

0x00000020

You cannot use + and space together, nor can you use – and 0 together. If either of 0 or – is given then a width must be given.

The floating-point conversions apply to arguments of type float and double. The different conversions are

In each case a value of NaN or infinity produces the strings "NaN" or "Infinity".

The width value specifies the minimum number of characters to output, with the result being padded if necessary to fit the requested width. For everything except general scientific conversion, the precision value specifies the number of digits to output after the decimal point (or hexadecimal point), with the value rounded as necessary to match that precision. If no precision is given then it defaults to 6, except for the hexadecimal exponential form where no precision means to use as many places as needed.

A conversion to general scientific notation chooses the output format based on the magnitude of the value and the precision that is given. In this case precision indicates the number of significant digits (not the number of fractional digits). If after rounding to the specified precision, the value is smaller than 10^-4, or it is greater than or equal to 10^precision, then computerized scientific notation is used; otherwise, decimal format is used. For example, given

System.out.
    printf("%1$.5g %1$.4g %1$.3g %1$.2g %n", Math.PI * 100);

The output is:

314.16 314.2 314 3.1e+02

You can see the value being rounded according to the number of significant digits requested, and see the switch to scientific notation when the value exceeds 10².

For the general scientific notation conversion, if the precision is not given it defaults to 6, but a precision of zero is treated as a precision of 1.

The flags applicable to each conversion are shown in the following table. An entry with × means the flag is not allowed.

As with the integer conversions, you cannot use + and space together, nor can you use – and 0 together. If either of 0 or – is given then a width must be given.

The character conversions apply to argument types of byte, short, and char, together with int provided that the int value represents a valid Unicode code point (otherwise IllegalFormatCodePointException is thrown).

The conversion indicator for characters is c or C and formats the argument as a Unicode character.

A width value specifies the minimum number of characters to output, with spaces used to pad the output as necessary. No precision value may be supplied. The only applicable flag is – to left-justify the output. If the – flag is given then a width must be given.

There are three general conversions that are unrelated but that are not numeric or character conversions, so we describe them together. They apply to an argument of any type.

The b and B conversions are boolean conversions. If the argument is null then the output is the string "false". If the argument is a boolean then the output is either the string "true" or the string "false" depending on the argument's value, otherwise the output is the string "true".

The h and H conversions are hash code conversions. If the argument is null then the output is the string "null". Otherwise, the output is the string obtained by passing the object's hashCode result to Integer.toHexString.

The s and S conversions are string conversions. If the argument is null then the output is the string "null". Otherwise, if the argument implements the Formattable interface (see next section) then its formatTo method is invoked to produce a customized output; otherwise, the output is the result of invoking toString on the argument. The # flag can be passed only to a Formattable argument, and its effect is determined by the object.

For all the general conversions, the width specifies the minimum number of characters, while the precision indicates the maximum. Spaces will be used to pad the output to the minimum width if necessary. The precision is applied first so if it is smaller than the width, the output will be truncated to that maximum size. The only flag applicable is the – flag to left-justify the output. If the – flag is given then a width must be given.

A class can support custom formatting by implementing the Formattable interface, which defines the single method formatTo. If an instance of that class is passed as an argument for a string conversion (s or S) then its formatTo method will be invoked, passing the current formatter as an argument.

Additionally, the flag value encodes whether the conversion indicator was lowercase or uppercase using the FormattableFlags.UPPERCASE constant. You can test whether a flag is set by ANDing the flags value with the desired constant and seeing if the result is non-zero.

By implementing Formattable a class can provide greater flexibility in how it represents itself textually, compared to the fixed output of the toString method. For example, it can use a long or short form, depending on the width and precision that were supplied; or it can adapt itself to the locale used by the formatter—see page 632.

If any errors occur while processing a format string with a given set of arguments, an exception that is a subclass of IllegalFormatException will be thrown. The types of exception you can encounter are

A Formatter object is always associated with some object that is the destination for the formatted text produced by the formatter. By default the destination is a StringBuilder. The other destination objects that can be passed to the various constructors are

The constructors for the byte-oriented destinations (files and output streams) either work with the platform's default encoding or take an additional string parameter that represents the character set encoding name to use. An additional constructor takes a PrintStream as a destination, but only the default platform encoding is used.

Formatter supports localization (see Chapter 24) and for each type of destination there is an additional constructor that takes a Locale object with which the formatter should work. If you do not specify a locale to the constructor, the default locale is used. The locale being used is returned by the locale method.

The destination of a Formatter object can be obtained from the out method, which returns an Appendable object.

If writing to the destination object could throw an IOException, then the last IOException to occur (if any) can be obtained from the ioException method.

Formatter implements Closeable and Flushable. If close or flush are invoked, then the destination is also closed or flushed if it too implements Closeable or Flushable. Once a formatter has been closed the only method that can be invoked is ioException; all others throw FormatterClosedException.

The primary method of Formatter is, of course, format:

The Formatter class's support of localized output is restricted to a subset of the numeric conversions (and the date/time conversions discussed in Chapter 24). The localized numeric conversions are the integer decimal conversion (d) and the three main floating-point conversions (e, f, g and their uppercase counterparts). For these conversions, digit characters are taken from the active locale, and the correct sign characters, decimal point character, and grouping character (for the , flag) are also taken from the locale. Similarly, if zero-padding is requested the zero character is that defined by the locale. The NumberFormat class described in Chapter 24, on page 710, provides more extensive formatting capabilities for localized numbers.

Exercise 22.1: Write a method that takes an array of floating-point values and a number indicating how many columns to use, and prints the array contents. Try to ensure that the entries in each column line up neatly. Assume a line is 80 characters wide.

When reading input as a stream of values, you simply ask Scanner to return the next value in the stream. The scanner determines the next value by first skipping over any delimiter characters in the input (which by default matches whitespace) and examining the input up to the next delimiter. For example, the following code is a rewrite of the sumStream example from page 533:

static double sumStream(Readable source) throws IOException {
    Scanner in = new Scanner(source);
    double result = 0.0;
    while (in.hasNext()) {
        if (in.hasNextDouble())
            result += in.nextDouble();
        else
            in.next();
    }
    IOException ex = in.ioException();
    if (ex != null)
        throw ex;

    return result;
}

You can iterate through the input one token at a time, asking if the next token is a string, int, or double value, and so forth, and storing it into a variable of the appropriate type.^[2] In contrast to StreamTokenizer, Scanner can produce primitive values other than just double, and it recognizes numbers in a range of formats, not just the common decimal format. (Essentially, how ever Formatter— see page 624—might write out a number as a string, Scanner can read it back as a number.)

A scanner needs a source of characters to read from, which in the most general case is represented by an object that implements the java.lang.Readable interface—this interface defines a single method, read, that takes a java.nio.CharBuffer object into which the characters read from the Readable should be stored. The sumStream method accepts a Readable parameter and passes it to the constructor for Scanner to indicate it should be used as the input source for that scanner. The java.io.Reader class implements Readable, so any of the input character streams can be used as a source for a Scanner object.

The loop continues as long as hasNext indicates that the scanner has another token available. Knowing that there is a token doesn't tell you what kind of token it may be, so we use the hasNextDouble method to ask if the token is a double value. If so, the value is read with nextDouble and added to the sum. Otherwise, the value is read (as a String) with next and ignored. The hasNext and next methods should be familiar to you—they are two of the methods of Iterator. And indeed Scanner implements Iterator<String> so you can easily iterate through the tokens in an input source.^[3] The remove method of Scanner throws UnSupportedOperationException.

The hasNext method returns true if the scanner has another token to return from next. For each primitive type except char, hasNextType asks if the next token is of that type, and if so, nextType will return it. For the integer types there are overloads of hasNextType and nextType that take an int radix for the number. For example, hasNextInt(8) will return true if the next token is a valid octal representation of an int value. The default radix of the scanner (initially 10) can be changed with the useRadix method, and retrieved from the radix method.

The hasNext method also has two additional overloads that take a pattern and will return true only if the next token matches the given pattern. Similarly, next also has two overloads that take a pattern describing the token to return.

All the methods that get the next token operate as follows: Delimiters from the current position to the first non-delimiter are ignored; then the next delimiter is found; if the input between the delimiters matches the pattern then a token has been found and the current position is advanced to the first character after the token. If there is no next token then NoSuchElementException is thrown. If the method requires a specific type of token or a token that matches a given radix or pattern, and the next token does not meet those criteria, then InputMismatchException is thrown. If an exception is thrown then the position of the scanner is unchanged.

Input sources for scanners can come from several places. There is a constructor that takes a Readable character source, one that takes a String, and several that take byte sources: File, InputStream or java.nio.ReadableByteBuffer. By default these byte sources will be converted to characters using the platform's default encoding, or you can use a form of the constructor that takes the character set name to use as the encoding.

When the source for a scanner is a file or other input stream, it is possible that when the scanner tries to read the source it will encounter an IOException. The scanner methods themselves do not declare or throw the IOException. The responsibility for dealing with these potential exceptions falls to you. If the scanner encounters an IOException it considers the end of the input source to have been reached—so hasNext will return false, and any attempt to take the next token will cause NoSuchElementException to be thrown. To determine whether tokenizing of the source ended because of a true end of input or an exception, you can use the ioException method. It returns the last IOException that was thrown by the underlying source, or null if there have been no exceptions. So, as in the example, when the iteration is complete you should check to see that it didn't complete prematurely due to an exception.

A scanner identifies tokens by looking for a delimiter pattern in the input stream of characters. By default the delimiter pattern matches any whitespace, as determined by the Character.isWhitespace method. You can set the delimiter to a different pattern with the useDelimiter method. The current delimiter pattern is returned as a Pattern object from the delimiter method. For example, here is a simple method that reads values from a source that is in comma-separated-values (CSV) format and returns them in a List:

public static List<String> readCSV(Readable source)
    throws IOException {
    Scanner in = new Scanner(source);
    in.useDelimiter(",|" +LINE_SEPARATOR_PATTERN);
    List<String> vals = new ArrayList<String>();

    while (in.hasNext())
        vals.add(in.next());

    IOException ex = in.ioException();
    if (ex!= null)
        throw ex;

    return vals;
}

Here the delimiter pattern is either a single comma or a line separator, since the values themselves can contain any other characters, including whitespace. The definition of a line separator is documented in the Pattern class. To make it easier to read the example, we defined a constant to represent that pattern:

static final String LINE_SEPARATOR_PATTERN =
    "
|[

u2028u2029u0085]";

The Scanner class also provides a close method. When you invoke close on a scanner, the input source will be closed if it is Closeable. Once a scanner has been closed, attempts to invoke any of the scanning operations will result in an IllegalStateException. Take care when closing a scanner: The input source may not be yours to close, such as an InputStreamReader for System.in. As a general rule, don't close a scanner unless you opened its input source.

In line mode, the scanner can process a line at a time using the regular expression pattern that you supply. If the line matches the pattern then a java.util.regex.MatchResult object can be obtained from the match method, from which you can extract the groups the pattern defined. Line matching is done with the findInLine method.

Note that if the pattern does not match the entire line then the scanner will be positioned to continue reading the remainder of the current line; and if the pattern starts in the middle of the line, then all preceding input is skipped. As usual a second form of this method takes a String representing the pattern.

You can use a scanner's hasNextLine and nextLine methods to tokenize input a complete line at a time. The hasNextLine method returns true if there is another line of input in this scanner. This means that between the current position of the scanner and the end of the input there is either a line separator or one or more other characters. The nextLine method advances the scanner to the start of the next line and returns all the input that was skipped, excluding the line separator itself.

Consider the example of input that is in comma-separated-value format. This format is often used to export values from a table, where each line in the output represents a row of the table and each value before a comma (or a newline) represents the contents of a cell. With a fixed number of cells per row we can process each line of input as a single entity:

static final int CELLS = 4;

public static List<String[]> readCSVTable(Readable source)
    throws IOException {
    Scanner in = new Scanner(source);
    List<String[]> vals = new ArrayList<String[]>();
    String exp = "^(.*),(.*),(.*),(.*)";
    Pattern pat = Pattern.compile(exp, Pattern.MULTILINE);
    while (in.hasNextLine()) {
        String line = in.findInLine(pat);
        if (line != null) {
            String[] cells = new String[CELLS];
            MatchResult match = in.match();
            for (int i = 0; i < CELLS; i++)
                cells[i] = match.group(i+1);
            vals.add(cells);
            in.nextLine();  // skip newline
        }


        else {
            throw new IOException("input format error");
        }
    }

    IOException ex = in.ioException();
    if (ex!= null)
        throw ex;

    return vals;
}

Each line of the input is expected to have four values separated by commas (naturally you'd want to adapt the pattern to account for different numbers of cells in different calls to the method). If the line matches what is expected then each value is extracted from the match group. When a match is found, the newline is left in the input so we use nextLine to skip over it. As we don't know how many lines there will be, we use a List to accumulate the rows.

The use of a pattern in multiline mode is common when scanning line-oriented input. The pattern we used has to match from the start of a line, so we need to include the start-of-line (^) boundary marker, and that only matches actual lines when the pattern is in multiline mode.

The findWithinHorizon methods operate similarly to findInLine except that they take an additional int parameter that specifies the maximum number of characters to look-ahead through. This “horizon” value is treated as a transparent, non-anchoring bound (see the Matcher class for details—Section 13.3.4 on page 329). A horizon of zero means there is no look-ahead limit.

The skip method can be used to skip over the input that matches a given pattern. As with findInLine and findWithinHorizon, it ignores the scanner's delimiters when looking for the pattern. The skipped input is not returned, rather skip returns the scanner itself so that invocations can be chained together.

Exercise 22.7: Rewrite readCSVTable so that the number of cells of data expected is passed as an argument.

Exercise 22.8: As it stands, readCSVTable is both too strict and too lenient on the input format it expects. It is too strict because an empty line at the end of the input will cause the IOException to be thrown. It is too lenient because a line of input with more than three commas will not cause an exception. Rectify both of these problems.

Exercise 22.9: Referring back to the discussion of efficiency of regular expressions on page 329, devise at least four patterns that will parse a line of comma-separated-values. (Hint: In addition to the suggestion on page 329 also consider the use of greedy versus non-greedy quantifiers.) Write a benchmark program that compares the efficiency of each pattern, and be sure that you test with both short strings between commas and very long strings.

Scanner and StreamTokenizer have some overlap in functionality, but they have quite different operational models. Scanner is based on regular expressions and so can match tokens based on whatever regular expressions you devise. However, some seemingly simple tasks can be difficult to express in terms of regular expression patterns. On the other hand, StreamTokenizer basically processes input a character at a time and uses the defined character classes to identify words, numbers, whitespace, and ordinary characters. You can control the character classes to some extent, but you don't have as much flexibility as with regular expressions. So some things easily expressed with one class are difficult, or at best awkward, to express with the other. For example, the built-in ability to handle comment lines is a boon for StringTokenizer, while using a scanner on commented text requires explicit, unobvious handling. For example:

Scanner in = new Scanner(source);
Pattern COMMENT = Pattern.compile("#.*");
String comment;
// ...
while (in.hasNext()) {
    if (in.hasNext(COMMENT)) {
        comment = in.nextLine();
    }
    else {
        // process other tokens
    }
}

This mostly works. The intent is that if we find that the next token matches a comment, then we skip the rest of the line by using nextLine. Note that you can't, for example, use a pattern of "#.*$" to try to match from the comment character to the end of the line, unless you are guaranteed there are no delimiters in the comment itself.

The above fails to work because it can act like there were two comments when there was only one. Consider a non-comment line followed by a comment line. The input stream might look something like this:

token
# This is a comment line
token2

After the last token on the non-comment line is processed, the current input position is just before the line separator that delimited the end of that token. When hasNext is invoked it looks past the line separator and sees that there is something there, so it returns true, leaving the current input position where it was. When hasNext(COMMENT) is invoked, it too ignores the line separator delimiter and sees a pattern on the next line that matches the comment, so it also returns true, leaving the current input position where it was. When nextLine is invoked its job is to advance the current position to the beginning of the next line and return the input that was skipped. The current position is immediately before the line separator, so moving it after the line separator is a very short move and, other than the line separator, no input is skipped. Consequently, nextLine returns an empty string. In our example this is not a problem because we loop around again and match the comment a second time, and this time we remove it properly. However, if your code assumed that the comment itself was gone after nextLine was invoked, then that assumption would be incorrect. We can fix this problem like so:

Scanner in = new Scanner(source);
Pattern COMMENT = Pattern.compile("#.*");
String comment;
// ...
while (in.hasNext()) {
    if (in.hasNext(COMMENT)) {
        comment = in.findWithinHorizon(COMMENT, 0);
        in.nextLine();
    }
    else {
        // process other tokens
    }
}

Now when hasNext(COMMENT) tells us that there is a comment ahead, we use findWithinHorizon(COMMENT,0) to skip the line separator, find the actual comment, and return it. We don't need to set any horizon because we know from hasNext that the comment is there. After findWithinHorizon returns the comment, the current position is just before the line separator at the end of the comment line, so we use nextLine to skip over that to the next line.

Another way to get the scanner to skip comments would be to make the comment pattern part of the delimiter pattern. But that is not quite as straightforward as it might sound. We leave that approach as an exercise for the reader.

So skipping comments was trivial with StreamTokenizer and quite involved with Scanner. In contrast, it is quite simple to change the scanner's delimiter to parse a comma-separated-variable file, but to do the same with StringTokenizer requires careful manipulation of the character classes to make the comma a whitespace character and to stop space from being considered a whitespace character. Although relatively simple to state, the API makes it awkward to do and it is conceptually bizarre.

StreamTokenizer is very good for working with free-format files such as that used in the attribute reading example on page 533. It read input that consisted of names and values, seperated by whitespace, with an optional = character in between, and stored them into an Attr object. The names were simply words, and values could be words or numbers, while the = character was an ordinary character. Pairing of names and values was trivially done, as was detecting a misplaced = character. In contrast, Scanner has a lot of trouble dealing with such a flexible format. If you add the = character to the delimiter it isn't too hard to simply treat every two words as a name and value pair. However, because delimiters are ignored, you can't detect misplaced = characters. If you don't make the = character a delimiter, then you have problems when there is no whitespace between the = and the name or value.

If you wanted to store Attr objects and read them back with a Scanner, you could have more flexibility with the values than if you used StreamTokenizer, but the file would be more restricted in format. For example, here is a pair of methods to print and scan attributes:

public static void printAttrs(Writer dest, Attr[] attrs) {
    PrintWriter out = new PrintWriter(dest);
    out.printf("%d attrs%n", attrs.length);
    for (int i = 0; i < attrs.length; i++) {
        Attr attr = attrs[i];
        out.printf("%s=%s%n",
                   attr.getName(), attr.getValue());
    }
    out.flush();
}

public static Attr[] scanAttrs(Reader source) {
    Scanner in = new Scanner(source);
    int count = in.nextInt();
    in.nextLine();    // skip rest of line


    Attr[] attrs = new Attr[count];
    Pattern attrPat =
        Pattern.compile("(.*?)=(.*)$", Pattern.MULTILINE);
    for (int i = 0; i < count; i++) {
        in.findInLine(attrPat);
        MatchResult m = in.match();
        attrs[i] = new Attr(m.group(1), m.group(2));
    }
    return attrs;
}

The printAttrs method uses printf to print an attribute in one line, using a = character to separate name from value. The scanAttrs method will read such a file back in. It uses a combination of stream-based and line-based Scanner calls. It first reads back the count as an int and then consumes the rest of the line (the “attrs” from the printf is just for human readability). It then loops getting name/value lines. The pattern used has two capture groups, the first to get the attribute name (using a non-greedy qualifier to get the fewest possible characters before the =) and the second to get the value. These groups are pulled out to create the actual Attr object.

It is no accident that these two methods are paired. Although a Scanner can read many formats of data, it is best used for data formatted in relatively straightforward ways. Stream mode is useful for reading user input as well, but when you use Scanner for data you should be able to visualize (if not actually write) the printf usage that would generate that data. Scanner is extremely powerful, but that power can be difficult to harness.

Exercise 22.10: Write a method to tokenize input that ignores comments, using the comment pattern as part of the scanner's delimiter.

Exercise 22.11: Write a version of readCSV that uses a StreamTokenizer rather than a Scanner.

Exercise 22.12: Write a version of the attribute reading method from page 533 that uses a Scanner. For this exercise it is only necessary that both versions accept the same correctly formatted input.

Exercise 22.13: Extend your solution to Exercise 22.12 so that misplaced = characters are detected, as in the StreamTokenizer version. (Hint: You might want to try to dynamically change the delimiter pattern between certain tokens.)

Localization is covered in detail in Chapter 24, but since the Scanner class has some support for localization, we cover that briefly here.

By default a scanner works in the current locale, but you can change it with the useLocale method, which takes a Locale instance. The locale being used by the scanner can be retrieved from the locale method.

The type-specific numeric scanning methods, like hasNextDouble and nextInt, are fully localized. They recognize numerical values that are formed from the numeric digit characters of the locale being used, including the locale-specific decimal separator and grouping separator. In contrast, to use a regular expression to match a local number, you would need to know what the separator characters were so that you could account for them in your pattern. Further, any potentially numeric value that is obtained as a string from a MatchResult must be separately converted into a numeric value. To localize such numeric values you must use the parsing methods of the java.text.NumberFormat class described in Chapter 24.