The trim and strip functions are mainly used to clean up strings that may be padded at the start or the end, either with whitespace or other characters.

trim is the simplest operation, removing whitespace from the beginning and end of the input text, so that you're left with the content only. The returned string is a slice of your original, so you will need to duplicate it if you want to manipulate it further.

strip is a generalized version of trim, where you can pass it any character that should be removed from either end of the input string. As with trim, a clean slice of the original argument is returned.

Several functions let you edit the content itself. replace and substitute both replace occurrences of a specified pattern with a different one. replace replaces single characters with a new character. substitute replaces a substring of the input with a new string. Both of these operations do the replacement in place to avoid allocations. If you want to keep the original, duplicate it before passing it to either of these functions, as in the following example:

char[] original = "A string to have some letters replaced";
auto result = replace (original.dup, 'r', 'l'),

The end result of this example is the string "A stling to have some lettels leplaced."

The utilities include two simple operations to build strings: join and repeat. join concatenates a list of strings into a single big one. An optional postfix can be passed to the function that will be appended to each of the joined strings except the last. If you want to control the output buffer, you can pass your own to the function. repeat builds a string by repeating a pattern n times. This function also accepts an optional output buffer.

The next set of operations provides querying capabilities to help you find the location of certain substrings or to determine whether they are present.

contains and containsPattern check an input string for an embedded character or string, respectively. They both will return true if a match is found.

locate, locatePrior, locatePattern, and locatePatternPrior operate in a similar manner, except that they return the position in the input string of the first match. The first two look for a character. locate searches from the start of the string or starts at the optional start parameter. locatePrior starts at the end going toward the beginning or starts at the optional starting position. locatePattern and locatePatternPrior do the same, but they look for substrings (patterns) instead. In the case that no match is found, the length of the source string is returned instead. Here is an example:

locatePatternPrior ("ababababaaaab", "aba", 8);

This example returns 6 as the first position where the pattern aba starts, going backward from index 8.

If you need to check if a given character is whitespace, pass it to isSpace, which will return true if it is.

matching, indexOf, and mismatch provide highly efficient routines that you can use for testing certain aspects of your strings. matching returns true if both of the strings are equal up to the specified length. indexOf returns the index of the first match up to the specified length. In the case where no match is found, this length is returned. The returned index itself is zero-based. mismatch does the opposite, returning the first index where the strings no longer match. If the strings are actually matching, the length is returned instead.

In the cases where you need to separate strings on a given pattern, several approaches are available through Tango. The first approach uses the operation delimit, split, or splitLines, returning an array of strings. delimit takes an array of delimiting characters, each of which results in a new string being put into the result array where one of those delimiters is found in the source string. split does the same thing, except that it looks for one given pattern. splitLines returns an array of distinct lines (as given by the presence of or in the source string). All of these functions remove from the resulting arrays the characters or patterns used.

If you prefer to iterate over the resulting components instead of receiving an array of them, tango.text.Util provides some efficient alternatives, using slices to make sure no allocations are made during the split operation itself. lines, delimiters, patterns, and quotes are all entities that can be used in foreach. Here is an example:

foreach (segment; patterns("one, two, three", ", ")) {
    . . .
}

This example will loop on each segment found in the string passedin this case, one, two, and three.

If you would rather replace the delimiting pattern with a new one, the new pattern can be passed as an additional string to patterns. lines will let you iterate over the lines in your string, similar to splitLines. delimiters is the iterator version of delimit, whereas quotes will ignore delimiters found inside a pair of quotation marks, such as in the following example:

foreach (segment, quotes ("one two 'three four' five", " ")) {
    . . .
}

The results of this example are the segments one, two, three four, and five.

In Tango, you will also find proper support for splitting operations over streams, whether you stream data from a file, a socket, a pipe, or any other class implementing the InputStream interface (see Chapter 7). Four stream iterators are available, all based on the StreamIterator superclass, and you can further extend this number by creating other subclasses. The iterators can all be found in the tango.text.stream package.

The following example demonstrates iterating over the elements of a stream:

import tango.io.FileConduit,
       tango.io.Console;
import tango.text.stream.LineIterator;

foreach (line; new LineIterator!(char) ( new FileConduit ("filename") ))
    Cout (line).newline;

This example uses the LineIterator on a stream from a file. The iterators are templated for each of the three character types in D. In this example, char is used. Each line is simply output to the console.

The other iterators are SimpleIterator, which takes a delimiter parameter in addition to the stream; QuoteIterator, which ignores delimiters within quotation marks; and RegexIterator, which lets you specify a regular expression to use as the delimiting pattern.

The methods in the read-only view of Text mainly let you query the text for various information and compare it to other instances, whether they are of the TextView type or a D string type.

The length of the text, a hash of it, and the encoding (represented as a TypeInfo instance) are available through the properties length, toHash, and encoding.

Both opCmp and opEquals are overloaded, as is the duplicate functionality also available through the methods compare and equals. In addition, you can check whether your text starts with or ends with a given substring.

The other methods of the read-only view are slice, copy, and comparator. If you use slice, you get the underlying array as a slice. The array is not in itself safe, as there is no way in D 1.0 to restrict its use, and so you are expected to respect that it should not be changed through this interface.

With comparator, you can set the algorithms that are used in different comparison methods, and copy accepts a target array into which you can copy the text's content.

After instantiating Text, either with or without content, the content can be set/reset using two set methods. One takes a D string of the type that the instance was created with, and the other takes a TextView instance. Both have an optional parameter that can be set to false if the instance is not intended for modification of the contents.

Almost all of the methods in Text return an instance of the enclosing class, making it possible to chain calls, as in this example:

auto text = new Text!(char)("The couple danced the rumba");
text.select("couple");
text.replace("party").prepend("large ").select("rumba");
text.replace("tango");

This sequence results in the content "The large party danced the tango."

For additional power, Text can be combined with the utilities in tango.text.Util, as in the following example, where the line iterator is used to iterate over lines in a Text instance.

auto source = new Text!(char)("one
two
three");

foreach (line; Util.lines(source.slice)) {
    . . .
}

Similarly, all instances of a word in a block of text can be replaced with another:

auto dst = new Text!(char);
foreach (element; Util.patterns ("all cows eat grass", "eat", "chew")) {
    dst.append (element);
}

Following the Tango convention, format requires that you pass along an output buffer for the result. Here is an example where the number 15 is formatted:

char[10] output;
auto result = format (output, 15);

In this example, the formatted number is placed into output, which, in this case, is allocated on the stack. The result returned is a slice of output containing the exact string. Many of the format functions provide additional parameters so you can better control the formatted output.

Creating a value in a numeric type based on a textual representation is the opposite operation to formatting. parse supports additional parameters to control the output, such as to say something about the radix used.

The Integer module lets you convert to or from the types short, ushort, int, uint, long, and ulong. In this module, parse can take three arguments:

In the same vein, format has two additional parameters with default values. The first two are always the output buffer and the number to be formatted. The third is a style identifier that says how the number is to be represented in the resulting text. You can choose from the values shown in Table 6-1.

In addition to specifying how the number should be formatted, you can pass along a style flag, as shown in Table 6-2.

The following example shows how you can format a number into hexadecimals including a prefix:

auto text = Integer.format (new char[32], 12345L, Style.HexUpper, Flags.Prefix);

Along with parse and format, the Integer module contains some convenience functions. toString, toString16, and toString32 will format a number for you, allocating the necessary storage as it goes along. toInt and toLong will parse a string, assuming that it is fully parsable.

You can also use convert, which does not look for a radix in the input text. The trim function will extract optional signs and the radix, while removing extraneous space at the start, leaving the digits ready for parsing.

For the various floating-point types in Dfloat, double, and realyou should use tango.text.convert.Float to convert between them and their text representations. This module is somewhat simpler to use than Integer, as there are fewer variations in how the results can be formatted.

parse normally takes only the string representing the number, but can take an additional pointer to a uint, which will represent how much of the string was processed (or eaten) to create the resulting numeric type.

When formatting a floating-point number using format, you can customize it using two additional parameters. The first parameter after the number to be formatted is the number of decimals to be used. The default is 6. The second parameter is an int indicating how many exponent places should be emitted, effectively saying at which point a number should start being formatted in scientific notation. You can use 0 for always and 2 for numbers larger than 100 or smaller than 0.01. The default is 10. Here is an example of formatting a floating-point number:

auto text = Float.format (new char[64], 223.1456667, 5, 2);

This will convert the number into a string using five decimal places and scientific notation: 2.23145e+02. The result will be a slice from the buffer, which is created on the heap in this example. Also in this example, the renaming of imports is used to create the Float namespace.

As with Integer, Float has wrappers for the most common setup, negating the need to preallocate a buffer for the output. toString, toString16, and toString32 wrap format, whereas toDouble wraps the parsing, requiring that the full string is parsable as a number.

Strings representing points in time can also be converted into a numeric value, a so-called timestamp, by importing tango.text.convert.TimeStamp. The resulting value is usually how many units of time have passed since a particular point, called an epoch. The most common type of timestamp in modern computing has been the number of milliseconds since 1970.

You can pass a string to parse in one of the formats specified in Table 6-3, getting a timestamp in return. The one additional optional parameter is a pointer to a uint saying how much of the string was parsed to create the timestamp. If the parsing fails, the predefined value InvalidEpoch will be returned instead.

Formatting a timestamp using format will yield a string in the RFC 1123 format. toString, toString16, and toString32 wrap this for the cases where you don't want to or don't need to preallocate a buffer for the output. The following example shows a string that is parsed before the resulting value is formatted.

auto date = "Sun, 06 Nov 1994 08:49:37 GMT";
auto msSinceJan1st0001 = TimeStamp.parse (date);
auto text = TimeStamp.format (new char[64], msSinceJan1st0001);

dostime and iso8601 can be used to convert from the DOS time and ISO-8601 formats, respectively. rfc1123, rfc850, and asctime do the work for parse, and can be used directly if you are sure of the format of the timestamp's textual representation.

If you are going to print a formatted string to the console, you will typically do so using Stdout, as in the following example:

import tango.io.Stdout;
Stdout.format ("Printing the value {} to the {}", 5, "console").newline;

Here, the first string passed to format is the format string. It describes how you want your output to look, with the braces ({}) as placeholders for dynamic content. In this example, the first pair of braces will be substituted with 5 and the second with console. Typically, you will want to specify your output in more detail and also say something about where in the template the arguments should be put. The newline at the end emits a newline and flushes Stdout so that the message is seen on the console immediately.

A number within the braces functions as a zero-based index into format's argument list. Thus, the previous format call is equivalent to this:

Stdout.format ("Printing the value {0} to the {1}", 5, "console").newline;

Since the order for the example already is implicit, there is not much point to using this technique here. The line can be rewritten to the following:

Stdout.format ("Printing the value {1} to the {0}", "console", 5).newline;

The output will be the same, but now the second argument after the format string is used first. This becomes particularly useful when internationalizing your application. By passing format strings from different languages, and having a fixed order of the arguments, you can use the indices to make the words of the different languages be printed in a sane and correct order. The following example shows how this is done with an English phrase, where the second form tends to be more common in poetry, "I can see Bill" versus "Bill I can see."

char[] s = "I";  // subject
char[] o = "Bill"; // object
Stdout.format ("{0} can see {1}.", s, o).newline;
Stdout.format ("{1} {0} can see.", s, o).newline;

An index can also be reused several times so that a particular value can be repeated, as follows:

Stdout.format ("Printing the value {0} and then the same again {0}", 5);

A pair of braces is called a format item, and can contain up to three components, all of which are optional. The first one is the index (as shown in the previous example), the second is an alignment component, and the third is a format string component. Let's take a closer look at the latter two components.

char[] myFName = "Johnny";
Stdout.formatln("First Name = |{0,15}|", myFName);
Stdout.formatln("Last Name = |{0,15}|", "Foo de Bar");

Stdout.formatln("First Name = |{0,-15}|", myFName);
Stdout.formatln("Last Name = |{0,-15}|", "Foo de Bar");

Stdout.formatln("First name = |{0,5}|", myFName);

First Name = |         Johnny|
Last Name = |     Foo de Bar|
First Name = |Johnny         |
Last Name = |Foo de Bar     |
First name = |Johnny|

Stdout.formatln ("I have {:G} birds on the roof", 100);

Stdout.formatln ("A hexadecimal number follows: {0:X9}", 0xafe0000);

When you are formatting using Tango, tango.text.convert.Layout does most of the work in its convert method, or sprint when you want to format to a preallocated buffer. For sprint, the first argument should be the output, and the format string should be the second buffer. For convert, the format string is first. The arguments following the format string are those that will be formatted and substituted into the template string.

If you need to format a string, but don't want to print it to the console, you can instantiate Layout directly. convert is aliased to Layout's opCall, thus you can format as in this example:

import tango.text.convert.Layout;
auto layout = new Layout!(char); // Need to specify encoding you are going to use
auto result = layout("A format {}", "string");

If you want to format text in a different encoding, you can use the fromUtf8, fromUtf16, or fromUtf32 method.

The locale support works by subclassing Layout, hooking into the various format string components, and specifying a lot more functionality. To use Locale, you just need to instantiate it instead of Layout.

import tango.text.locale.Locale;
auto locale = new Locale;

Note that Locale has UTF-8 set as its default encoding. To enable localized output using Stdout, set it as the layout engine using Stdout's layout property, as follows:

Stdout.layout = new Locale;

If nothing more is specified, Locale will try to look up the locale settings of the user's computer and format according to what it finds. If you want to specify a specific locale setting, do this via Locale's constructor, as follows:

auto locale = new Locale(Culture.getCulture("fr_FR"));

The format string components that are understood by Locale when given a numeric value are shown in Table 6-5.

All the format string components support an additional number to specify the precision of the formatted number.

The other kind of values localized by Locale are tango.time.Time.Time objects. Table 6-6 shows which format string components are available as shortcuts when a Time instance is passed to the formatter. These shortcuts are substituted with a pattern in the formatter.

If the format string component for a Time instance is more than one character, it means that it is a custom format, which may include the components shown in Table 6-7. The patterns shown in Table 6-6 are predefined examples of such patterns. Elements in these patterns that are enclosed in single quotation marks are printed as is. This is mainly used to escape parts of the pattern that otherwise may be interpreted in some capacity by the formatter.

The following is an example of printing a Time instance to the console in a customized format, using direct console output via tango.io.Console.Cout.

import tango.io.Console;
import tango.time.WallClock;
import tango.text.locale.Locale;

auto layout = new Locale;
Cout (layout ("{:ddd, dd MMMM yyyy HH':'mm':'ss z}", WallClock.now)).newline;

You should by now have a good understanding of the basic and intermediate routines present in Tango's text-processing functionality. In the next chapter, you'll learn about Tango's input/output packages.