Streaming File Writing

One of the best methods to use when writing large amounts of data to a file is streaming, which involves opening a file as a Writable stream. As discussed in Chapter 5, “Handling Data I/O in Node.js,” Writable streams can easily be implemented and linked to Readable streams, using the pipe() method; this makes it very easy to write data from a Readable stream source such as an HTTP request.

To stream data to a file asynchronously, you first need to create a Writable stream object, using the following syntax:

Click here to view code image

fs.createWriteStream(path, [options])

The path parameter specifies the path to the file, which can be relative or absolute. The optional options parameter is an object that can contain encoding, mode, and flag properties that define the string encoding as well as the mode and flags used when opening the file.

Once you have opened the Writable file stream, you can write to it using the standard stream write(buffer) methods. When you are finished writing, call the end() method to close the stream.

The code in Listing 6.4 shows how to implement a basic Writable file stream. Notice that when the code is finished writing, the end() method is executed on line 13, which triggers the close event. Figure 6.4 shows the output of the code in Listing 6.4.

Listing 6.4 file_write_stream.js: Implementing a Writable stream to allow streaming writes to a file

Click here to view code image

01 var fs = require('fs'),
02 var grains = ['wheat', 'rice', 'oats'];
03 var options = { encoding: 'utf8', flag: 'w' };
04 var flieWriteStream = fs.createWriteStream("../data/grains.txt", options);
05 fileWriteStream.on("close", function(){
06   console.log("File Closed.");
07 });
08 while (grains.length){
09   var data = grains.pop() + " ";
10   fileWriteStream.write(data);
11   console.log("Wrote: %s", data);
12 }
13 fileWriteStream.end();

Figure 6.4 Output of file_write_stream.js, implementing streaming writes to a file.