Understanding the Readable stream interface

The Readable stream has a few properties that we have available to us. All of them are explained in the Node.js documentation, but the main ones that we are interested in are highWaterMark and objectMode.

highWaterMark allows us to state how much data the internal buffer should hold before the stream will state that it can no longer take any data. One problem with our implementation is that we do not handle pauses. A stream can pause if this highWaterMark is reached. While we may not be worried about it most of the time, it can cause problems and is usually where stream implementors will run into issues. By setting a higher highWaterMark , we can prevent these problems. Another way of handling this would be to check the outcome of running this.push. If it comes back true , then we are able to write more data to the stream, otherwise, we should pause the stream and then resume when we get the signal from the other stream. The default highWaterMark for streams is around 16 KB.

objectMode allows us to build streams that are not Buffer based. This is great when we want to run through a list of objects. Instead of utilizing a for loop or even a map function, we could set up a piping system that moves objects through the stream and performs some type of operation on it. We are also not limited to plain old objects, but to almost any data type other than the Buffer. One thing to note about objectMode is it changes what the highWaterMark counts. Instead of it counting how much data to store in the internal buffer, it counts the number of objects that it will store until it pauses the stream. This defaults to 16, but we can always change it if need be.

With these two properties explained, we should discuss the various internal methods that are available to us. For each stream type, there is a method that we need to implement and there are methods that we can implement.

For the Readable stream, we only need to implement the _read method. This method gives us a size argument, which is the number of bytes to read from our underlying data source. We do not always need to heed this number, but it is available to us if we want to implement our stream with it.

Other than the _read method, we need to utilize the push method. This is what pushes data onto our internal buffer and helps emit the data event as we have seen previously. As we stated before, the push method returns a Boolean. If this value is true, we can continue using push, otherwise, we should stop pushing data until our _read implementation gets called again.

As stated previously, when first implementing a Readable stream, the return value can be ignored. However, if we notice that data is not flowing or data is getting lost, the usual culprit is that the push method returned false and we continued to try pushing data on our stream. Once this happens, we should implement pausing by stopping the use of the push method until _read gets called again.

Two other pieces of the readable interface are the _destroy method and how to make our stream error out if something comes through that we are not able to handle. The _destroy method should be implemented if there are any low-level resources that we need to let go of.

This can be file handles opened with the fs.open command or sockets created with the net module. If an error occurred, we should also use that so we can emit an error event.

To handle errors that may come up with our streams, we should emit our error through the this.emit system. If we throw an error, as per the documentation, it can lead to unexpected results. By emitting an error, we let the user of our stream deal with the error and handle it as they see fit.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
13.58.220.83