Using concat()

The challenge with the data event is that the chunk of data that it gives to our event handler isn't something that's parsable. For example, it could leave off half of a word. This is no big deal—we'll just have to account for this:

let myWordList = List();
let last = '';

console.time('elapsed');
const wordInput = fs.createReadStream('./input/words');

wordInput.on('data', (data) => {
const words = (last + data.toString()).split(os.EOL);
last = words[words.length - 1];

myWordList = myWordList.concat(words.slice(0, words.length - 1));
});

wordInput.on('end', () => {
console.log('word count', myWordList.count().toLocaleString());
console.timeEnd('elapsed');
// -> word count 235,886
// -> elapsed: 2012.116ms
});

The data that's passed to the handler function as an argument is just a small chunk of the overall stream. So, since we don't know where this chunk cuts off, we have to assume that the last word has missing characters. We store the last word so that we can join it with the next chunk. Then we concatenate the words with myWordList. Based on the timer we've set up, this takes about 2 seconds to complete. Can we do better than this?

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.222.111.24