Solution based on MappedByteBuffer

The last solution that we'll talk about here is based on Java NIO.2, MappedByteBuffer, and FileChannel. This solution opens a memory-mapped byte buffer (MappedByteBuffer) from a FileChannel on the given file. We traverse the fetched byte buffer and look for matches with the searched string (this string is converted into a byte[] and searching take place byte by byte).

For small files, it is faster to load the entire file into memory. For large/huge files, it is faster to load and process the files in chunks (for example, a chunk of 5 MB). Once we have loaded a chunk, we have to count the number of occurrences of the searched string. We store the result and pass it to the next chunk of data. We repeat this until the entire file has been traversed.

Let's take a look at the core lines of this implementation (take a look at the source code bundled with this book for the complete code):

private static final int MAP_SIZE = 5242880; // 5 MB in bytes

public static int countOccurrences(Path path, String text)
throws IOException {

final byte[] texttofind = text.getBytes(StandardCharsets.UTF_8);
int count = 0;

try (FileChannel fileChannel = FileChannel.open(path,
StandardOpenOption.READ)) {
int position = 0;
long length = fileChannel.size();

while (position < length) {
long remaining = length - position;
int bytestomap = (int) Math.min(MAP_SIZE, remaining);

MappedByteBuffer mbBuffer = fileChannel.map(
MapMode.READ_ONLY, position, bytestomap);

int limit = mbBuffer.limit();
int lastSpace = -1;
int firstChar = -1;

while (mbBuffer.hasRemaining()) {
// spaghetti code omitted for brevity
...
}
}
}

return count;
}

This solution is extremely fast because the file is read directly from the operating system's memory without having to be loaded into the JVM. The operations take place at the native level, called the operating system level. Note that this implementation works only for the UTF-8 charset, but it can be adapted for other charsets as well.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
13.59.173.242