Storage

In Part I, we saw how HBase stores the data into HFile. Each table is split into regions, each region will be a directory on HDFS, and each column family is a subdirectory of the region directory. For the table we’re looking at here, after everything has been flushed from memory, if we consider we did not use any specific namespace, the following directory structure will be created on HDFS:

├── data
│   ├── default
│   │   └── mob_test
│   │       └── 4ef02a664043021a511f4b4585b8cba1
│   │           ├── f
│   │           │   └── adb1f3f22fa34dfd9b04aa273254c773
│   │           └── recovered.edits
│   │               └── 2.seqid

This looks exactly like what we had for any other regular table. However, we now also have a MOB directory created under the HBase folder:

├── mobdir
│   └── data
│       └── default
│           └── mob_test
│               └── 0660f699de69e8d4c52e4a037e00a732
│                   └── f
│                       └── d41d8cd98f00b204e9800998ecf8427e20160328fa75df3f...0

As you can see here, we still have our regular HFile that contains the small cells that are not filtered as MOB, but we also have an additional file that contains the MOB values. This new file will not be handled the same way as the regular files. Indeed, when HBase has to compact files together, it has to read the entire HFiles and create a new one containing the merged content. The goal of the compaction is to reduce the number of small files and create bigger ones. Because MOB cells are already big, storing them in separate will reduce the read and write amplification caused by the compactions. Only references to those MOB cells will be read and rewritten each time there is a compaction.

MOB files still have to be compacted. You might have deleted a value that was stored as a MOB. There is a shell command for that, but HBase will also automatically trigger those compactions when required.

If most of what you are doing with HBase is writes of big cells and almost no deletes, MOB will give you a very good improvement compared to the regular format.

Usage

At this point, you might be wondering what constitutes a big cell versus a small one. HBase performs very well with cells up to 1 MB. It is also able to process cells up to 5 MB. However, above that point, regular write path will start to have some performance challenges. The other thing you have to consider is how many times you are going to write big cells versus regular ones. MOBs required some extra work for HBase. When you are reading a cell, we first need to lookup into the related HFile and then into the MOB file. Because of this extra call, latency will be slightly impacted. Now, if your MOB is 50 MB, the time to transfer it to the client versus the extra read access on HBase might not be significant. The extra read access on the HBase side will add few milliseconds to your call. But transferring 50 MB of data to your client might take more than that and might just hide the extra milliseconds for the additional read.

The main question you will need to ask yourself is how often are you going to insert big cells into HBase. If you expect to do so only, once in a while, and not more than once per row, a cell that might be a big bigger than the others, like up to 5 MB, you might just be able to use the regular put API. That will save you the extra management of the MOB files and HBase will be able to handle that. Because there will be only a few big cells, the read and write amplification will be limited. However, if you are planning to have multiple big cells per row, if some of your cells are going to be bigger than 5 MB, or if most of your puts are big, you might want to consider using the MOBs.

Too Big

As we have just seen, it’s fine for regular HBase cells to occasionally grow up to 5 MB cells. MOBs will allow you to easily handle cells up 20 MB. Even if it is not necessarily recommended, some people have pushed it a bit further and are using MOBs with 50 MB cells.

Now, if your use case is to store only very big cells, like 100 MB cells or more, you are better off using a different approach.

There are two best practices around this. First, if your cells are bigger than 100 MB (and you know for sure that they will continue to be over that limit), a better approach is to store the content as a file on HDFS and store the reference of this file into HBase. Keep in mind that it’s inadvisable to store millions of files in the same directory, so you should devise a directory structure based on your use case (e.g., you might create a new folder every month or every day, or for every source).

The second approach is the one described in the previous chapter. If only some of your cells will be very big, but some will be smaller, and you don’t know in advance the distribution of the size, think about splitting your cell into multiple smaller cells. Then when writing the data, check its size and split it if required, and when reading it, perform the opposite operation and merge it back.

The best design to do this is to abstract that from the client application. Indeed, if the client has to split the content itself, it will add complexity and might hide some business logic. However, if you extend the existing HBase API to parse the Puts before they are sent to HBase, you can silently and easily perform the split operations. The same thing is true for the Get where you can, again, parse the content retrieved from HBase and merge it back to one single object.

Example 13-1 gives you an example of how to perform that.

To keep things simple and illustrate how it should work, we are not extending the client API.

  public static void putBigCell (Table table, byte[] key, byte[] value)
               throws IOException {
    if (value.length < 10 * MB) {
      // The value is small enough to be handled as a single HBase cell.
      // There is no need to split it in pieces.
      Put put = new Put(key).addColumn(columnFamily, columnQualifierOnePiece, value);
      table.put(put);
    } else {
      byte[] buffer = new byte[10*MB];
      int index = 0;
      int piece = 1;
      Put put = new Put(key);
      while (index < value.length) {
        int length = Math.min((value.length - index), buffer.length);
        byte[] columnQualifier = Bytes.toBytes(piece);
        System.arraycopy(value, index, buffer, 0, length); 
        KeyValue kv = new KeyValue(key, 0, key.length, 
                                   columnFamily, 0, columnFamily.length,
                                   columnQualifier, 0, columnQualifier.length,
                                   put.getTimeStamp(), KeyValue.Type.Put,
                                   buffer, 0, length);
        put.add(kv);
        index += length + 1;
        piece ++;
      }
      table.put(put);
    }
  }

We do not want to create a new 10 MB object each time we read a new slice, so we keep reusing the same buffer array.

The HBase Put object doesn’t allow you to give a byte array or to specify the length of your value, so we have to make use of the KeyValue object.