You can choose from a fixed list of supported compression algorithms. They have different qualities when it comes to compression ratio, as well as CPU and installation requirements.

Before looking into each available compression algorithm, refer to Table 11-1 to see the compression algorithm comparison Google published in 2005.^[118] While the numbers are old, they still can be used to compare the qualities of the algorithms.

Note that some of the algorithms have a better compression ratio while others are faster during encoding, and a lot faster during decoding. Depending on your use case, you can choose one that suits you best.

Once you have installed a supported compression algorithm, it is highly recommended that you check if the installation was successful. There are a few mechanisms in HBase to do that.

$ ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest
Usage: CompressionTest <path> none|gz|lzo|snappy

For example:
  hbase class org.apache.hadoop.hbase.util.CompressionTest file:///tmp/testfile gz

$ ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest 
  /user/larsgeorge/test.gz gz
11/07/01 20:27:43 WARN util.NativeCodeLoader: Unable to load native-hadoop  
  library for your platform... using builtin-java classes where applicable
11/07/01 20:27:43 INFO compress.CodecPool: Got brand-new compressor
11/07/01 20:27:43 INFO compress.CodecPool: Got brand-new compressor
SUCCESS

$ ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest 
  file:///tmp/test.lzo lzo
Exception in thread "main" java.lang.RuntimeException:  
 java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec
    at org.apache.hadoop.hbase.io.hfile.Compression$Algorithm$1.getCodec)
    at org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor

<property>
  <name>hbase.regionserver.codecs</name>
  <value>snappy,lzo</value>
</property>

Enabling compression requires installation of the JNI and native compression libraries (unless you only want to use the Java code-based GZIP compression), as described earlier, and specifying the chosen algorithm in the column family schema.

One way to accomplish this is during table creation. The possible values are listed in Column Families.

hbase(main):001:0> create 'testtable', { NAME => 'colfam1', COMPRESSION => 'GZ' }  
0 row(s) in 1.1920 seconds

hbase(main):012:0> describe 'testtable'                                            
DESCRIPTION                                                 ENABLED
{NAME => 'testtable', FAMILIES => [{NAME => 'colfam1',      true 
BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS 
=> '3', COMPRESSION => 'GZ', TTL => '2147483647', BLOCKSIZE
=> '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}
1 row(s) in 0.0400 seconds

The describe shell command is used to read back the schema of the newly created table. You can see the compression is set to GZIP (using the shorter GZ value as required). Another option to enable—or change, or disable—the compression algorithm is to use the alter command for existing tables:

hbase(main):013:0> create 'testtable2', 'colfam1'
0 row(s) in 1.1920 seconds

hbase(main):014:0> disable 'testtable2'
0 row(s) in 2.0650 seconds

hbase(main):016:0> alter 'testtable2', { NAME => 'colfam1', COMPRESSION => 'GZ' }
0 row(s) in 0.2190 seconds

hbase(main):017:0> enable 'testtable2'
0 row(s) in 2.0410 seconds

Note how the table was first disabled. This is necessary to perform the alteration of the column family definition. The final enable command brings the table back online.

Changing the compression format to NONE will disable the compression for the given column family.

Usually HBase handles the splitting of regions automatically: once the regions reach the configured maximum size, they are split into two halves, which then can start taking on more data and grow from there. This is the default behavior and is sufficient for the majority of use cases.

There is one known problematic scenario, though, that can cause what is called split/compaction storms: when you grow your regions roughly at the same rate, eventually they all need to be split at about the same time, causing a large spike in disk I/O because of the required compactions to rewrite the split regions.

Rather than relying on HBase to handle the splitting, you can turn it off and manually invoke the split and major_compact commands. This is accomplished by setting the hbase.hregion.max.filesize for the entire cluster, or when defining your table schema at the column family level, to a very high number. Setting it to Long.MAX_VALUE is not recommended in case the manual splits fail to run. It is better to set this value to a reasonable upper boundary, such as 100 GB (which would result in a one-hour major compaction if triggered).

The advantage of running the commands to split and compact your regions manually is that you can time-control them. Running them staggered across all regions spreads the I/O load as much as possible, avoiding any split/compaction storm. You will need to implement a client that uses the administrative API to call the split() and majorCompact() methods. Alternatively, you can use the shell to invoke the commands interactively, or script their call using cron, for instance. Also see the RegionSplitter (added in version 0.90.2), discussed shortly, for another way to split existing regions: it has a rolling split feature you can use to carefully split the existing regions while waiting long enough for the involved compactions to complete (see the -r and -o command-line options).

An additional advantage to managing the splits manually is that you have better control over which regions are available at any time. This is good in the rare case that you have to do very low-level debugging, to, for example, see why a certain region had problems. With automated splits it might happen that by the time you want to check into a specific region, it has already been replaced with two daughter regions. These regions have new names and tracing the evolution of the original region over longer periods of time makes it much more difficult to find the information you require.

Using the metrics discussed in Region Server Metrics,^[122] you can determine if you are dealing with a write pattern that is causing a specific region to run hot.

If this is the case, refer to the approaches discussed in Chapter 9, especially those discussed in Key Design: you may need to salt the keys, or use random keys to distribute the load across all servers evenly.

The only way to alleviate the situation is to manually split a hot region into one or more new regions, at exact boundaries. This will divide the region’s load over multiple region servers. As you split a region you can specify a split key, that is, the row key where you can split the given region into two. You can specify any row key within that region so that you are also able to generate halves that are completely different in size.

This might help only when you are not dealing with completely sequential key ranges, because those are always going to hit one region for a considerable amount of time.

Managing the splits is useful to tightly control when load is going to increase on your cluster. You still face the problem that when initially loading a table, you need to split the regions rather often, since you usually start out with a single region per table. Growing this single region to a very large size is not recommended; therefore, it is better to start with a larger number of regions right from the start. This is done by presplitting the regions of an existing table, or by creating a table with the required number of regions.

The createTable() method of the administrative API, as well as the shell’s create command, both take a list of split keys, which can be used to presplit a table when it is created. HBase also ships with a utility called RegionSplitter, which you can use to create a presplit table. Starting it without a parameter will show usage information:

$ ./bin/hbase org.apache.hadoop.hbase.util.RegionSplitter
usage: RegionSplitter <TABLE>
 -c <region count>        Create a new table with a pre-split number of
                                regions
 -D <property=value>      Override HBase Configuration Settings
 -f <family:family:...>   Column Families to create with new table.
                                Required with -c
 -h                             Print this usage help
 -o <count>               Max outstanding splits that have unfinished
                                major compactions
 -r                             Perform a rolling split of an existing region
    --risky                     Skip verification steps to complete
                                quickly.STRONGLY DISCOURAGED for production
                                systems.

By default, it used the MD5StringSplit class to partition the row keys into ranges. You can define your own algorithm by implementing the SplitAlgorithm interface provided, and handing it into the utility using the -D split.algorithm=<your-algorithm-class> parameter. An example of using the supplied split algorithm class and creating a presplit table is:

$ ./bin/hbase org.apache.hadoop.hbase.util.RegionSplitter 
  -c 10  testtable -f colfam1

In the web UI of the master, you can click on the link with the newly created table name to see the generated regions:

testtable,,1309766006467.c0937d09f1da31f2a6c2950537a61093.
testtable,0ccccccc,1309766006467.83a0a6a949a6150c5680f39695450d8a.
testtable,19999998,1309766006467.1eba79c27eb9d5c2f89c3571f0d87a92.
testtable,26666664,1309766006467.7882cd50eb22652849491c08a6180258.
testtable,33333330,1309766006467.cef2853e36bd250c1b9324bac03e4bc9.
testtable,3ffffffc,1309766006467.00365940761359fee14d41db6a73ffc5.
testtable,4cccccc8,1309766006467.f0c5045c304c2ff5338be27e81ae698e.
testtable,59999994,1309766006467.2d854f337aa6c09232409f0ba1d4964b.
testtable,66666660,1309766006467.b1ec9df9fd90d91f54cb18da5edc2581.
testtable,7333332c,1309766006468.42e179b78663b64401079a8601d9bd06.

Or you can use the shell’s create command:

hbase(main):001:0> create 'testtable', 'colfam1', 
  { SPLITS => ['row-100', 'row-200', 'row-300', 'row-400'] }
0 row(s) in 1.1670 seconds

This generates the following regions:

testtable,,1309768272330.37377c4ab0a944a326ba8b6596a29396.
testtable,row-100,1309768272331.e6092cc777f58a08c61bf081aba14916.
testtable,row-200,1309768272331.63c9630a79b37ebce7b58cde0235dfe5.
testtable,row-300,1309768272331.eead6ad2ff3303ffe6a3126e0df3ff7a.
testtable,row-400,1309768272331.2bee7417fa67e4ac8c7210ce7325708e.

As for the number of presplit regions to use, you can start low with 10 presplit regions per server and watch as data grows over time. It is better to err on the side of too few regions and using a rolling split later, as having too many regions is usually not ideal in regard to overall cluster performance.

Alternatively, you can determine how many presplit regions to use based on the largest store file in your region: with a growing data size, this will get larger over time, and you want the largest region to be just big enough so that is not selected for major compaction—or you might face the mentioned compaction storms.

If you presplit your regions too thin, you can increase the major compaction interval by increasing the value for the hbase.hregion.majorcompaction configuration property. If your data size grows too large, use the RegionSplitter utility to perform a network I/O safe rolling split of all regions.

Use of manual splits and presplit regions is an advanced concept that requires a lot of planning and careful monitoring. On the other hand, it can help you to avoid the compaction storms that can happen for uniform data growth, or to shed load of hot regions by splitting them manually.

HBase ships with its own tool to execute a performance evaluation. It is aptly named Performance Evaluation (PE) and its usage details can be gained from using it with no command-line parameters:

$ ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
Usage: java org.apache.hadoop.hbase.PerformanceEvaluation 
  [--miniCluster] [--nomapred] [--rows=ROWS] <command> <nclients>

Options:
 miniCluster     Run the test on an HBaseMiniCluster
 nomapred        Run multiple clients using threads (rather than use mapreduce)
 rows            Rows each client runs. Default: One million
 flushCommits    Used to determine if the test should flush the table.  
                   Default: false
 writeToWAL      Set writeToWAL on puts. Default: True

Command:
 filterScan      Run scan test using a filter to find a specific row based 
                   on it's value (make sure to use --rows=20)
 randomRead      Run random read test
 randomSeekScan  Run random seek and scan 100 test
 randomWrite     Run random write test
 scan            Run scan test (read every row)
 scanRange10     Run random seek scan with both start and stop row (max 10 rows)
 scanRange100    Run random seek scan with both start and stop row (max 100 rows)
 scanRange1000   Run random seek scan with both start and stop row (max 1000 rows)
 scanRange10000  Run random seek scan with both start and stop row (max 10000 rows)
 sequentialRead  Run sequential read test
 sequentialWrite Run sequential write test

Args:
 nclients        Integer. Required. Total number of clients (and HRegionServers)
                 running: 1 <= value <= 500
Examples:
 To run a single evaluation client:
 $ bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1

By default, the PE is executed as a MapReduce job—unless you specify for it to use 1 client, or because you used the --nomapred parameter. You can see the default values from the usage information in the preceding code sample, which are reasonable starting points, and the command to run a test is given as well:

$ ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1
11/07/03 13:18:34 INFO hbase.PerformanceEvaluation: Start class 
  org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest at 
  offset 0 for 1048576 rows
...
11/07/03 13:18:41 INFO hbase.PerformanceEvaluation: 0/104857/1048576
...
11/07/03 13:18:45 INFO hbase.PerformanceEvaluation: 0/209714/1048576
...
11/07/03 13:20:03 INFO hbase.PerformanceEvaluation: 0/1048570/1048576
11/07/03 13:20:03 INFO hbase.PerformanceEvaluation: Finished class 
  org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest 
  in 89062ms at offset 0 for 1048576 rows

The command starts a single client and performs a sequential write test. The output of the command shows the progress, until the final results are printed. You need to increase the number of clients (i.e., threads or MapReduce tasks) to a reasonable number, while making sure you are not overloading the client machine.

There is no need to specify a table name, nor a column family, as the PE code is generating its own schema: a table named TestTable with a family called info.

Using the random or sequential read and write tests allows you to emulate these specific workloads. You cannot mix them, though, which means you must execute each test separately.

The Yahoo! Cloud Serving Benchmark^[124] (YCSB) is a suite of tools that can be used to run comparable workloads against different storage systems. While primarily built to compare various systems, it is also a reasonable tool for performing an HBase cluster burn-in—or performance test.

$ git clone http://github.com/brianfrankcooper/YCSB.git
Initialized empty Git repository in /private/tmp/YCSB/.git/
...
Resolving deltas: 100% (475/475), done.

$ cd YCSB/
$ cp $HBASE_HOME/hbase*.jar db/hbase/lib/
$ cp $HBASE_HOME/lib/*.jar db/hbase/lib/
$ ant
Buildfile: /private/tmp/YCSB/build.xml
...
makejar:
      [jar] Building jar: /private/tmp/YCSB/build/ycsb.jar

BUILD SUCCESSFUL
Total time: 1 second

$ ant dbcompile-hbase
...
BUILD SUCCESSFUL
Total time: 1 second

Before you can use YCSB you need to create the required test table, named usertable. While the name of the table is hardcoded, you are free to create a column family with a name of your choice. For example:

$ ./bin/hbase shell

hbase(main):001:0> create 'usertable', 'family'
0 row(s) in 0.3420 seconds

Starting YCSB without any options gives you its usage information:

$ java -cp build/ycsb.jar:db/hbase/lib/* com.yahoo.ycsb.Client
Usage: java com.yahoo.ycsb.Client [options]
Options:
  -threads n: execute using n threads (default: 1) - can also be specified as the 
              "threadcount" property using -p
  -target n: attempt to do n operations per second (default: unlimited) - can also
             be specified as the "target" property using -p
  -load:  run the loading phase of the workload
  -t:  run the transactions phase of the workload (default)
  -db dbname: specify the name of the DB to use (default: com.yahoo.ycsb.BasicDB) - 
              can also be specified as the "db" property using -p
  -P propertyfile: load properties from the given file. Multiple files can
                   be specified, and will be processed in the order specified
  -p name=value:  specify a property to be passed to the DB and workloads;
                  multiple properties can be specified, and override any
                  values in the propertyfile
  -s:  show status during run (default: no status)
  -l label:  use label for status (e.g. to label one experiment out of a whole 
             batch)

Required properties:
  workload: the name of the workload class to use 
            (e.g. com.yahoo.ycsb.workloads.CoreWorkload)

To run the transaction phase from multiple servers, start a separate client
on each. To run the load phase from multiple servers, start a separate client 
on each; additionally, use the "insertcount" and "insertstart" properties to 
divide up the records to be inserted

The first step to test a running HBase cluster is to load it with a number of rows, which are subsequently used to modify the same rows, or to add new rows to the existing table:

$ java -cp $HBASE_HOME/conf:build/ycsb.jar:db/hbase/lib/* 
com.yahoo.ycsb.Client -load -db com.yahoo.ycsb.db.HBaseClient 
-P workloads/workloada -p columnfamily=family -p recordcount=100000000 
-s > ycsb-load.log

This will run for a while and create the rows. The layout of the row is controlled by the given workload file, here workloada, containing these settings:

$ cat workloads/workloada
# Yahoo! Cloud System Benchmark
# Workload A: Update heavy workload
#   Application example: Session store recording recent actions
#                        
#   Read/update ratio: 50/50
#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)
#   Request distribution: zipfian

recordcount=1000
operationcount=1000
workload=com.yahoo.ycsb.workloads.CoreWorkload

readallfields=true

readproportion=0.5
updateproportion=0.5
scanproportion=0
insertproportion=0

requestdistribution=zipfian

Refer to the online documentation of the YCSB project for details on how to modify, or set up your own, workloads. The description specifies the data size and number of columns that are created during the load phase. The output of the tool is redirected into a logfile, which will contain lines like these:

YCSB Client 0.1
Command line: -load -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada 
-p columnfamily=family -p recordcount=100000000 -s
[OVERALL], RunTime(ms), 915.0
[OVERALL], Throughput(ops/sec), 1092.896174863388
[INSERT], Operations, 1000
[INSERT], AverageLatency(ms), 0.457
[INSERT], MinLatency(ms), 0
[INSERT], MaxLatency(ms), 314
[INSERT], 95thPercentileLatency(ms), 1
[INSERT], 99thPercentileLatency(ms), 1
[INSERT], Return=0, 1000
[INSERT], 0, 856
[INSERT], 1, 143
[INSERT], 2, 0
[INSERT], 3, 0
[INSERT], 4, 0
...

This is useful to keep, as it states the observed write performance for the initial set of rows. The default record count of 1000 was increased to reflect a more real-world number. You can override any of the workload configuration options on the command line. If you are running the same workloads more often, create your own and refer to it on the command line using the -P parameter.

The second step for a YCSB performance test is to execute the workload on the prepared table. For example:

$ java -cp $HBASE_HOME:build/ycsb.jar:db/hbase/lib/* 
com.yahoo.ycsb.Client -t -db com.yahoo.ycsb.db.HBaseClient 
-P workloads/workloada -p columnfamily=family -p operationcount=1000000 -s 
-threads 10 > ycsb-test.log

As with the loading step shown earlier, you need to override a few values to make this test useful: increase (or use your own modified workload file) the number of operations to test, and set the number of concurrent threads that should perform them to something reasonable. If you use too many threads you may overload the test machine (the one you run YCSB on). In this case, it is more useful to run the same test at the same time from different physical machines.

The output is also redirected into a logfile so that you can evaluate the test run afterward. The output will contain lines like these:

]$ cat transactions.dat 
YCSB Client 0.1
Command line: -t -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p 
columnfamily=family -p operationcount=1000 -s -threads 10
[OVERALL], RunTime(ms), 575.0
[OVERALL], Throughput(ops/sec), 1739.1304347826087
[UPDATE], Operations, 507
[UPDATE], AverageLatency(ms), 2.546351084812623
[UPDATE], MinLatency(ms), 0
[UPDATE], MaxLatency(ms), 414
[UPDATE], 95thPercentileLatency(ms), 1
[UPDATE], 99thPercentileLatency(ms), 1
[UPDATE], Return=0, 507
[UPDATE], 0, 455
[UPDATE], 1, 49
[UPDATE], 2, 0
[UPDATE], 3, 0
...
[UPDATE], 997, 0
[UPDATE], 998, 0
[UPDATE], 999, 0
[UPDATE], >1000, 0
[READ], Operations, 493
[READ], AverageLatency(ms), 7.711967545638945
[READ], MinLatency(ms), 0
[READ], MaxLatency(ms), 417
[READ], 95thPercentileLatency(ms), 3
[READ], 99thPercentileLatency(ms), 416
[READ], Return=0, 493
[READ], 0, 1
[READ], 1, 165
[READ], 2, 257
[READ], 3, 48
[READ], 4, 11
[READ], 5, 4
[READ], 6, 0
...
[READ], 998, 0
[READ], 999, 0
[READ], >1000, 0

Note that YCSB can hardly emulate the workload you will see in your use case, but it can still be useful to test a varying set of loads on your cluster. Use the supplied workloads, or create your own, to emulate cases that are bound to read, write, or both kinds of operations.

Also consider running YCSB while you are running batch jobs, such as a MapReduce process that scans subsets, or entire tables. This will allow you to measure the impact of either on the other.