The Flume installation is straightforward and has similar prerequisites to previous tools we have installed.

Firstly, we retrieved the latest version of Flume NG (any version of 1.2.x or later will do) and saved it to the local filesystem. We moved it to the desired location, uncompressed it, and created a convenience symlink to the location.

We needed to define the FLUME_HOME environment variable and add the bin directory within the installation directory to our classpath. As before, this can be done directly on the command line or within convenience scripts.

Flume requires JAVA_HOME to be defined and we confirmed this is the case. It also requires Hadoop libraries, so we checked that the Hadoop classes are in the classpath.

The last steps are not strictly necessary for demonstration though will be used in production. Flume looks for a configuration directory within which are files defining the default logging properties and environment setup variables (such as JAVA_HOME). We find Flume performs most predictably when this directory is properly set up, so we did this now and don't need to change it much later.

We assumed /home/hadoop/flume is the working directory within which the Flume configuration and other files will be stored; change this based on what's appropriate for your system.

Now that we have Flume installed, let's use it to capture some network data.