APPENDIX
Installation and Setup Instructions
WHAT’S IN THIS APPENDIX?
- Installing and setting up a few popular NoSQL products
- Understanding the minor installation differences between platforms
- Compiling products from source, where applicable
- Configuring some of the NoSQL products
Software installation and setup instructions usually vary depending on the underlying operating system. Most instructions work on Linux, Unix, and Mac OS X. Instructions specific to installation and setup on Windows OS are included in a few select places. The installation instructions are not exhaustive.
Note that these instructions frequently refer to installing different software components in the /opt directory. By default, this directory is not usually writable by users other than root. If the directions refer to extracting archives or other operations in the /opt directory and it is not writable by the user extracting the archive, run the commands under the sudo(8) command or use the chmod(1) command to make the /opt directory writable.
INSTALLING AND SETTING UP HADOOP
This section contains the instructions for installing and configuring Hadoop Common, Hadoop Distributed File System (HDFS), and Hadoop MapReduce.
The following software is required:
- Java 1.6.x — Hadoop is tested against the Sun (now Oracle) JDK.
- SSH and sshd — SSH must be installed and sshd must be running. Hadoop scripts use sshd to connect to remote Hadoop daemons.
Hadoop can be installed and run as a single-node or as a multi-node cluster. A single-node installation can also be configured to run in a pseudo-distributed mode. In this section, the focus is on Hadoop setup for a single-node, with configuration for pseudo-distributed mode. Cluster setup and configuration is not covered here but references to external documentation on the subject are provided.
Installing Hadoop
To install Hadoop, follow these steps:
1. Download a stable Hadoop distribution release from http://hadoop.apache.org/common/releases.html. Currently, the latest release is version 0.21.0 but go in favor of version 0.20.2. Using Hadoop 0.20.2 avoids a set of inconsistencies that version 0.21.0 imposes, especially when used with HBase.
2. Unpack, untar, and unzip the downloaded distribution.
3. Move the unpacked distribution to a desired place in the filesystem. I prefer moving the unpacked distribution to /opt.
4. (Optional) Create a symbolic link named hadoop to point to the unpacked distribution folder. A symbolic link can be created as follows: ln -s hadoop-0.20.2 hadoop. This command assumes you are in the directory in which you’ve extracted the archive.
After Hadoop is installed, follow these essential configuration steps:
1. Edit conf/hadoop-env.sh. Set JAVA_HOME to the relevant JDK. On Ubuntu, with OpenJDK, JAVA_HOME could be /usr/lib/jvm/java-1.6.0-openjdk. On Mac OS X JAVA_HOME is most likely /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home.
2. Run bin/hadoop. All is good, if you see an output like so:
Usage: hadoop [--config confdir] COMMAND where COMMAND is one of: namenode -format format the DFS filesystem secondarynamenoderun the DFS secondary namenode namenode run the DFS namenode datanode run a DFS datanode dfsadmin run a DFS admin client mradmin run a Map-Reduce admin client fsck run a DFS filesystem checking utility fs run a generic filesystem user client balancer run a cluster balancing utility jobtracker run the MapReduce job Tracker node pipes run a Pipes job tasktracker run a MapReduce task Tracker node job manipulate MapReduce jobs queue get information regarding JobQueues version print the version jar <jar> run a jar file distcp <srcurl> <desturl> copy file or directories recursively archive -archiveName NAME <src>* <dest> create a hadoop archive daemonlog get/set the log level for each daemon or CLASSNAME run the class named CLASSNAME Most commands print help when invoked w/o parameters.
If you do not see the Hadoop command options, ensure that the JAVA_HOME is pointing to the correct JDK.
Configuring a Single-node Hadoop Setup
By default, Hadoop is configured to run in single-node mode. To test whether Hadoop is working properly, take HDFS for a test drive as follows:
- $ mkdir input
- $ cp bin/*.sh input
- $ bin/hadoop jar hadoop-examples-*.jar grep input output 'start[a-z.]+'
This command should kick-off MapReduce tasks, which create verbose output that would start as follows:
<date time>INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= <date time>INFO mapred.FileInputFormat: Total input paths to process : 12 <date time> INFO mapred.JobClient: Running job: job_local_0001 <date time> INFO mapred.FileInputFormat: Total input paths to process : 12 <date time> INFO mapred.MapTask: numReduceTasks: 1 <date time> INFO mapred.MapTask: io.sort.mb = 100 <date time> INFO mapred.MapTask: data buffer = 79691776/99614720 <date time> INFO mapred.MapTask: record buffer = 262144/327680 <date time> INFO mapred.MapTask: Starting flush of map output <date time> INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting <date time> INFO mapred.LocalJobRunner: file:/opt/hadoop-0.20.2/input/hadoop- config.sh:0+1966 ...
Verify the output by listing its contents with cat output/*.
While it may vary based on the release you’ve installed, the output should be similar to:
2 starting 1 starts 1 startup
Configuring a Pseudo-distributed Mode Setup
An important prerequisite for configuring Hadoop in a pseudo-distributed mode setup is to have the ability to SSH to localhost without a passphrase.
Try the following:
ssh localhost
If you are prompted to accept the authenticity of localhost, answer yes to the prompt.
If you are logged in successfully without having to enter your password, you are ready to proceed. Otherwise, you will need to run the following commands to set up key-based authentication, without the need of a password:
$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa $ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
Hadoop can be run on a single-node in pseudo-distributed mode. In the pseudo-distributed mode, each Hadoop daemon runs as a separate Java process.
Here are the essential installation steps:
1. Edit conf/core-site.xml by replacing the empty <configuration></configuration> tags with:
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
(This configures the HDFS daemon.)
2. Edit conf/hdfs-site.xml by replacing the empty <configuration></configuration> tags with:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
(This configures the replication factor. A replication factor of 1 means no replication. You need more than a single node to have a higher replication factor.)
3. Edit conf/mapred-site.xml by replacing the empty <configuration></configuration> tags with:
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
(This configures the MapReduce daemon.)
4. Test the pseudo-distributed setup by formatting the Hadoop Distributed File System (HDFS) on this single system:
bin/hadoop namenode -format
All is good if you see output similar to the following (your hostname will be different):
11/05/26 23:05:36 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = treasuryofideas-desktop/127.0.1.1 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 0.20.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 ************************************************************/ 11/05/26 23:05:37 INFO namenode.FSNamesystem: fsOwner=treasuryofideas,treasuryofideas,adm,dialout, cdrom,plugdev,lpadmin,admin,sambashare 11/05/26 23:05:37 INFO namenode.FSNamesystem: supergroup=supergroup 11/05/26 23:05:37 INFO namenode.FSNamesystem: isPermissionEnabled=true 11/05/26 23:05:37 INFO common.Storage: Image file of size 105 saved in 0 seconds. 11/05/26 23:05:37 INFO common.Storage: Storage directory /tmp/hadoop- treasuryofideas/dfs/name has been successfully formatted. 11/05/26 23:05:37 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at treasuryofideas-desktop/127.0.1.1 ************************************************************/
5. Start all Hadoop daemons:
bin/start-all.sh
6. Verify all of the expected components are running by checking for the log files, which are available by default in the logs directory of the Hadoop distribution.
You should see the following log files with your username in place of username and your system’s hostname in place of hostname:
$ ls logs/ hadoop-username-datanode-hostname.log hadoop-username-datanode-hostname.out hadoop-username-jobtracker-hostname.log hadoop-username-jobtracker-hostname.out hadoop-username-namenode-hostname.log hadoop-username-namenode-hostname.out hadoop-username-secondarynamenode-hostname.log hadoop-username-secondarynamenode-hostname.out hadoop-username-tasktracker-hostname.log hadoop-username-tasktracker-hostname.out history/
7. Access the Namenode and JobTracker web interface at http://localhost:50070/ and http://localhost:50030/, respectively.
8. Run jps, which lists the Java processes. You should see the following output, along with other Java processes you may have running:
2675 JobTracker 2442 DataNode 2279 NameNode 3027 Jps 2828 TaskTracker 2603 SecondaryNameNode
(The process IDs will in all probability be different on your machine.)
9. Next, re-run the HDFS test drive like so:
bin/hadoop fs -put bin input bin/hadoop jar hadoop-*-examples.jar grep input output 'start[a-z.]+' 11/06/04 11:53:07 INFO mapred.FileInputFormat: Total input paths to process : 17 11/06/04 11:53:08 INFO mapred.JobClient: Running job: job_201106041151_0001 11/06/04 11:53:09 INFO mapred.JobClient: map 0% reduce 0% 11/06/04 11:53:24 INFO mapred.JobClient: map 11% reduce 0% (...) 11/06/04 11:54:58 INFO mapred.JobClient: map 100% reduce 27% 11/06/04 11:55:10 INFO mapred.JobClient: map 100% reduce 100% 11/06/04 11:55:15 INFO mapred.JobClient: Job complete: job_201106041151_0001 (...) 11/06/04 11:55:48 INFO mapred.JobClient: Combine output records=0 11/06/04 11:55:48 INFO mapred.JobClient: Reduce output records=4 11/06/04 11:55:48 INFO mapred.JobClient: Map output records=4
10. To verify the output, copy the output from HDFS to the local filesystem and then concatenate and print the contents to standard output:
bin/hadoop fs -get output pseudo-output cat pseudo-output/* cat: psuedo-output/_logs: Is a directory 5 starting 1 started 1 starts 1 startup
11. Concatenate and print the MapReduce job directly from HDFS. This should match what you see from getting the output on your local filesystem previously.
bin/hadoop fs -cat output/part* 5 starting 1 started 1 starts 1 startup
That completes the essential pseduo-distributed setup. For Hadoop cluster setup and configuration start at http://hadoop.apache.org/common/docs/r0.20.2/cluster_setup.html.
INSTALLING AND SETTING UP HBASE
To install and configure HBase in standalone mode, do the following:
1. Download the latest stable HBase distribution from an Apache Mirror (www.apache.org/dyn/closer.cgi/hbase/). Currently, the latest stable version is hbase-0.90.3. You should double-check compatibility with the version of Hadoop you are using, as there are often specific dependencies from HBase.
2. Unpack, untar, and unzip the HBase distribution:
tar zxvf hbase-0.90.3.tar.gz.
3. Move the unpacked distribution to a desired place in the filesystem. I moved it to /opt.
4. Create a symbolic link as follows: ln -s hbase-0.90.3 hbase.
5. Edit conf/hbase-site.xml by replacing the empty <configuration.</configuration> tags with:
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///opt/hbase_rootdir</value>
</property>
</configuration>
(The hbase.rootdir is the directory HBase writes to. I have set hbase.rootdir to /opt/hbase_rootdir. You can choose any other location on your filesystem. The default hbase.rootdir value is /tmp/hbase-${user.name}, which may be deleted every time the server reboots.)
6. Verify by starting HBase:
bin/start-hbase.sh
7. Connect to HBase via the shell:
bin/hbase shell
INSTALLING AND SETTING UP HIVE
The following software is required to set up Hive:
- Java 1.6.x — Hadoop is tested against the Sun (now Oracle) JDK.
- Hadoop 0.20.2 — Hive works with Hadoop versions between 0.17.x to 0.20.x.
To install and configure Hive, do the following:
1. Download the stable binary distribution from an Apache Mirror (www.apache.org/dyn/closer.cgi/hive/). The current stable release version is hive-0.70.0. The binary distributions have -bin in the filename, and are cross-platform since they are written in Java.
2. Unpack, untar, and unzip the Hive distribution:
tar zxvf hive-0.7.0-bin.tar.gz.
3. Move the unpacked distribution to a desired place in the filesystem. I moved it to /opt.
4. Create a symbolic link:
ln -s hive-0.7.0-bin hive
5. Set the HIVE_HOME environment variable:
export HIVE_HOME=/opt/hive
(Point to the directory that contains Hive distribution.)
6. Add the hive executable to the PATH environment variable:
export PATH=$HIVE_HOME/bin:$PATH
7. Verify that it is set correctly by running which hive. You should see the path you’ve configured.
which hive /opt/hive/bin/hive
Configuring Hive
Make sure to have either Hadoop in your path or set the HADOOP_HOME environment variable to point to the Hadoop folder. You can set HADOOP_HOME as follows:
1. Set the environment variable:
export HADOOP_HOME=/opt/hadoop
(Point to the directory that contains the HADOOP distribution.)
2. Create /tmp directory on HDFS:
$HADOOP_HOME/bin/hadoop fs -mkdir /tmp
(Note that this may already exist.)
3. Give write permission on the /tmp HDFS directory to the group:
$HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp
4. Create hive.metastore.warehouse.dir directory (default value: /user/hive/warehouse) on HDFS:
$HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
5. Give write permission on HDFS /user/hive/warehouse directory to the group:
$HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse
Overlaying Hadoop Configuration
Hive configuration lays on top of the Hadoop configuration. The default Hive configuration is included in the conf/hive-default.xml file in the Hive distribution. You can override the default Hive configuration by redefining configuration variables and their values in conf/hive-site.xml. You can also change the Hive configuration directory by pointing the HIVE_CONF_DIR variable to the new configuration directly. In addition to redefining variables in conf/hive-site.xml, you can also redefine the configuration variables using any of the following:
- Hive Command Line Interface (CLI) SET command — For example, hive > SET mapred.job.tracker=hostName.organizationName.com:50030; sets the MapReduce cluster to the one specified.
- hiveconf configuration variable and value pair — Pass the hiveconfig variable and value pair to the hive executable.
For example, bin/hive -mapred.job.tracker=hostName.organizationName.com:50030 sets the MapReduce cluster exactly the same way as the earlier SET command does. Multiple hiveconf variable and value pairs can be passed to the hive executable. Sometimes passing a number of configuration parameters can be a little difficult to maintain. In such cases, concatenate all configuration variable and value pairs and set it as the value of the HIVE_OPTS environment variable.
To verify the Hive installation and setup, run $HIVE_HOME/bin/hive.
INSTALLING AND SETTING UP HYPERTABLE
The easiest way to install Hypertable is to use the binary downloads. Hypertable binaries are compliant with all systems built with glibc 2.4+. If your system is built with a version of glibc older than 2.4, compile and package Hypertable manually using the instructions at http://code.google.com/p/hypertable/wiki/HowToPackage.
Hypertable offers binaries for both 64-bit and 32-bit platforms. Packages are available in .rpm, .deb, and .dmg formats. A binary distribution is also available in .tar.bz2 format. To try to be package-agnostic, I usually choose the .tar.bz2 format.
Follow these steps to install the Hypertable 64-bit .tar.bz2 platform:
1. Download the latest release distribution from www.hypertable.com/download/. The latest release distribution version is 0.9.5.0.pre5.
2. Unpack, untar, and unzip the distribution:
tar jxvf hypertable-0.9.5.0.pre5-linux-x86_64.tar.bz2.
3. Move the contents of the unpacked distribution to a desired directory on your filesystem. The unpacked distribution content is in a directory structure like so:
/opt/hypertable/<version>
Therefore, making the distribution available at /opt/hypertable/<version> is recommended. It can be achieved using:
cd hypertable-0.9.5.0.pre5-linux-x86_64/opt, and mv hypertable /opt
4. Create a symbolic link, named current, to point to the release distribution:
ln -s /opt/hypertable/0.9.5.0.pre5 /opt/hypertable/current
5. Optionally, make the distribution compliant with the Filesystem Hierarchy Standard (FHS).
Making the Hypertable Distribution FHS-Compliant
FHS is a recommended way to organize the files in a Linux/Unix filesystem. The standard recommends that host-specific configuration files for add-on software packages be stored in /etc/opt. The standard also suggests that variable data for such software should be in /var/opt.
To make the Hypertable distribution FHS-compliant, do the following:
1. Replace <userName>:<groupName> with your user and group (available from the id command) and run the following commands:
sudo mkdir /etc/opt/hypertable /var/opt/hypertable sudo chown <userName>:<groupName> /etc/opt/hypertable /var/opt/hypertable
2. Run the following:
$ bin/fhsize.sh: Setting up /var/opt/hypertable Setting up /etc/opt/hypertable fshize /opt/hypertable/current: success
Making Hypertable FHS-compliant avoids re-creation of the config, log, hyperspace, and localBroker root directory when Hypertable is upgraded.
3. Verify that Hypertable is FHS-compliant by listing the contents of /opt/hypertable/current and to verify the symbolic links. The listing should be as follows:
$ cd /opt/hypertable/current $ ls -l bin conf -> /etc/opt/hypertable examples fs -> /var/opt/hypertable/fs hyperspace -> /var/opt/hypertable/hyperspace include lib log -> /var/opt/hypertable/log Monitoring run -> /var/opt/hypertable/run
Configuring Hadoop with Hypertable
If you configured Hadoop according to the instructions earlier in this chapter, you have the HDFS daemon configured by putting the following in the Hadoop conf/core-site.xml as follows:
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
(contents of conf/core-site.xml)
Now you need to edit conf/hypertable.cfg:
1. Verify that the configuration for HDFS in hypertable.cfg, by ensuring the following HdfsBroker.fs.default.name is
# HDFS Broker HdfsBroker.fs.default.name=hdfs://localhost:9000 matches with the HDFS daemon configuration in Hadoop conf/core-site.xml Create /hypertable directory on HDFS: $HADOOP_HOME/bin/hadoop fs -mkdir /hypertable
2. Give write permission on the HDFS /hypertable directory to the group with this command:
$HADOOP_HOME/bin/hadoop fs -chmod g+w /hypertable
INSTALLING AND SETTING UP MONGODB
Download the latest release distribution from www.mongodb.org/downloads. Binary distributions for most mainstream operating systems are available via the MongoDB download link. I download and install 1.8.2-rc2 for 64-bit Linux. If you choose any other distribution, the following steps of installing and setting up will still largely be the same.
1. Unpack, untar, and unzip the .tgz format as follows:
tar zxvf mongodb-osx-x86_64-1.8.2-rc2.tgz
2. Move the extracted package to a desired place on your filesystem. I prefer to move it to /opt as follows:
mv mongodb-osx-x86_64-1.8.2-rc2 /opt
3. Create a symbolic link named mongodb to point to the directory that contains the MongoDB distribution:
ln -s mongodb-osx-x86_64-1.8.2-rc2 mongodb
Configuring MongoDB
By default, MongoDB stores the data files in the /data/db directory. If you want to use the default directory, create the directory and set the appropriate permissions as follows:
$ sudo mkdir -p /data/db $ sudo chown `id -u` /data/db
If you would like an alternative directory like /opt/data/db to store the MongoDB data files, change the directory creation and permission setting commands as follows:
$ sudo mkdir -p /opt/data/db $ sudo chown `id -u` /opt/data/db
When you use a data directory other than the default value, remember to pass it in as the value of the --dbpath argument of the mongodb server executable program. For example:
bin/mongod --dbpath /opt/data/db
INSTALLING AND CONFIGURING COUCHDB
To install CouchDB, Erlang and Erlang OTP are required.
It’s easy to install Erlang on Linux and Unix. On Mac OS X you can leverage brew (http://mxcl.github.com/homebrew/) to install Erlang. On Windows, the easiest way to install CouchDB is to install Couchbase’s Couchbase Server 1.1, which is available at www.couchbase.com/downloads and includes both an Erlang Windows distribution and CouchDB with some additional features. There are instructions for installing Apache CouchDB on Windows from components online at http://wiki.apache.org/couchdb/Installing_on_Windows; include the steps involved in installing Erlang.
Apache CouchDB has installers for most platforms. Access the installers and instructions on how to use them at http://wiki.apache.org/couchdb/Installation. Couchbase, the company behind CouchDB, provides binary installations for many platforms.
So far, most of the installation instructions included in this appendix have focused on installing the binaries. All the software programs listed in this appendix are open source. The source of these products is freely available and you can choose to build and install the software programs from source. As an example, the following section demonstrates how to build and install CouchDB from source on Unbuntu 10.04.
Installing CouchDB from Source on Ubuntu 10.04
CouchDB can be installed from source on Ubuntu Linux using the following steps:
1. Install the dependencies:
sudo apt-get build-dep couchdb sudo apt-get install xulrunner-1.9.2-dev libicu-dev libcurl4-gnutls-dev libtool
2. Get the xulrunner version as follows:
xulrunner -v
The output on Ubunutu 10.04 on my machine was Mozilla XULRunner 1.9.2.17 - 20110424212116.
3. Create an xulrunner shared library loading configuration. This is needed because there may be many xulrunner versions, and the OS:
sudo vi /etc/ld.so.conf.d/xulrunner.conf
4. Add the following lines:
/usr/lib/xulrunner-1.9.2.17 /usr/lib/xulrunner-devel-1.9.2.17
5. Run ldconfig:
sudo /sbin/ldconfig
6. Get the source code from the code repository. You can use either SVN or Git.
git clone git://git.apache.org/couchdb.git
7. Change to the source directory:
cd couchdb
8. Bootstrap the build:
./bootstrap
Note that if you get an error here, you may have to install dependencies, which are covered in the INSTALL.Unix file included in the distribution. aclocal may also need to be installed with sudo apt-get install automake.
9. Configure the build:
./configure
10. Build and install:
make && sudo make install
11. Create a user name couchdb:
useradd couchdb
12. Change permissions on CouchDB directories to user couchdb.
13. Change ownership on CouchDB directories to user couchdb:
chown -R couchdb:couchdb /usr/local/etc/default/couchdb chown -R couchdb:couchdb /usr/local/etc/init.d/couchdb chown -R couchdb:couchdb /usr/local/etc/couchdb chown -R couchdb:couchdb /usr/local/etc/logrotate.d/couchdb chown -R couchdb:couchdb /usr/local/lib/couchdb chown -R couchdb:couchdb /usr/local/bin/couchdb chown -R couchdb:couchdb /usr/local/var/lib/couchdb chown -R couchdb:couchdb /usr/local/var/run/couchdb chown -R couchdb:couchdb /usr/local/var/log/couchdb chown -R couchdb:couchdb /usr/local/share/doc/couchdb chown -R couchdb:couchdb /usr/local/share/couchdb
INSTALLING AND SETTING UP REDIS
To begin the Redis installation process, follow these steps:
1. Download the latest stable release from http://redis.io/download. Version 2.2.8 is the latest version.
2. Unpack, untar, and unzip the Redis distribution: tar zxvf redis-2.2.8.tar.gz.
3. Move the Redis unpacked distribution to a desired location on the filesystem. I usually move it to /opt using mv redis-2.2.8 /opt.
4. Create a symbolic link to point to the Redis distribution:
ln -s redis-2.2.8 redis
5. To build, change to the directory that contains Redis and run make:
cd redis make
6. To verify, run make test.
INSTALLING AND SETTING UP CASSANDRA
To begin the process of installing Cassandra, follow these steps:
1. Download a binary development release version from http://cassandra.apache.org/download/. The latest version is 0.8.0-rc1. The downloaded file is apache-cassandra-0.8.0-rc1-bin.tar.gz.
2. The distribution is available in the .tar.gz format. Unpack, untar, and unzip the distribution:
tar zxvf apache-cassandra-0.8.0-rc1-bin.tar.gz
3. Move the extracted distribution to a desired location on the filesystem:
mv apache-cassandra-0.8.0-rc1 /opt
4. Create a symbolic link named apache-cassandra to point to the directory that contains Cassandra:
ln -s apache-cassandra-0.8.0-rc1 apache-cassandra
Configuring Cassandra
Cassandra can be configured by defining a configuration variable in conf/cassandra.yaml. Most default configurations are good for a single-node setup. Simply make sure that all the paths to the directories specified in cassandra.yaml exist.
The following configurations point to a file on the filesystem:
# directories where Cassandra should store data on disk.
data_file_directories:
- /var/lib/cassandra/data
# commit log
commitlog_directory: /var/lib/cassandra/commitlog
# saved caches
saved_caches_directory: /var/lib/cassandra/saved_caches
Then create /var/lib/cassandra using sudo mkdir -p /var/lib/cassandra. Make sure to set appropriate permissions on this directory so the user that runs the Cassandra process can write to these directories.
Configuring log4j for Cassandra
The log4j server properties are specified in log4j-server.properties. The log4j appender file is specified in this file as follows:
log4j.appender.R.File=/var/log/cassandra/system.log
Make sure the directory /var/log/cassandra exists and is set with appropriate permissions so that the user running the Cassandra process can write to it.
Installing Cassandra from Source
The following sources are required:
- Java 1.6.x
- Ant 1.8.2
Build Cassandra from source using the following steps:
1. Download the source of the latest development release from http://cassandra.apache.org/download/. The current version is 0.8.0.rc1.
2. Unpack, untar, and unzip the downloaded source:
tar zxvf apache-cassandra-0.8.0-rc1-src.tar.gz.
3. Run the Ant build task:
ant
INSTALLING AND SETTING UP MEMBASE SERVER AND MEMCACHED
Download the relevant edition from www.couchbase.com/downloads. Three different distributions are available to download and install:
- Membase server
- Memcached server
- Couchbase server
To install Membase server, download a copy for your operating system from www.couchbase.com/downloads. Binaries are easy to install. A sample walk through for Mac OS X illustrates how you could install the binaries.
The next few steps relate to Mac OS X:
1. The Membase version on Mac OS X is packaged as a zip file. The specific file is membase-server-community-1.6.5.3.zip.
2. Unzip the file:
unzip membase-server-community-1.6.5.3.zip.
The unzipped application is available in a Mac OS X distribution format folder Membase.app.
3. Move Membase.app to /Applications or any other folder that you save the applications in.
INSTALLING AND SETTING UP NAGIOS
This section only covers instructions that will help you install Nagios from source on Ubuntu. For additional details, read the Nagios documentation, available online at www.nagios.org/documentation.
The following software is required:
- Apache 2
- PHP
- The GNU Compiler Collection (GCC), http://gcc.gnu.org/, compiler and development libraries
- GD development libraries
You can install these required pieces of software as follows:
1. Install Apache 2:
sudo apt-get install apache2
2. Install PHP:
sudo apt-get install libapache2-mod-php5
3. Install GCC and development libraries:
sudo apt-get install build-essential
4. Install GD development libraries:
sudo apt-get install libgd2-xpm-dev
It’s recommended that a user named nagios be created and that the Nagios process be owned and run by this user. Create a user named nagios on Ubuntu as follows:
sudo /usr/sbin/useradd -m -s /bin/bash nagios sudo passwd nagios
(Set a password. I default it to nagios. You will be prompted to enter a password and will be prompted to confirm the password.)
Create a nagcmd group and add both the nagios and the apache user to this group:
sudo /usr/sbin/groupadd nagcmd sudo /usr/sbin/usermod -a -G nagcmd nagios sudo sudo /usr/sbin/usermod -a -G nagcmd nagios
Downloading and Building Nagios
If all the required pieces of software are installed, download and build Nagios as follows:
1. Download Nagios Core and Nagios Plugins from www.nagios.org/download/. The current release version of Nagios Core is 3.2.3 and the current release version of Nagios Plugins is 1.4.15.
2. Unpack, untar, and unzip the Nagios distribution:
tar zxvf nagios-3.2.3.tar.gz
3. Change to the nagios-3.2.3 directory:
cd nagios-3.2.3
4. Configure Nagios:
./configure --with-command-group=nagcmd
5. Build Nagios:
make all
6. Install the binaries:
sudo make install
7. Install the init script:
sudo make install-init
The output of this command should be as follows:
/usr/bin/install -c -m 755 -d -o root -g root /etc/init.d /usr/bin/install -c -m 755 -o root -g root daemon-init /etc/init.d/nagios *** Init script installed ***
8. Install sample config files:
sudo make install-config.
The output of this command would be as follows:
/usr/bin/install -c -m 775 -o nagios -g nagios -d /usr/local/nagios/etc /usr/bin/install -c -m 775 -o nagios -g nagios -d /usr/local/nagios/etc/objects /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/nagios.cfg /usr/local/nagios/etc/nagios.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/cgi.cfg /usr/local/nagios/etc/cgi.cfg /usr/bin/install -c -b -m 660 -o nagios -g nagios sample-config/resource.cfg /usr/local/nagios/etc/resource.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/templates.cfg /usr/local/nagios/etc/objects/templates.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/commands.cfg /usr/local/nagios/etc/objects/commands.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/contacts.cfg /usr/local/nagios/etc/objects/contacts.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/timeperiods.cfg /usr/local/nagios/etc/objects/timeperiods.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/localhost.cfg /usr/local/nagios/etc/objects/localhost.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/windows.cfg /usr/local/nagios/etc/objects/windows.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/printer.cfg /usr/local/nagios/etc/objects/printer.cfg /usr/bin/install -c -b -m 664 -o nagios -g nagios sample-config/template- object/switch.cfg /usr/local/nagios/etc/objects/switch.cfg *** Config files installed ***
9. Set permission on the directory for holding the external command file:
sudo make install-commandmode
The output of the command is as follows:
/usr/bin/install -c -m 775 -o nagios -g nagcmd -d /usr/local/nagios/var/rw chmod g+s /usr/local/nagios/var/rw *** External command directory configured ***
Configuring Nagios
1. Configure email address.
2. Edit the contacts config file:
sudo vi /usr/local/nagios/etc/objects/contacts.cfg.
Change the email field from nagios@localhost to your email address.
The next couple of steps help configure the Nagios web interface.
3. Install Nagios web config file in the Apache config.d directory: sudo make install-webconf
/usr/bin/install -c -m 644 sample-config/httpd.conf /etc/apache2/conf.d/nagios.conf *** Nagios/Apache conf file installed ***
4. Create an account for logging in to the Nagios web interface:
sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
You will be prompted for a password and be asked to confirm it.
5. Restart Apache:
sudo /etc/init.d/apache2 reload
Compiling and Installing Nagios Plugins
In an earlier step you downloaded the Nagios Plugin from www.nagios.org/download/. The latest release file version is 1.4.15.
Nagios Plugins can be compiled and installed as follows:
1. Unpack, untar, and unzip the Nagios Plugins distribution:
tar zxvf nagios-plugins-1.4.15.tar.gz
2. Change to the Nagios Plugins extracted distribution:
cd nagios-plugins-1.4.15
3. Configure Nagios Plugins:
./configure --with-nagios-user=nagios --with-nagios-group=nagios
4. Build Nagios Plugins:
make
5. Install Nagios Plugins:
sudo make install
Nagios and Nagios Plugins are now installed and you can start Nagios. Additional configuration is not covered in this document but you can read the official documentation, available at www.nagios.org/documentation to get all the details.
INSTALLING AND SETTING UP RRDTOOL
Instructions for installing RRDtool on Linux and Unix are covered in this section.
SVN client, automake, autoconf, and libtool are required to install RRDtool.
RRDtool can be installed from source as follows:
svn checkout svn://svn.oetiker.ch/rrdtool/trunk/program mv program rrdtool-trunk cd rrdtool-trunk ./autogen.sh ./configure --enable-maintainer-mode make sudo make install
INSTALLING HANDLER SOCKET FOR MYSQL
Handler Socket for MySQL works with MySQL server 5.x. Handler Socket for MySQL can be installed as follows:
git clone git://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL.git cd HandlerSocket-Plugin-for-MySQL ./autogen.sh ./configure --with-mysql-source=/root/install/mysql-<version number> --with-mysql-bindir=/usr/bin make make install