When viewing the collected data and statistics regarding our highly available database, we can simply settle for the raw numbers. They tell a story and include precise measurements necessary for making decisions regarding architecture and incidence response. However, many would argue that this is much easier with graphs and charts, as they enable the identification of trends.
There are a lot of graphing libraries and tools, but relatively few of them are tailored to the needs of an agile monitoring team. The makers of Graphite helped fill this role, and they did so with an extremely versatile tool. Graphite makes visualizing the collected system statistics easy. Unfortunately, due to the number of its installation requirements, administrators might skip it in favor of something easier to use.
We don't want this to happen to our readers! Follow along, and we'll help you take advantage of one of the most powerful system visualization suites available.
Red-Hat-based systems will need to add the EPEL library. The most recent EPEL packages are available for several Red-Hat-based distributions at http://download.fedoraproject.org/pub/epel.
Look for the package file that begins with epel-release and download it to the monitoring server. Once the package is downloaded, install it with this command as a root-level user:
sudo rpm -ivh epel-release-*.rpm
Once epel has been installed, install the python-pip, django, and cairo packages and their requirements with this command:
sudo yum install python-pip django cairo
Debian-based systems should have an easier time due to the larger standard repositories. Execute these commands to install equivalent packages:
sudo apt-get install python-django python-django-tagging
sudo apt-get install python-pip python-cairo
Some build requirements include Python development libraries. These will depend on the Linux distribution in use but will likely be called python-dev, python26-devel, or some variant. Find and install the latest version available in the package repository before continuing.
Follow these steps to install, configure, and start Graphite on the dedicated monitoring server:
- Install the web-based visualization frontend with this command:
sudo pip install graphite-web - Install the data-caching daemon with this command:
sudo pip install carbon Twisted=11.1 - Install the data storage engine with this command:
sudo pip install whisper - Create a file named
local_settings.pyin the/opt/graphite/webapp/graphite/directory with these contents:SECRET_KEY = 'Put some unique text here.'
- Initialize the Graphite management database with this command:
sudo python /opt/graphite/webapp/graphite/manage.py syncdb - Copy two of the default storage configuration files with these commands:
cd /opt/graphite/conf sudo mv carbon.conf.example carbon.conf sudo mv storage-schemas.conf.example storage-schemas.conf
- Start the carbon daemon with the following command:
sudo /opt/graphite/bin/carbon-cache.py start - Start the Graphite website with the following commands:
cd /opt/graphite/bin sudo su -c "./run-graphite-devel-server.py /opt/graphite &> /var/log/graphite.log &"
Once the prerequisites are installed, we need to install all of the pieces Graphite needs in order to function. These modules include Graphite-web for web-based graph construction, carbon for aggregating inputs, and whisper to store raw graph data. In the case of carbon, we must also specify which version of the twisted module to use, as carbon is currently incompatible with newer versions.
The next step isn't strictly necessary, but each Graphite installation maintains a unique secret series of characters. We recommend that you generate one and save it in the SECRET_KEY variable of the local_settings.py file. When it is time to secure the Graphite installation, having a secret key will make it easier.
As we have changed no other configuration settings, initializing the Graphite management database will create a sqlite database file in the /opt/graphite/storage directory. This file will store Graphite users, saved graphs and dashboards, and other elements specific to Graphite. We could have installed this in a PostgreSQL database as well. If the amount of Graphite users increases significantly, we recommend that you reinstall the management database into a PostgreSQL database to avoid usage contention. Until then, SQLite should suffice.
Next, there are two configuration files that carbon uses to control its cache and aggregation abilities as well as the output storage format. When we copy the example configuration files for carbon.conf and storage-schemas.conf, carbon will save data with the whisper module that we installed earlier. Furthermore, whisper will aggregate and store data according to the contents of storage-schemas.conf.
Finally, we start the carbon daemon and Graphite itself. Starting carbon is fairly easy due to the manner in which its management script was written. However, Graphite is meant to be displayed through a web server such as Apache or Nginx. As we're skipping the process of integrating Graphite with a web server, we have the option of starting Graphite with a Python-based development web server instead. The command we invoke sets up this Python development web server and directs it to serve Graphite pages. We recommend that you use a more formal installation process on an actual monitoring server.
If everything was successful, we should be able to see Graphite. The default port is 8080, so if we direct a web browser to the monitoring server on that port, we should see this:
We selected a basic data point that carbon tracks for itself, and set the graph time range for 10 minutes. Currently the data available to Graphite is very minimal, but we hope to fix that soon.
Graphite has rather extensive documentation as does the pip utility that we used to install most of its components. We suggest that you read further on these topics if possible, as our installation and configuration examples were extremely minimalistic. Use the following links for more information:
- Graphite wiki: http://graphite.wikidot.com/
- Updated Graphite Documentation: http://graphite.readthedocs.org/en/latest/
- Python Package Index | pip: https://pypi.python.org/pypi/pip