Monitoring local services on a remote machine with NRPE

In this recipe, we'll learn how to install and run an NRPE (Nagios Remote Plugin Executor) server on a target host, roma.example.net. We'll use this to check the load average on that host with the check_load plugin.

The plugins for these checks will be executed on the target server by the NRPE daemon, but the results will be returned to our Nagios Core monitoring server, olympus.example.net. This requires installing the check_nrpe plugin on the monitoring server and the full Nagios Plugins set (but not Nagios Core itself) on the target server.

This is a reasonably long and in-depth recipe as it involves installing a total of three software packages on two servers.

Getting ready

You will need a monitoring server with Nagios Core 4.0 or newer installed. You should also have a UNIX-like target host that you intend to monitor that can run the NRPE daemon. Most modern UNIX-like systems including Linux and BSD should be able to do this. Both the monitoring server and the target host will need internet connectivity and you should already be monitoring the target host itself with a host definition to which we'll be adding service checks.

If your servers don't have a direct gateway to the internet, you can work around this by uploading the relevant files after downloading them onto a workstation, or another machine with internet access.

You should understand the basics of configuring, compiling, and installing software from source. In most cases, a simple ./configure, make, and make install will be all that's necessary and the recipe will walk you through this. You will need to have make(1) installed along with any other tools needed for the process, including a C compiler like gcc(1).

You should also have a good grasp on how hosts and services interrelate in Nagios Core, which is discussed in the recipes in Chapter 1, Understanding Hosts, Services, and Contacts, and how Nagios Core uses commands and plugins, discussed in Chapter 2, Working with Commands and Plugins. You should not need an in-depth understanding of the use of any particular plugin; the recipe will demonstrate the usage of the plugins to which it refers.

Finally, you should be able to configure any firewalls to allow connectivity from the Nagios Core server to the server being monitored with the TCP destination port 5666.

How to do it...

This first part of the recipe is done on the target server.

  1. Download and inflate the latest Nagios Plugins package. At the time of writing, the link is available at http://nagiosplugins.org/download/.
    $ wget http://nagios-plugins.org/download/nagios-plugins-2.1.1.tar.gz
    $ tar -xzf nagios-plugins-2.1.1.tar.gz
    
  2. Configure, compile, and install the plugins, the same way you would on a new monitoring server. You will need to have root privileges for the command make install call.
    $ cd nagios-plugins-2.1.1
    $ ./configure
    $ make
    # make install
    

    You may need to install some shared libraries and headers on the system to do this for certain plugins, such as a libssl implementation. The output of the ./configure script should alert you of any such problems.

  3. Download and inflate the latest version of NRPE from the Nagios Exchange website. At the time of writing, the link is available at: http://exchange.nagios.org/directory/Addons/Monitoring-Agents/NRPE--2D-Nagios-Remote-Plugin-Executor/details.
    $ wget http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.15/nrpe-2.15.tar.gz 
    $ tar -xzf nrpe-2.15.tar.gz
    
  4. Enter the nrpe-2.15 source directory and configure, compile, and install the daemon and a stock configuration for it. You will need to have root privileges for both the make install-daemon and the make install-daemon-config calls.
    $ cd nrpe-2.15
    $ ./configure
    $ make all
    # make install-daemon
    # make install-daemon-config
    

    If you do not already have a nagios user on the target host, you may need to create one before the daemon installs properly:

    # groupadd nagios
    # useradd -r -g nagios nagios
    
  5. Edit the newly installed file at /usr/local/nagios/etc/nrpe.cfg and find the line beginning with allowed_hosts. Add a comma and the IP address of your monitoring server to this line. In this case, we've added the IP address 192.0.2.11:
    allowed_hosts=127.0.0.1,192.0.2.11
  6. Start the nrpe daemon and check it's running by searching the process table with pgrep(1) or ps(1):
    # /usr/local/nagios/bin/nrpe -c /usr/local/nagios/etc/nrpe.cfg -d
    # pgrep nrpe
    18593
    # ps -e | grep [n]rpe
    nagios 18593 1 0 21:55 ? 00:00:01 nrpe
    

If you would like the nrpe daemon to start on boot, add an init script appropriate to your system. An example init-script is generated at ./configure time in the source directory. Versions are also generated for Debian-derived systems and SUSE systems in init-script.debian and init-script.suse respectively. Exactly how this should be done will depend on your particular system, for which you may need to consult its documentation.

This next part of the recipe is done on the monitoring server.

  1. Again, download the latest version of NRPE, the same way as was done for the target server:
    $ wget http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.15/nrpe-2.15.tar.gz
    $ tar -xzf nrpe-2.15.tar.gz
    
  2. Again, configure and build the software. However, note that this time the install line is different, as we're installing the check_nrpe plugin rather than the nrpe daemon:
    $ cd nrpe-2.15.tar.gz
    $ ./configure
    $ make all
    # make install-plugin
    
  3. Check that the plugin has been correctly installed. It should be saved at /usr/local/nagios/libexec/check_nrpe:
    $ ls /usr/local/nagios/libexec/check_nrpe
    /usr/local/nagios/libexec/check_nrpe
    
  4. Move to the directory containing the Nagios Core object configuration. By default, this is /usr/local/nagios/etc/objects.
    $ cd /usr/local/nagios/etc/objects
    
  5. Edit an appropriate file for defining new commands. For the default installation, /usr/local/nagios/etc/objects/commands.cfg is a good choice. Add the following definition to the end of this file:
    define command {
        command_name  check_nrpe
        command_line  $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
    }
  6. Edit the file defining the target host as an object. The definition might look something like this:
    define host {
        use        linux-server
        host_name  roma.example.net
        alias      roma
        address    192.0.2.61
    }
  7. Beneath the definition for the host, after any other services defined for it, add the following service definition:
    define service {
        use                  generic-service
        host_name            roma.example.net
        service_description  LOAD
        check_command        check_nrpe!check_load
    }
  8. Validate the configuration and restart the Nagios Core server:
    # /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
    # /etc/init.d/nagios reload
    

With this done, a new service with the description LOAD should appear in the web interface, ready to be checked, and with an appropriate status, including the load average as read from the nrpe daemon on the target host, as shown in the following screenshot:

How to do it...

We can see more detail about how the check was performed and its results in the details page for the service:

How to do it...

If the load average on roma.example.net exceeds the limits defined for the check_load command in /usr/local/nagios/etc/nrpe.cfg on the target host, the service will enter WARNING or CRITICAL states, and will send notifications if configured to do so, all in the same manner as a non-NRPE service.

How it works...

The NRPE plugin and daemon are used to run Nagios Core plugins on the target host rather than on the monitoring server itself. The results of the check are then passed back to the monitoring server and recorded and analyzed by Nagios Core the same way as if the service was running a plugin on the monitoring server, for example, check_http or check_ssh.

The recipe we followed does four main things:

  • We installed the latest Nagios Plugins package to the target host, including the check_load plugin. This is necessary, because the plugin is actually run on the target host and not on the monitoring server as is the case with plugins that check network services.
  • We installed the nrpe daemon to the target host along with a stock configuration file nrpe.cfg. This is the network service through which the check_nrpe plugin will request commands to be run on the target host. The plugins will be run by this process, typically as the nagios user.
  • We installed the check_nrpe plugin to the monitoring host and defined a command of the same name to use it. The command accepts one argument in the $ARG1$ macro; its value is the command that should be run on the target host. In this case, we supplied check_load for this argument.
  • We set up a service to monitor the output of the standard check_load plugin via check_nrpe.

Like other Nagios Core plugins, the check_nrpe program can be run directly from the command line. If we wanted to test the response of the configuration we've arranged in the preceding recipe, we might run the following:

$ /usr/local/nagios/libexec/check_nrpe -H roma.example.net -c check_load
OK - load average: 0.00, 0.00, 0.00|load1=0.000;15.000;30.000;0;
load5=0.000;10.000;25.000;0; load15=0.000;5.000;20.000;0;

In this case, the state of OK and the load average values, as retrieved by check_load, were returned by the nrpe daemon as the result of the check_nrpe call.

How it works...

It's very important to note that this simple configuration of NRPE is not completely secure by default. The recipes listed under See also for this recipe provide some basic means to secure NRPE instances from abuse. These should be used in concert with a sensible firewall policy.

There's more...

Of course, check_load is not the only plugin that can be run on the target server this way. If we inspect the file /usr/local/nagios/etc/nrpe.cfg on the target host, near the end of the file we find some other example definitions of commands that check_nrpe will run upon requests issued from the monitoring server:

command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200

We recognize check_load as the second of these. Note that it already includes some thresholds for WARNING and CRITICAL alerts in its -w and -c parameters.

If we also wanted to check the number of processes on this server, we could add a service check for roma.example.net, defined like this:

define service {
    use                  generic-service
    host_name            roma.example.net
    service_description  PROCS
    check_command        check_nrpe!check_total_procs
}

This service will generate a WARNING alert if the number of processes exceeds 150 and a CRITICAL alert if it exceeds 200. Again, the plugin is run on the target server and not the monitoring server.

Another useful and common application of check_nrpe is to make remote checks on database servers with plugins like check_mysql and check_pgsql in the case where the servers do not listen on network interfaces for security reasons. Instead, they listen only on localhost or UNIX sockets and are hence inaccessible to the monitoring server. To work around this problem, we could add a new command definition to the end of nrpe.cfg on the target server as follows:

command[check_mysql]=/usr/local/nagios/libexec/check_mysql -u nagios -d nagios -p wGG7H233bq

A corresponding check that uses the check_mysql command can then be made on the monitoring server:

define service {
    use                  generic-service
    host_name            roma.example.net
    service_description  MYSQL
    check_command        check_nrpe!check_mysql
}

See the Monitoring database services recipe in Chapter 5, Monitoring Methods, for some detail on how to use the check_mysql and check_pgsql plugins.

NRPE is thus useful not just for making checks of system properties or hardware, but for any plugin that needs to be run on the target host rather than on the monitoring host.

Finally, it's important to note that the command definitions included in the default nrpe.cfg file are intended as examples; you will probably want to fine-tune the parameters for some of them and remove the ones you don't use along with adding your own.

See also

  • The Setting the listening address for NRPE recipe in this chapter
  • The Setting allowed client hosts for NRPE recipe in this chapter
  • The Creating new NRPE command definitions securely recipe in this chapter
  • The Giving limited sudo(8) privileges to NRPE recipe in this chapter
  • Monitoring database services, Chapter 5, Monitoring Methods
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.119.235.79