At the base of everything in Nagios is a plugin, the minion who carries out the job of retrieving the information, evaluating it, raising the alarm, and providing a meaningful message. Left alone, Nagios does not know how to call a plugin, what options to pass to it or how to handle it, so we need a command definition, which defines how the script will be called.

Let's take as an example the command definition for the ssh service check, which is failing because the port used for the check is not the one the daemon is listening on:

# 'check_ssh' command definition
define command{
command_name check_ssh
command_line /usr/lib/nagios/plugins/check_ssh '$HOSTADDRESS$'
}

We can see here a command definition named command_name check_ssh.

Let's keep check_ssh in mind, because it will be the handle we will use to refer to this command definition later on. As we can see, this definition is really short; it defines a handle, and, most importantly, the command line to call the plugin. In this case, it is really easy: the plugin accepts the host address and that is enough for a basic check. Look at $HOSTADDRESS$. This is one of the so-called Nagios standard macros: essentially a place holder, which will be instantiated by Nagios with the host address of the host you will associate the service making use of this command. Nothing complicated so far, let's move onto the service definition, making use of this command definition:

# check that ssh services are running
define service {
use generic-service
host_name localhost
service_description SSH
check_command check_ssh
}

The ssh service definition introduces something new, and this is the inheritance of properties by the Nagios objects. As we discussed previously, the script carries out the checking, evaluation, and alarm raising; the core does all the rest, and lots of stuff. Looking at this service definition, it does not seem a lot, but focus on the first line named use generic-service. This rings a bell. Looking at the definition, it seems that generic-service is actually a template, doesn't it?

# generic service template definition
define service{
name generic-service ; The 'name' of this service template
active_checks_enabled 1 ; Active service checks are enabled
passive_checks_enabled 1 ; Passive service checks are enabled/accepted
parallelize_check 1 ; Active service checks should be parallelized (disabling this can lead to major performance problems)
obsess_over_service 1 ; We should obsess over this service (if necessary)
check_freshness 0 ; Default is to NOT check service 'freshness'
notifications_enabled 1 ; Service notifications are enabled
event_handler_enabled 1 ; Service event handler is enabled
flap_detection_enabled 1 ; Flap detection is enabled
failure_prediction_enabled 1 ; Failure prediction is enabled
process_perf_data 1 ; Process performance data
retain_status_information 1 ; Retain status information across program restarts
retain_nonstatus_information 1 ; Retain non-status information across program restarts
 notification_interval 0 ; Only send notifications on status change by default.
is_volatile 0
check_period 24x7
normal_check_interval 5
retry_check_interval 1
max_check_attempts 4
notification_period 24x7
notification_options w,u,c,r
contact_groups admins
register 0 ; 
DONT REGISTER THIS DEFINITION - ITS NOT A REAL SERVICE, JUST A TEMPLATE!
}

Well, as we can see, there is a lot we can define service-wise, just so much to clutter a service definition so we hide the complexity in a template and recall it, like sourcing a library. Once the template is imported, all its definition will apply to the service that called it and if we want to modify some values from the template, we just write them in the service definition with the new values, because if we have multiple definitions with the same name and different values, the closest to the final object wins. So, a definition at the service level wins over a definition in the template. We will not explain all the definitions in the template, as they are not useful for our goal, since our script will rely on the generic service definition without any alterations.

Let's go back to the service definition and have a look at the second line host_name localhost. We already mentioned the fact that each service check must refer to one (or more) host, so here is where we see what host this service applies to. We could also have used hostgroup_name name_of_the_hostgroup.

To apply a single check to multiple hosts enclosed in a host group definition. Let's move onto the service_description ssh. As for the command definition, this is the handle used to refer to this service definition throughout Nagios:

check_command check_ssh

This is where we call the command definition passing optional arguments. In our predefined configuration, there are no parameters to give to the command, so nothing special. With this line, the service definition recalls the syntax defined in the command definition called by the handled, and optionally passes some arguments to it. All the configurations for services, commands, hosts, and templates follow the same structure:

define object {
definitions_1 
definitions_2 
definitions_n
}

You can then have the different definitions on one files closed in their snippets.

We just saw how the ssh check works in Nagios, but it actually does not work, since it throws us an error. What we would need is a way to change the port that the service is being checked on. How do we accomplish this task? By simply bearing in mind that the actual plugin is the star here, it will drive all our efforts, so let's invoke it and see what it has to say. Let's have a look at the command line definition:

command_line /usr/lib/nagios/plugins/check_ssh '$HOSTADDRESS$'

From here, we know where the script is, so let's call it:

root:~$ /usr/lib/nagios/plugins/check_ssh 
check_ssh: Could not parse arguments
Usage:
check_ssh [-4|-6] [-t <timeout>] [-r <remote version>] [-p <port>] <host>

What we see here is that the script accepts some arguments and options on the command line, but each script is usually coded with a full help message invoked by a -h option:

root:~$ /usr/lib/nagios/plugins/check_ssh -h
check_ssh v2.1.1 (monitoring-plugins 2.1.1)
Copyright (c) 1999 Remi Paulmier <[email protected]>
Copyright (c) 2000-2007 Monitoring Plugins Development Team
<[email protected]>

Try to connect to an SSH server at specified server and port

Usage:
check_ssh [-4|-6] [-t <timeout>] [-r <remote version>] [-p <port>] <host>

Options:
-h, --help
Print detailed help screen
-V, --version
Print version information
--extra-opts=[section][@file]
Read options from an ini file. See https://www.monitoring-plugins.org/doc/extra-opts.html
for usage and examples.
-H, --hostname=ADDRESS
Host name, IP Address, or unix socket (must be an absolute path)
-p, --port=INTEGER
Port number (default: 22)
-4, --use-ipv4
Use IPv4 connection
-6, --use-ipv6
Use IPv6 connection
-t, --timeout=INTEGER
Seconds before connection times out (default: 10)
-r, --remote-version=STRING
Warn if string doesn't match expected server version (ex: OpenSSH_3.9p1)
-P, --remote-protocol=STRING
Warn if protocol doesn't match expected protocol version (ex: 2.0)
-v, --verbose
Show details for command-line debugging (output may be truncated by
the monitoring system)

Send email to [email protected] if you have questions regarding use of this software. To submit patches or suggest improvements, send email to [email protected]

Let's keep in mind this help, because it is something we will have to implement in our plugin. Anyway, what we see, among other options is that we can actually change the port the service is being checked on using the option: -p.

Let's check where our ssh server is listening for connections:

root:~$ netstat -tapn | grep ssh
tcp 0 0 0.0.0.0:1472 0.0.0.0:* LISTEN 685/sshd 
tcp6 0 0 :::1472 

Now we know that our ssh daemon is listening on port 1472. So we have to make a manual check to be sure on how to invoke the plugin with the new parameters and values:

root:~$ /usr/lib/nagios/plugins/check_ssh -H localhost -p 1472
SSH OK - OpenSSH_6.7p1 Debian-5+deb8u3 (protocol 2.0) | time=0.011048s;;;0.000000;10.000000

It worked we processed -H localhost to identify which host we are executing our check against and -p 1472 to query the correct port for this ssh daemon configuration. Now, let's pay attention to the reply from the plugin:

SSH OK - OpenSSH_6.7p1 Debian-5+deb8u3 (protocol 2.0) | time=0.011048s;;;0.000000;10.000000

This is the standard structure of the message provided by a Nagios plugin:

1. The name of the service (SSH).
2. Service status (OK).
3. Message given by the service being checked (or a message we crafted ourselves).

Then there is something we never saw before:

| time=0.011048s;;;0.000000;10.000000

This is a pipe, followed by one or more labels, time in our example, and some values usually related to how the service is working. Whatever is written, it is not a Nagios concern, since it will not process this part of the output line. These values are there for third-party applications such as pnp4nagios or Nagios graph to process them and eventually draw out some performance graphics.

We will see later how a graph for a service looks like, now let's remember one thing: the output of a plugin is usually one line long and even though you have a multiline output, it is always better to stick to a simple message.

Now, let's go back to the definition of the SSH service check, and let's see how to modify it to enable a different port check. This is the check_ssh command that we have already seen:

# 'check_ssh' command definition
define command{
command_name check_ssh
command_line /usr/lib/nagios/plugins/check_ssh '$HOSTADDRESS$'
}

To enable the definition of an arbitrary port check, we have to modify the command_line row so that it will accept the new -p parameter with an argument:

# 'check_ssh' command definition
define command{
command_name check_ssh
command_line /usr/lib/nagios/plugins/check_ssh '$HOSTADDRESS$' -p $ARG1$
}

What we did is simple: we just added a -p followed by $ARG1$. What is this new bit? In Nagios, you can pass whatever arguments you want to a script, and you refer to them using a positional variable. Think of $ARG1$ as $1 for a standard bash script; it identifies the first argument passed to the command line. Bear in mind that options like -p are not counted as arguments. So $ARG2$ will be the second positional argument, $ARG3$ the third, and so on. Do not forget the leading and trailing dollar signs. So, we modified the way Nagios can call the plugin, and now we can pass it an extra argument. What is left is to actually provide the extra argument to the script; this is done by modifying the service definition for ssh. We previously had this:

# check that ssh services are running
define service {
use generic-service
host_name localhost
service_description SSH
check_command check_ssh
}

This definition must be modified so that we can store and pass the port number to the command, so this is how we do it:

# check that ssh services are running
define service {
use generic-service
host_name localhost
service_description SSH
check_command check_ssh!1472
}

The exclamation mark (!) after the command name is a standard field separator and identifies the different positional arguments passed to the plugin. Let's make an example modifying the command_line of ssh to accept it:

-p 1472
-4
-P 2.0
-t 30

We must modify the command line to accept five parameters instead of one:

# 'check_ssh' command definition
define command{
command_name check_ssh
command_line /usr/lib/nagios/plugins/check_ssh '$HOSTADDRESS$' -p $ARG1$ -$ARG2$ -r $ARG3$ -P $ARG4$ -t $ARG5$
}

The modification is quite straightforward, we just wrote down all the switches and their arguments using the positional $ARGn$ variables. Now that the command line is ready to accept the new values, we must fill in the placeholders:

# check that ssh services are running
define service {
use generic-service
host_name localhost
service_description SSH
check_command check_ssh!1472!4!2.0!30
}

Not so complicated; each argument must be written in the order expected by the command line:

One thing to bear in mind is that the standard macros do not play a positional parameters so they do not have to be taken in to account when counting the slot indexes.

Now that we have all the bits in order, with the right switches and values, we have to write our new configuration down. Okay, but where? The location of the configuration files differs from distribution to distribution and the way the files are fragmented too: some distributions have commands and service definitions inside a host file along with the host definition, some others have fragmented in single files. How do we deal with it? Let the Nagios process tell you how it reads the information:

root:~$ ps ax | grep nagios
803 ? SNs 0:02 /usr/sbin/nagios3 -d /etc/nagios3/nagios.cfg
2502 pts/1 S+ 0:00 grep nagios

A ps command shows us that Nagios is reading its main configuration directives from/etc/nagios3/nagios.cfg. So, it is worth having a look at it:

# Commands definitions
cfg_file=/etc/nagios3/commands.cfg

# Debian also defaults to using the check commands defined by the debian
# nagios-plugins package
cfg_dir=/etc/nagios-plugins/config

# Debian uses by default a configuration directory where nagios3-common,
# other packages and the local admin can dump or link configuration
# files into.
cfg_dir=/etc/nagios3/conf.d

# OBJECT CONFIGURATION FILE(S)
# These are the object configuration files in which you define hosts,
# host groups, contacts, contact groups, services, etc.
# You can split your object definitions across several config files
# if you wish (as shown below), or keep them all in a single config file.

# You can specify individual object config files as shown below:
#cfg_file=/etc/nagios3/objects/commands.cfg
#cfg_file=/etc/nagios3/objects/contacts.cfg
#cfg_file=/etc/nagios3/objects/timeperiods.cfg
#cfg_file=/etc/nagios3/objects/templates.cfg

# Definitions for monitoring a Windows machine
#cfg_file=/etc/nagios3/objects/windows.cfg

# Definitions for monitoring a router/switch
#cfg_file=/etc/nagios3/objects/switch.cfg

# Definitions for monitoring a network printer
#cfg_file=/etc/nagios3/objects/printer.cfg

# You can also tell Nagios to process all config files (with a .cfg
# extension) in a particular directory by using the cfg_dir
# directive as shown below:

#cfg_dir=/etc/nagios3/servers
#cfg_dir=/etc/nagios3/printers
#cfg_dir=/etc/nagios3/switches
#cfg_dir=/etc/nagios3/routers

This is a standard section in the Nagios main configuration file, and you will find it in each and every installation, so pay attention to the lines that are not commented out by a # character:

# Commands definitions
cfg_file=/etc/nagios3/commands.cfg

# Debian also defaults to using the check commands defined by the debian
# nagios-plugins package
cfg_dir=/etc/nagios-plugins/config

# Debian uses by default a configuration directory where nagios3-common,
# other packages and the local admin can dump or link configuration
# files into.
cfg_dir=/etc/nagios3/conf.d

So, from the main configuration file, we can see that the configurations are stored in one file and two directories. Since we are dealing with a command plugin modification, we start from cfg_dir=/etc/nagios-plugins/config.

Looking for a file that cold bear an ssh configuration, let's move to root:~$ cd /etc/nagios-plugins/config and grep for ssh in each file:

root:~$ egrep -lr ssh *
disk.cfg
ssh.cfg

Just egrep -l will print only the names of the files where a match has been found; if you are not sure and want to see the actual matched line, use -ir instead of -lr , and you will see a lot more information. Anyway, between the two files, it seems pretty clear that the one we will have to modify is ssh.cfg.

Let's open it and go to the end of the file, adding our new command definition:

define command{
command_name check_ssh_arguments
command_line /usr/lib/nagios/plugins/check_ssh '$HOSTADDRESS$' -p $ARG1$ -$ARG2$ -P $ARG3$ -t $ARG4$
}

As appears evident, we changed command_name; since there cannot be two command definitions with the same handle, we just chose something unique for our purposes. It will not be displayed to users, so it does not need to be fancy, just useful and meaningful. Let's save the file and proceed to define a new service configuration; from the main configuration file, it seems quite clear we have to look into cfg_dir=/etc/nagios3/conf.d.

So, let's move to this directory: root:~$ cd /etc/nagios3/conf.d, and grep for ssh again:

root:~$ egrep -lr ssh *
hostgroups_nagios2.cfg
services_nagios2.cfg

In this case, it is not clear what is bearing what, so an extended grep will be handy:

root:~$ egrep -ir ssh *
hostgroups_nagios2.cfg:# A list of your ssh-accessible servers
hostgroups_nagios2.cfg: hostgroup_name ssh-servers
hostgroups_nagios2.cfg: alias SSH servers
services_nagios2.cfg:# check that ssh services are running
services_nagios2.cfg: hostgroup_name ssh-servers
services_nagios2.cfg: service_description SSH
services_nagios2.cfg: check_command check_ssh

Now, it is clear that hostgroups_nagios.cfg bears the configurations related to the host groups, and among those, the configuration of the group of hosts that are being checked for the ssh service. The second file, services_nagios2.cfg, holds the configuration for the ssh service check, so let's open it:

# check that ssh services are running
define service {
hostgroup_name ssh-servers
service_description SSH
check_command check_ssh
use generic-service
notification_interval 0 ; set > 0 if you want to be re-notified
}

Here is the ssh service check configuration we were looking for. In a production environment, we would have to estimate the impact of our configuration, since if we now modify this definition, it will apply to all the servers we are checking against. Notice hostgroup_name ssh-servers, we are checking a group of servers being this group populated by one or one thousand servers is not important.

In a production or a staging scenario, we would have to see which servers we are checking for the ssh service, understand if our modifications will have some odd effects on some of them, and if so, tear these servers out of the new check and create a special group for them using the old definition for them. In our case, since this is a demo installation and has localhost as the only group member, we can just modify the existing configuration and go with it:

# check that ssh services are running
define service {
hostgroup_name ssh-servers
service_description SSH
check_command check_ssh_arguments!1472!4!2.0!30
use generic-service
notification_interval 0 ; set > 0 if you want to be renotified
}

This definition is quite similar to the one we crafted before; just here we are in a real scenario. Nagios is configured to apply this check on a hostgroup instead of a single server, but since the hostgroup is made by one server only, the localhost, the two definitions have the same scope. What we are left with is to force Nagios to reload the definitions so that our new configurations will be read by the core. A reload or restart will suffice:

service nagios3 reload

Now edit the /etc/nagios3/nagios.cfg file and enable the following configuration bit:

check_external_commands=1

With 0 meaning disabled and 1 enabled, we just told Nagios to accept external command, so we can reload the configuration service nagios3 reload, go to the service name, and enter the server details page. Here we just have to click on Re-schedule the next check of this service.

Let's select Force check and commit; a new check will be forced whatever schedule is at play.

In the Debian and Ubuntu standard Nagios installation, you could face Error: Could not stat() command file '/var/lib/nagios3/rw/nagios.cmd'!

When you try to force a check. You can solve it with the following procedure:

service nagios3 stop
dpkg-statoverride --update --add nagios www-data 2710 /var/lib/nagios3/rw
dpkg-statoverride --update --add nagios nagios 751 /var/lib/nagios3
service nagios3 start

If you face any issues with your plugin, enable debug mode in /etc/nagios3/nagios.cfg by setting the following configuration bit:

debug_level=-1
debug_verbosity=2

This will generate a lot of information written in the debug file, which in our installation is in /var/log/nagios3/nagios.debug file, which is important to understand what is going on, but they will slow down the system a bit, so we must keep the debug on only for the time it is needed, then we must revert to the normal logging.

A Nagios reload will enforce the activation of the new configuration. But let's have a look at what the debug log has to say about our new modified command:

[1489655900.213562] [016.0] [pid=13954] Checking service 'SSH' on host 'localhost'...
[1489655900.213602] [2320.2] [pid=13954] Raw Command Input: /usr/lib/nagios/plugins/check_ssh -p $ARG1$ -$ARG2$ -P $ARG3$ -t $ARG4$ '$HOSTADDRESS$'
[1489655900.213787] [2320.2] [pid=13954] Expanded Command Output: /usr/lib/nagios/plugins/check_ssh -p $ARG1$ -$ARG2$ -P $ARG3$ -t $ARG4$ '$HOSTADDRESS$'
[1489655900.213825] [2048.1] [pid=13954] Processing: '/usr/lib/nagios/plugins/check_ssh -p $ARG1$ -$ARG2$ -P $ARG3$ -t $ARG4$ '$HOSTADDRESS$''
[1489655900.213839] [2048.2] [pid=13954] Processing part: '/usr/lib/nagios/plugins/check_ssh -p '
[1489655900.213846] [2048.2] [pid=13954] Not currently in macro. Running output (37): '/usr/lib/nagios/plugins/check_ssh -p '
[1489655900.213906] [2048.2] [pid=13954] Uncleaned macro. Running output (41): '/usr/lib/nagios/plugins/check_ssh -p 1472'
[1489655900.213911] [2048.2] [pid=13954] Just finished macro. Running output (41): '/usr/lib/nagios/plugins/check_ssh -p 1472'
[1489655900.213921] [2048.2] [pid=13954] Not currently in macro. Running output (43): '/usr/lib/nagios/plugins/check_ssh -p 1472 -'
[1489655900.214051] [2048.2] [pid=13954] Uncleaned macro. Running output (44): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4'
[1489655900.214064] [2048.2] [pid=13954] Just finished macro. Running output (44): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4'
[1489655900.214074] [2048.2] [pid=13954] Not currently in macro. Running output (48): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P '
[1489655900.214109] [2048.2] [pid=13954] Uncleaned macro. Running output (51): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0'
[1489655900.214114] [2048.2] [pid=13954] Just finished macro. Running output (51): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0'
[1489655900.214123] [2048.2] [pid=13954] Not currently in macro. Running output (55): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t '
[1489655900.214161] [2048.2] [pid=13954] Uncleaned macro. Running output (57): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30'
[1489655900.214175] [2048.2] [pid=13954] Just finished macro. Running output (57): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30'
[1489655900.214200] [2048.2] [pid=13954] Not currently in macro. Running output (59): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30 ''
[1489655900.214263] [2048.2] [pid=13954] Uncleaned macro. Running output (68): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30 '127.0.0.1'
[1489655900.214276] [2048.2] [pid=13954] Just finished macro. Running output (68): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30 '127.0.0.1'
[1489655900.214299] [2048.2] [pid=13954] Not currently in macro. Running output (69): '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30 '127.0.0.1''
[1489655900.214310] [2048.1] [pid=13954] Done. Final output: '/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30 '127.0.0.1''

It is clear how Nagios builds up the command line piece by piece, so we can understand how it parses all our definitions and how it allocates the value we passed through the service definition itself. If we take the last row, copy and paste the command line of the final output, and execute it on the server, we are monitoring the service onto this:

/usr/lib/nagios/plugins/check_ssh -p 1472 -4 -P 2.0 -t 30 '127.0.0.1'
SSH OK - OpenSSH_6.7p1 Debian-5+deb8u3 (protocol 2.0) | time=0.015731s;;;0.000000;30.000000

We get the service checked, a status (OK), and performance data as well:

Performance data is a useful bit of information that can give you some ready-made projections on service running pattern, once you plot it on a chart. With such data and charts we can do this:

• Adopt a service capacity management strategy: Since we can easily forecast the consumption curve for a service, we can predict when it will be time to upgrade the hardware needed to provision it.
• Find out usage patterns: A service can be used in a uneven pattern. For example, a company mail server is most used during office hours and less during the night or weekends; disk space for a data warehouse server is used more during the data consolidation batches than during other moments. So, a service that appears adequate while you are checking it can be under-equipped in other moments: a graph will show you how the usage curve moves over time.
• Find failures at a glance: Watching at the gaps into the graph, you can easily spot service interruptions, and selecting the piece of graph you want to inspect can be exploded in detail.
• Create fancy reports for management: Seems a joke but busy management prefers to have a comprehensive glance at services than pages and pages of numeric data.

So, let's quickly see how to install one of this graphing tools. Since this is not a book on Nagios, we will not go into much detail, but we are going to see only what is needed to enable this third-party service and have our plugin performance data graphed.

Let's start editing the /etc/nagios3/nagios.cfg file.

Look for the following snippets of configuration and modify them so the final result will be this:

process_performance_data=1
host_perfdata_command=process-host-perfdata
service_perfdata_command=process-service-perfdata

We enabled the performance processing data and defined the name of the commands, which will deal with them; the next logical step is to define the commands we just pointed out. Edit the /etc/nagios3/commands.cfg file and add the following snippet:

# ‘process-host-perfdata' command definition
define command{
command_name process-host-perfdata
command_line /usr/bin/perl /usr/lib/pnp4nagios/libexec/process_perfdata.pl -d HOSTPERFDATA
}
# ‘process-service-perfdata' command definition
define command{
command_name process-service-perfdata
command_line /usr/bin/perl /usr/lib/pnp4nagios/libexec/process_perfdata.pl
}

Comment out any pre-existing snippets sporting command_name process-host-perfdata and command_name process-service-perfdata.

We will use the new ones, as the old ones are useless for our purposes, so again comment them out. Now that we have the command in place and the data will be processed as intended, we have to tell Nagios how to trigger the chart visualization. So, time to edit /etc/nagios3/conf.d/services_nagios2.cfg and modify the previously edited ssh service check configuration so that now it appears as this:

# check that ssh services are running
define service {
hostgroup_name ssh-servers
service_description SSH
check_command check_ssh_arguments!1967!4!2.0!30
action_url /pnp4nagios/index.php/graph?host=$HOSTNAME$&srv=$SERVICEDESC$
use generic-service
notification_interval 0 ; set > 0 if you want to be renotified
}

We added an action URL configuration line so that Nagios will draw a small clickable icon close to the service name. So, let's restart Nagios and go to the service page just to find out something new:

The icon is clickable, so we just click it and the result is similar to the one in the next screenshot:

From now on, our performance data will be graphed, so our Nagios environment is ready to host our first Nagios plugin.