There are quite a few prerequisites to getting this recipe to work. This method is among the most powerful but also most complex methods of Nagios Core monitoring.

First of all, the recipe assumes some knowledge of the Simple Network Management Protocol (SNMP). Unfortunately, SNMP is not very simple, despite its name! You should be familiar with the concepts of SNMP checks and SNMP traps. The documentation for Net-SNMP (the implementation of SNMP used for this example) may help here, which you can find at http://www.net-snmp.org/docs/readmefiles.html.

On the same host as your Nagios Core server with version 4.0 or greater, you should have snmptrapd installed to collect trap information and snmptt, the SNMP Trap Translator, to filter useful information from the traps and submit the information to Nagios Core in a workable format. The documentation for SNMPTT is available at http://snmptt.sourceforge.net/docs/snmptt.shtml.

Both systems are free software and relatively popular, so check whether there are packages available for your particular system to save the hassle of compiling them from source. On Debian-derived systems, such as Ubuntu, for example, they are available in the snmpd and snmptt packages.

We will use an event handler called submit_check_result, which is available in the Nagios Core distribution. You will, therefore, need to have access to the original sources handy. If you have misplaced them, you can download them again from Nagios' website (https://www.nagios.org/downloads/).

It's also necessary to use values of host_name for your hosts that actually correspond to hostnames resolvable by DNS from the monitoring server. This is because when the SNMP trap is received by SNMPTT, the only way it can translate the trap's source IP address to a hostname is with DNS. The host_name value for your host might be crete.example.net, but the trap will arrive from an IP address like 192.0.2.27. The system will, therefore, need to be able to resolve this with a reverse DNS lookup. An easy way to test whether this is working is to use host(1) or dig(1):

$ host 192.0.2.27
27.2.0.192.in-addr.arpa domain name pointer crete.example.net.
$ dig -x 192.0.2.27 +short
crete.example.net.

Finally, you should, of course, actually have a device configured to send SNMP traps to your monitoring server, which in turn is configured to listen for SNMP traps with the snmpd daemon. I don't really want to encourage you to unplug one of your core switches to test this, so we'll generate a trap manually with snmptrap(1) on the monitored server to demonstrate the principle.

We can configure a new service to receive SNMP traps for an existing host using the following steps:

We now need to configure snmptrapd and snmpd to receive traps and call the submit_check_result script for us. Follow the given steps to do so:

With this done, we've confirmed that SNMP traps from the crete.example.net host can be received and processed by the olympus.example.net server. We can apply the same setup for other hosts that generate SNMP traps in our network by configuring them to send their traps to the Nagios Core monitoring server and adding appropriate handlers for the expected traps.

If this doesn't work, the first thing to check should be that your monitoring server is actually listening for checks on the relevant IP address, as snmptrap does not throw errors when it cannot deliver traps. On Debian-derived systems, you should verify that the snmptrapd process is actually running; it may require a change to /etc/default/snmptrapd and a restart of snmptrapd.

When an SNMP trap is generated and delivered to the monitoring server by whatever means, the snmpd daemon passes it to the snmptt program for processing.

The snmptt handler checks whether the event OID matches any of the traps for which it has defined events in snmptt.conf. In our example, the program finds a handler defined for the OID .1.3.6.1.6.3.1.1.5.3, which corresponds to the linkDown event, with the number of relevant interfaces as an additional argument in $1.

Using this information, the program fires the submit_check_result handler that we installed in the first part of the recipe, setting the state of the TRAP service to WARNING and including the information linkDown for interface 1, as specified by the final argument to submit_check_result in the EXEC handler. The service can be set to notify the appropriate contacts or contact groups, just as it would for an actively monitored service.

If a trap arrives on the Nagios Core server for a host that Nagios Core doesn't know about, even if it has an event handler defined for snmptt, Nagios Core will simply ignore it.

In order to clear the state of the service and return it to OK, we can simply schedule an active check for it from the web interface with the Force Check tick.

Because check_command is defined as check-host-alive, as long as the monitoring host is actually responding to PING, the service should assume the OK state: