In this recipe, you'll learn how to use the debugging log file in Nagios Core to get various kinds of process information from the running program, considerably more than available in the standard log_file. This is useful not just for debugging purposes when Nagios Core is doing something unexpected at runtime, but it is also useful to get a better idea of how the server is working in general and with your particular configuration.
You will need a Nagios Core server with version 4.0 or greater. You will need access to change the nagios.cfg file and to restart the server.
In this example, we'll simply log everything we possibly can and then explain how to refine the logging behavior if necessary in the How it works section.
We can enable very verbose debugging for our monitoring server as follows:
- Change to the configuration directory for Nagios Core. In the default installation, the location is
/usr/local/nagios/etc. Edit thenagios.cfgfile:# cd /usr/local/nagios/etc # vi nagios.cfg
- Look for the
debug_level,debug_verbosity, anddebug_filedirectives. Ensure that they are uncommented or add them to the end of the file if they don't exist, and then define them as follows:debug_level=-1 debug_verbosity=2 debug_file=/usr/local/nagios/var/nagios.debug
- Validate the configuration and restart the Nagios Core server:
# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg # /etc/init.d/nagios restart
- With this done, the
/usr/local/nagios/var/nagios.debugfile should start filling up with information about the running process quite quickly. You may find it instructive to watch it for a while withtail -f, which will show you the contents of the file as it updates:# tail -f /usr/local/nagios/var/nagios.debug [1446454473.296558] [016.1] [pid=2023] ** Async check result for host 'sparta.example.net' handled: new state=1 [1446454473.296565] [064.1] [pid=2023] Making callbacks (type 7)... [1446454473.296571] [064.2] [pid=2023] Callback #1 (type 7) return code = 0 [1446454473.296578] [4096.2] [pid=2023] ## 1 descriptors had input [1446454473.296584] [12288.1] [pid=2023] ## Polling 1500ms; sockets=6; events=13; iobs=0x2243140 [1446454474.798162] [4096.2] [pid=2023] ## 0 descriptors had input [1446454474.798204] [064.1] [pid=2023] Making callbacks (type 11)... [1446454474.798213] [12288.1] [pid=2023] ## Polling 1500ms; sockets=6; events=13; iobs=0x2243140
The debug_level directive specifies how much information (and of what kind) should be written to the debugging log. Here, we've used the value -1, which is a shortcut for specifying that all debugging information should be written to the debugging log file.
In practice, however, we often only want to get information about particular kinds of Nagios Core tasks. In this case, we can use OR values for debug_level to specify which ones.
The different kinds of debugging information can be specified with the following numbers:
1: This refers to function enter and exit debugging2: This refers to config debugging4: This refers to process debugging8: This refers to scheduled event debugging16: This refers to host and service check debugging32: This refers to notification debugging64: This refers to event broker debugging
In version 3.3 and later of Nagios Core, the following can also be specified:
128: This refers to external commands debugging256: This refers to commands debugging512: this refers to scheduled downtime debugging1024: This refers to comments debugging2048: This refers to macros debugging
Rather than using comma-separated values to specify more than one value, they are added together. For example, if we want to save the process and scheduled event debugging information and nothing else, we would use 4 + 8 = 12:
debug_level=12
We can turn the debugging off completely by changing debug_level back to 0, its default value.
Nagios Core generates a great deal of information at the highest level of debugging, with over 30 lines per second even on minimal configurations, so be careful not to leave this running permanently if you don't always need it as it can slowly fill a disk. You can avoid this situation using the max_debug_file_size directive to specify a maximum size in bytes for the file. For example, to restrict the file to one megabyte, we could set the directive as follows:
max_debug_file_size=1000000
Nagios Core will roll an existing debugging log by adding a .old extension when it exceeds this size and will start a new one. It will also automatically delete any previous .old logs when it does this.
- Viewing and interpreting notification history, Chapter 7, Using the Web Interface
- The Monitoring Nagios performance with Nagiostats recipe in this chapter