One of the many other things Zabbix can do is monitor log files. In this recipe, we will show you how to test your log files with Zabbix for certain patterns.
For this recipe, we need a Zabbix server without agent installed on the server and configured. We also need Zabbix super administrator access.
Let's say we want to monitor the /var/log/messages file on our OS.
- First thing we need to do is make sure Zabbix has access to the file:
# ll /var/log/messages -rw-------. 1 root root 324715 Jan 20 18:54 /var/log/messages
- As we can see, only the user
roothas read and write access to this file. - Our next step is to add Zabbix to a new group example,
adm; then later we can give this group access to our log file:# usermod -a -G adm zabbix - Next step is to make the file readable for the group:
# chmod g+r /var/log/messages - Now we only have to add the file messages to the group
adm:# chgrp adm /var/log/messages - Now when we check, our permissions should look like this on the
/var/log/messagesfile:# ll /var/log/messages -rw-r-----. 1 root adm 327617 Jan 20 19:11 /var/log/messages
- Our next step is to add an item in our Zabbix server to monitor this file. Go to Configuration | Hosts | and select Item for our Zabbix server. (Or better still, add it to a template that is linked to our Zabbix server).
- Click Create item to create a new item.
- Give a new Name to our item, example,
Errors in /var/log/messages. - Select
Zabbix agent (active)as Type. - Add the following Key:
log[/var/log/messages,error]. - Type of information should be
Log. - Update interval can be set to
1. - Now save your item.
- On the Zabbix server console, type:
# logger error - This will generate an error in our log file, so we can go now to Configuration | Latest data and look how the log file was monitored by our Zabbix server.
- Create a Trigger so that we would be alarmed. We go to Configuration | Hosts | Triggers and click on Create trigger.
- Give a descriptive name.
- Add the following expression :
{<template or server>:log[/var/log/messages,error].logsource(error)}=0so that you get notifications when we get errors in the/var/log/messagesfile.
SELinux could be messing with you; so make sure to temporarily disable SELinux to make sure that this is not the problem. In case it is, a rule should be created for this.
The problem with logfile monitoring is that entries in log files do not have a status. If an entry in the log file indicates an error, there is usually no entry indicating that the error has been corrected. So in this case, the trigger will always retain the status error. We have to force Zabbix to update the status and this can be done with the nodata() function. In this case, we have to rewrite our previous trigger like this:
{<template or host>:log[/var/log/messages,error].nodata(300)}=0In this case, we get an alarm when there is an error in the log file and Zabbix will reset it's status after 120 seconds:
In case you want to work with logrotate option, it is very much possible with Zabbix, except that we would have to use logrt option instead of log option.
Zabbix can look in files for certain keywords; for this, Zabbix needs to have read permissions on those files. In this example, we added Zabbix to the adm group. Then we added our log file to this group and gave the group read permissions. Now by creating the proper item, Zabbix was able to look into the file for out keyword error. With the command logger, we were able to send the command error to our log file and Zabbix picked it up.
Later we saw how it was possible to create the correct trigger for this, and what the possible problem could be with the entry not having a status. To solve this problem, we made use of the nodata function. This function makes it possible for Zabbix to monitor our log file and reset it's status back to normal if no new errors were received for 300 seconds. Of course, in this case you need to be sure that Zabbix is configured to send email, SMS, and so on, else there is a chance that you will not get any notification about the error.
- There are more options that can be set for a full overview. I suggest you read the following page:
https://www.zabbix.com/documentation/2.4/manual/config/items/itemtypes/log_items.