Another method is to tap into existing knowledge, through the Zabbix database. As the database already holds information on host/IP address relationships, we can simply look up the corresponding hostname. Let's modify snmptraps.sh so that all traps coming from hosts defined in Zabbix end up in an snmptraps item for that specific host, but other traps are collected in the generic snmptraps host instead.

Start by modifying /home/zabbix/bin/snmptraps.sh and adding two lines:

oid=`echo $oid|cut -f11 -d'.'` 
community=`echo $community|cut -f2 -d'"'` 
zabbixhost=$(HOME=/root mysql -N -e "select host from zabbix.hosts left join zabbix.interface on zabbix.hosts.hostid=zabbix.interface.hostid where ip='$hostname' order by 'hostid' limit 1;" 2>/dev/null) 
[[ $zabbixhost ]] && HOST=$zabbixhost 
str="$hostname $address $community $enterprise $oid" 
$ZABBIX_SENDER $ZABBIX_SERVER $ZABBIX_PORT -s "$HOST" -k "$KEY" -o "$str" 

So what do these do?:

• The first line queries the MySQL database and checks whether a host is defined with the same IP as the trap source. If it is, the Zabbix host variable gets the hostname, as defined in Zabbix, assigned. Returned results are sorted by host ID and only the first match is taken. Hence, if there are multiple hosts with the same IP address (which is perfectly fine in Zabbix), only the oldest entry is selected. Any error output is discarded (redirected to /dev/null), so in case of a database misconfiguration, traps are not lost but end up in the generic trap-handling host.
• The second line simply sets the host used for sending data to Zabbix to the entry returned from the database, if it exists.
• But what's that HOME variable in the first line? The mysql command used there does not specify user, password, or any other connection information, so for the command to succeed, it would have to get this information from somewhere. For MySQL, this information can be placed in the .my.cnf file located in the user's HOME directory. Given that snmptrapd runs as root, but services often do not get all the environment variables normal logins do, we are directing further commands to look in /root for that file.

This means we're not done yet; we have to create the /root/.my.cnf file and fill it with the required information. As root, create /root/.my.cnf and place the following content in it:

[client] 
user=zabbix 
password=mycreativepassword 

For the password, use the same one you used for the Zabbix server and frontend (if you don't remember this password, you can look it up in zabbix_server.conf).

Now, we should prepare for trap receiving on the Zabbix side.

Open Configuration | Hosts, click on Items next to Another host, and then click on the Create item button. Enter these values:

• Name: snmptraps
• Type: Zabbix trapper
• Key: snmptraps
• Type of information: Character

When you are done, click on the Add button at the bottom.

Before we send a test trap, let's do one more thing: make sure that snmptrapd does not perform reverse lookups on received traps. While that might slightly decrease the prettiness of the data, we want to keep this script simple for now and this will also improve performance a bit. To do this, add the -n flag for snmptrapd to the startup scripts and restart it. This procedure is distribution specific.

Finally, we are ready to test our tricky setup. From Another host, execute this:

$ snmptrap -Ci -v 2c -c public <Zabbix server> "" "NET-SNMP-MIB::netSnmpExperimental" NET-SNMP-MIB::netSnmpExperimental s "test"  

Replace <Zabbix server> with the IP or DNS name of the Zabbix server. This command should complete without any error messages.

Back in the frontend, navigate to Monitoring | Latest data:

Great, snmptrap instances are now successfully sorted by host, if present.

If this trap was not sorted properly and still went into the snmptraps host, it could be caused by different output in some Net-SNMP versions. Instead of passing the IP address or hostname of the incoming connection as the first value, they pass a string like this:

UDP: [192.168.56.11]:56417->[192.168.56.10]:162 

In that case, try adding another line before the zabbixhost assignment:

oid=`echo $oid|cut -f11 -d'.'` 
community=`echo $community|cut -f2 -d'"'` 
hostname=$(echo "$hostname" | awk -F'[][]' '{print $2}') 

It will extract the first string enclosed in square brackets from the hostname variable. After making this change to the script, send the trap again.

That took us some time to set up, but now it's very simple. If we want traps from some host to be handled by a specific host, we create that host and an snmptraps item for it. All other traps go to the generic snmptraps host and snmptraps item.

But what about item lookup? The database holds information on item keys as well, so perhaps we could try using that.

We need to retrieve the item key from any database field based on the information received in the trap. As traps include SNMP OIDs, they are the best candidates to map traps against items. Now, the OID can be in numeric or textual form. In the Zabbix configuration, we have two fields that could be used:

• Name: While pretty much a free-form field, it is a friendly name, so we'd better keep it human-readable.
• Key: This field has more strict rules on the characters it accepts, but OIDs should be fine. While not used by humans much, this field is still referred to in the trigger expressions.

That means we will use the Key field. To keep it both short enough and somewhat human-readable, we'll set it to the last part of the received textual-form OID. As the trap will be received by snmptraps.sh, it will try to match the received OID to the item key and based on that decide where to send the data.

Remember that specific MIBs might have to be added to /etc/snmp/snmp.conf so that they are found by snmptrapd.

Again, as root, edit the /home/zabbix/bin/snmptraps.sh script. Replace the two lines we just added, so that it looks like this:

community=`echo $community|cut -f2 -d' '` 
enterprise=`echo $enterprise|cut -f2 -d' '` 
oid=`echo $oid|cut -f11 -d'.'` 
community=`echo $community|cut -f2 -d'"'` 
hostname=$(echo "$hostname" | awk -F'[][]' '{print $2}') 
zabbixhostid=$(HOME=/root mysql -N -e "select hosts.hostid,host from  
zabbix.hosts left join zabbix.interface on  
zabbix.hosts.hostid=zabbix.interface.hostid where ip='$hostname' order  
by 'hostid' limit 1;" 2>/dev/null) 
zabbixhost=$(echo $zabbixhostid | cut -d" " -f2-) 
[[ "$zabbixhost" ]] && { 
     zabbixid=$(echo $zabbixhostid | cut -d" " -f1) 
     trapoid=$(echo $oid | cut -d: -f3) 
     if [ "$trapoid" ]; then 
         zabbixitem=$(HOME=/root mysql -N -e "select key_ from  
zabbix.items where key_='$trapoid' and hostid='$zabbixid';" 2> /dev/null) 
         if [ "$zabbixitem" ]; then 
             HOST=$zabbixhost 
             KEY=$zabbixitem 
         fi 
     fi 
} 
[[ $KEY = snmptraps ]] && { 
     if [ "$(HOME=/root mysql -N -e "select key_ from zabbix.items where  
key_='snmptraps' and hostid='$zabbixid';" 2> /dev/null)" ]; then 
         HOST=$zabbixhost 
     fi 
} 
str="$hostname $address $community $enterprise $oid" 

Save the file. In functional terms, as regards our current configuration, it will work exactly the same as the previous version, with one minor improvement: if you look at the previous version carefully, you'll see it only checks for host availability, so if you created a host but forgot to create an item with the snmptraps key for it, the sent trap would be lost. This version will check whether an item with such a key exists for that host. If not, the generic host, snmptraps, will receive the trap.

Note that this is one benefit of the custom-script solution over the embedded Perl trap receiver we configured earlier. It is easier to have triggers for traps landing in this fallback host than checking for them in the Zabbix server log file.

Additionally, it will now check whether the host also has an item with a key, matching the last part of the OID received. A simple decision flow representation is shown in the following diagram:

To test this, send an SNMP trap from Another host (there is no need to restart snmptrapd):

$ snmptrap -Ci -v 2c -c public <Zabbix server> "" "NET-SNMP-MIB::netSnmpExperimental" NET-SNMP-MIB::netSnmpExperimental s "test"  

Replace <Zabbix server> with the Zabbix server's IP or DNS name. If you now check Monitoring | Latest data for Another host, the trap should be correctly placed in the snmptraps item for this host. A trap sent from any other host, including the Zabbix server, should be placed in the snmptraps host and snmptraps item—feel free to try this out. Previously, a trap sent from the Zabbix server would be lost, because the script did not check for the snmptraps item's existence—it would find the host and then try to push the data to this nonexistent item.

Let's try out our item mapping now:

1. Go to the Zabbix interface, Configuration | Hosts, click on Items next to Another host, and click on the Create item button. Fill in the following values:

• • Name: Experimental SNMP trap
  • Type: Zabbix trapper
  • Key: netSnmpExperimental
  • Type of information: Character

2. When you're done, click on the Add button at the bottom.

Again, send a trap from Another host:

$ snmptrap -Ci -v 2c -c public <Zabbix server> "" "NET-SNMP-MIB::netSnmpExperimental" NET-SNMP-MIB::netSnmpExperimental s "test"

In the frontend, look at Monitoring | Latest data. If all went right, this time the trap data should have been placed in yet another item—the one we just created:

Now, whenever we have a host that will be sending us traps, we will have to decide where we want its traps to go. Depending on that, we'll decide whether it needs its own host with an snmptraps item, or perhaps even individual items for each trap type.