In some cases, there might be custom data sources where none of the previously discussed methods would work sufficiently well. A script could run for a very long time, or we could have a system without the Zabbix agent but with a capability to push data. Zabbix offers a way to send data to a special item type, Zabbix trapper, using a command line utility, Zabbix sender. The easiest way to explain how it works might be to set up a working item like that—let's navigate to Configuration | Hosts, click on Items next to A test host, and click on Create item, then fill in the following:
- Name:
Amount of persons in the room - Type: Zabbix trapper
- Key:
room.persons
When you are done, click on the Add button at the bottom. We now have to determine how data can be passed into this item, and this is where zabbix_sender comes in. On the Zabbix server, execute the following:
$ zabbix_sender --help
We won't reproduce the output here, as it's somewhat lengthy. Instead, let's see which parameters are required for the most simple operation, sending a single value from the command line:
-zto specify the Zabbix server-sto specify the hostname, as configured in Zabbix-kfor the key name-ofor the value to send
Note that the hostname is the hostname in the Zabbix host properties—not the IP, not the DNS, not the visible name. Let's try to send a value then:
$ zabbix_sender -z 127.0.0.1 -s "A test host" -k room.persons -o 1
This command should succeed and show the following output:
info from server: "processed: 1; failed: 0; total: 1; seconds spent: 0.000046" sent: 1; skipped: 0; total: 1
Let's send another value—again, using zabbix_sender:
$ zabbix_sender -z 127.0.0.1 -s "A test host" -k room.persons -o 2
This one should also succeed, and now we should take a look at Monitoring | Latest data over at the frontend. We can see that the data has successfully arrived and the change is properly recorded:
Now we could try being smart. Let's pass a different data type to Zabbix:
$ zabbix_sender -z 127.0.0.1 -s "A test host" -k room.persons -o nobody
We are now trying to pass a string to the Zabbix item even though in the frontend, its data type is set to an integer:
info from server: "processed: 0; failed: 1; total: 1; seconds spent: 0.000074" sent: 1; skipped: 0; total: 1
Zabbix didn't like that, though. The data we provided was rejected because of the data type mismatch, thus it is clear that any process that is passing the data is responsible for the data contents and formatting.
Now, security-concerned people would probably ask—who can send data to items of the trapper type? A zabbix_sender can be run on any host by anybody, and it is enough to know the hostname and item key. It is possible to restrict this in a couple of ways—for one of them, see Configuration | Hosts, click on Items next to A test host and click on Amount of persons in the room in the NAME column. Look at one of the last few properties, Allowed hosts. We can specify an IP address or DNS name here, and any data for this item will be allowed from the specified host only:
Several addresses can be supplied by separating them with commas. In this field, user macros are supported as well. We discussed user macros in Chapter 8, Simplifying Complex Configuration with Templates.
Another option to restrict who can send the data to trapper items is by using the authentication feature with PSK or SSL certificates. That is discussed in Chapter 20, Encrypting Zabbix Traffic.
So far, we specified all the information that zabbix_sender needs on the command line. It is also possible to automatically retrieve some of that information from the agent daemon configuration file. Let's try this (use the correct path to your agent daemon configuration file):
$ zabbix_sender -c /usr/local/etc/zabbix_agentd.conf -k room.persons -o 3
This succeeds, because we specified the configuration file instead of the Zabbix server address and the hostname—these were picked up from the configuration file. If you are running zabbix_sender on many hosts where the Zabbix agent also resides, this should be easier and safer than parsing the configuration file manually. We could also use a special configuration file for zabbix_sender that only contains the parameters it needs.
The approach we used allows us to send one value every time we run zabbix_sender. If we had a script that returned a large number of values, that would be highly inefficient. We can also send multiple values from a file with zabbix_sender. Create a file like this anywhere—for example, in /tmp/:
"A test host" room.persons 4 "A test host" room.persons 5 "A test host" room.persons 6
Each line contains the hostname, item key, and value. This means that any number of hosts and keys can be supplied from a single file.
The flag for supplying the file is -i. Assuming a filename of sender_input.txt, we can run the following:
$ zabbix_sender -z 127.0.0.1 -i /tmp/sender_input.txt
That should send all three values successfully:
info from server: "processed: 3; failed: 0; total: 3; seconds spent: 0.000087" sent: 3; skipped: 0; total: 3
When sending values from a file, we could still benefit from the agent daemon configuration file:
$ zabbix_sender -c /usr/local/etc/zabbix_agentd.conf -i /tmp/sender_input.txt
In this case, the server address would be taken from the configuration file, while hostnames would still be supplied from the input file. Can we avoid that and get the hostname from the agent daemon configuration file? Yes, that is possible by replacing the hostname in the input file with a dash like this:
- room.persons 4 "A test host" room.persons 5 - room.persons 6
In this case, the hostname would be taken from the configuration file for the first and the third entry, while still overriding that for the second entry.
When there's a need to send lots of values constantly, one might wish to avoid repeatedly running the zabbix_sender binary. Instead, we could have a process write to a file new entries without closing the file, and then have zabbix_sender read from that file. Unfortunately, by default, values would be sent to the server only when the file is closed—or every 250 values received. Fortunately, there's also a command line flag to affect this behavior. Flag -r enables a so-called real-time mode. In this mode, zabbix_sender reads new values from the file and waits for 0.2 seconds. If no new values come in, the obtained values are sent. If more values come in, it waits for 0.2 seconds more, and so on up to 1 second. If there's a host that's constantly streaming values to the Zabbix server, zabbix_sender would connect to the server once per second at most and send all the values received in that second in one connection. Yes, in some weird cases, there could be more connections—for example, if we supplied one value every 0.3 seconds exactly.
If sending a huge number of values and using a file could became a performance issue, we could even consider a named pipe in place of the file—although that would be a quite rare occurrence.
The data that we sent so far was considered to be received at that exact moment—the values had the timestamp assigned by the server when it got them. Every now and then, there's a need to send values in batches for a longer period of time, or import a backlog of older values. This can be easily achieved with zabbix_sender—when sending values from a file, it supports supplying a timestamp. When doing so, the value field in the input file is shifted to the right and the timestamp is inserted as the third field. For a quick test, we could generate timestamps 1, 2, and 3 days ago:
$ for i in 1 2 3; do date -d "-$i day" "+%s"; done
Take the resulting timestamps and use them in a new input file:
- room.persons 1462745422 11 "A test host" room.persons 1462659022 12 - room.persons 1462572622 13
With a file named sender_input_timestamps.txt, we would additionally use the -T flag to tell zabbix sender that it should expect the timestamps in there:
$ zabbix_sender -c /usr/local/etc/zabbix_agentd.conf -T -i /tmp/sender_input_timestamps.txt
All three values should be sent successfully.
Looking at the graph or latest values for this item, it is probably slightly messed up. The timestamped values we just sent in are likely to be overlapping in time with the previous values. In most cases, sending in values normally and with timestamps for the same item is not suggested.
Note
If the Zabbix trapper items have triggers configured against them, timestamped values should only be sent with increasing timestamps. If values are sent in a reversed or chaotic older-newer-older order, the generated events will not make sense.
If data is sent in for a host which is in a no-data maintenance, the values are also discarded if the value timestamp is outside the current maintenance window. Maintenance was discussed in Chapter 5, Managing Hosts, Users, and Permissions.