A solution could be some method that would send us the proxy buffer size, bypassing the buffer itself. Zabbix does not offer such a method, thus we will have to implement it ourselves. Before we do that, let's figure out how we could obtain information on the buffer size. For that, we will delve into the proxy database.

You might have to install the SQLite 3 package to get the sqlite3 utility.

On the proxy host, run the following:

$ sqlite3 /tmp/zabbix_proxy.db

The proxy keeps all of the collected values in a single table, proxy_history. Let's grab the last three collected values:

sqlite> select * from proxy_history order by id desc limit 3;
1659|28805|1546002875|0||0|0|0|660348055|0|0|0|0
1658|28804|1546002874|0||0|1546002874|0|658660672|0|0|0|0
1657|28799|1546002869|0||0|1|0|654297524|0|0|0|0

We will discuss other fields in a bit more detail in Chapter 19, Working Closely with Data, but for now, it is enough to know that the first field is a sequential ID. Still, how does the proxy know which values it has sent to the server already? Let's look at the IDs table:

sqlite> select * from ids where table_name='proxy_history';
proxy_history|history_lastid|1701  

The history_lastid value here is the last ID that has been synchronized to the server. On a busy proxy, you might have to run these statements really quickly to see the real situation, as new values will be constantly added and sent to the server. We can get the current buffer (unsent values) size with this:

sqlite> select (select max(proxy_history.id) from proxy_history)-nextid from ids where field_name='history_lastid';  

It will calculate the difference between the biggest ID and the history_lastid value. On our proxy, this will likely return 0 all the time.

Now we should put this in an item. The most important thing is to make sure this item is processed directly by the server, without involving the Zabbix proxy. We have several options:

• Passive agent item
• Active agent item
• Zabbix trapper item that is populated by zabbix_sender

For a passive agent, the server should query it directly. For an active agent, it should point at the Zabbix server. For the trapper item, zabbix_sender should be used to connect to the Zabbix server. In all three cases, the host should be assigned to be monitored by the Zabbix server. If we are using internal monitoring to collect proxy values in a dedicated host, a separate host will be needed to collect the buffer data. This way, we will avoid these values getting stuck in the proxy buffer.

For the agent items, we could use UserParameter, like this:

UserParameter=proxy.buffer,sqlite3 /tmp/zabbix_proxy.db "select (select max(proxy_history.id) from proxy_history)-nextid from ids where field_name='history_lastid';" 

As for the Zabbix trapper approach, it could be run from crontab or using any other method. The command would be similar to this:

zabbix_sender -z zabbix_server -s target_host -k item_key -o $(sqlite3 /tmp/zabbix_proxy.db "select (select max(proxy_history.id) from proxy_history)-nextid from ids where field_name='history_lastid';") 

Here, we use the basic zabbix_sender syntax, but the value is obtained from the SQLite query. See Chapter 10, Advanced Item Monitoring, for more information on UserParameters and zabbix_sender. The Zabbix trapper item would receive the same data as the internal buffer monitoring—the buffer size. The trigger would check for this buffer exceeding some threshold.

Note that all three methods are likely to result in some missing values for the buffer item—the values would not be available while the connection between the server and proxy is down. The active agent item approach would suffer less as it has an in-memory buffer, but it there might still be missing values. If it would be valuable to know how the buffer changed during the communication breakdown. This item could be used for the trigger and an internal item, as discussed earlier, for more complete buffer statistics.

Regarding triggers and dependencies, it is suggested to make the buffer trigger depend on the last access trigger. This way, hosts behind the proxy will be silenced if the proxy disappears completely, and when the proxy comes back with a large buffer, the buffer trigger will keep those hosts silent.