The calculated items allowed us to write a specific formula, referencing exact individual items. This worked well for small-scale calculations, but the CPU Load item we created last would be very hard to create and maintain for dozens of hosts, and impossible for hundreds. If we want to calculate something for the same item key across many hosts, we would probably opt for aggregate items. They would allow us to find out the average load on a cluster or the total available disk space for a group of file servers, without naming each item individually. As with the calculated items, the result would be a normal item that could be used in triggers or graphs.

To find out what we can use in such a situation:

Go to Configuration | Hosts, select Linux servers in the Group drop-down menu and click on Items next to A test host, then click on Create item. Now, we have to figure out what item type to use. Expand the Type drop-down menu and look for an entry named Zabbix aggregate. That's the one we need, so choose it and click on Select next to the Key field. Currently, the key is listed as grpfunc, but that's just a placeholder; click on it. We have to replace it with the actual group key: one of grpsum, grpmin, grpmax, or grpavg. We'll calculate the average for several hosts, so change it to grpavg. This key, or group function, takes several parameters:

• group: As the name suggests, the host group name goes here. Enter Linux servers for this parameter.
• key: This is the key for the item to be used in calculations. Enter system.cpu.load here.
• func: This is a function used to retrieve data from individual items on hosts. While multiple functions are available, in this case we'll want to find out what the latest load is. Enter last for this field.
• param: This is a parameter for the previous function, following the same rules as normal function parameters (specifying either seconds or value count, prefixed with #). The function we used, last(), can be used without a parameter, so simply remove the last comma and the placeholder that follows it.

For individual item data, the following functions are supported:

For aggregate items, two levels of functions are available. They're nested—first, the function specified as the func parameter gathers the required data from all hosts in the group. Then, grpfunc (grpavg in our case) calculates the final result from all the intermediate results retrieved by func.

The final item key should be grpavg[Linux servers,system.cpu.load,last].

To finish the item configuration, fill in the following:

• Name: Average system load for Linux servers
• Type of information: Numeric (float)

The final item configuration should look like this:

When done, click on the Add button at the bottom. Go to Monitoring | Latest data, make sure all hosts are shown, and look for the three values again—CPU Load both for A test host and Another host, as well as Average system load for Linux servers:

The computed average across both hosts doesn't match our result if we look at the values on individual hosts, and the reason is exactly the same as we mentioned with the calculated items.

We used the grpavg aggregate function to find out the average load for a group of servers, but there are other functions, as follows:

This way, a limited set of functions can be applied across a large number of hosts. While less flexible than calculated items, it's much more practical in case we want to do such a calculation for a group that includes hundreds of hosts. Additionally, a calculated item has to be updated whenever a host or item is to be added or removed from the calculations. An aggregate item will automatically find all of the relevant hosts and items. Note that only enabled items on enabled hosts will be considered.

Nothing limits the use of aggregate items by servers. They can also be used on any other class of devices, such as calculating average CPU Load for a group of switches, monitored over SNMP.