We'll start by creating the certificates in a separate directory. For simplicity's sake, let's do this on A test host—choose any directory where our certificate signing will happen.

The following is not intended to be a good practice. It is actually doing quite a few bad and insecure things to get the certificates faster. Do not follow these steps for any production setup.

$ mkdir zabbix_ca
$ chmod 700 zabbix_ca
$ cd zabbix_ca

Generate the root CA key:

$ openssl genrsa -aes256 -out zabbix_ca.key 4096

When prompted, enter a password twice to protect the key. Generate and self-sign the root certificate:

$ openssl req -x509 -new -key zabbix_ca.key -sha256 -days 3560 -out zabbix_ca.crt

When prompted, enter the password you used for the key before. Fill in the values as prompted—the easiest might be supplying empty values for most except the country code and common name. The common name does not have to be anything too meaningful for our test, so using a simple string like zabbix_ca will suffice.

Now, on to creating a certificate we will use for the Zabbix server—first, let's generate a server key and certificate signing request (CSR):

$ openssl genrsa -out zabbix_server.key 2048
$ openssl req -new -key zabbix_server.key -out zabbix_server.csr

When prompted, enter the country code and common name strings as before. The common name does not have to match the server or agent name or anything else, so using a simple string such as zabbix_server will suffice. Let's sign this request now:

$ openssl x509 -req -in zabbix_server.csr -CA zabbix_ca.crt -CAkey zabbix_ca.key -CAcreateserial -out zabbix_server.crt -days 1460 -sha256

When prompted, enter the CA passphrase. Let's continue with the certificate we will use for the Zabbix agent. Generate an agent key and certificate signing request:

$ openssl genrsa -out zabbix_agent.key 2048
$ openssl req -new -key zabbix_agent.key -out zabbix_agent.csr

When prompted, enter the country code and common name strings as before. The common name does not have to match the server or agent name or anything else, so using a simple string such as zabbix_agent will suffice. Now, let's sign this request:

$ openssl x509 -req -in zabbix_agent.csr -CA zabbix_ca.crt -CAkey zabbix_ca.key -CAcreateserial -out zabbix_agent.crt -days 1460 -sha256

When prompted, enter the CA passphrase.

We're done with creating our test certificates. Both keys were created unencrypted—Zabbix does not support prompting for the key password at this time.

Now on to making the passive items on A test host use the certificates we just generated. We must provide the certificates to the Zabbix agent. In the directory where the Zabbix agent configuration file is located, create a new directory called zabbix_agent_certs. Restrict access to it, like this:

# chown zabbix zabbix_agent_certs
# chmod 500 zabbix_agent_certs

From the directory where we generated the certificates, copy the relevant certificate files over to the new directory:

# cp zabbix_ca.crt /path/to/zabbix_agent_certs/
# cp zabbix_agent.crt /path/to/zabbix_agent_certs/
# cp zabbix_agent.key /path/to/zabbix_agent_certs/

Edit zabbix_agentd.conf, and modify these parameters:

TLSAccept=cert
TLSConnect=unencrypted
TLSCAFile=/path/to/zabbix_agent_certs/zabbix_ca.crt
TLSCertFile=/path/to/zabbix_agent_certs/zabbix_agent.crt
TLSKeyFile=/path/to/zabbix_agent_certs/zabbix_agent.key

This will make the agent only accept connections when they are encrypted and use a certificate signed by that CA, either directly or through intermediates. We'll still use an unencrypted connection for active items. A user could supply certificates and expect all communication to be encrypted now, which would not be the case unless either of the TLSAccept or TLSConnect parameters required encryption. To prevent silently ignoring certificate files, Zabbix enforces one of TLSAccept or TLSConnect when certificates are supplied. Restart the Zabbix agent.

Let's take a look at the host configuration list in the Zabbix frontend:

Looks like connections to A test host do not work anymore. Let's check the agent log file:

failed to accept an incoming connection: from 127.0.0.1: unencrypted connections are not allowed

Looks like we broke it. We did set up encryption on the agent but did not get around to configuring the server side. What if we would like to roll out encryption to all the agents and deal with the server later? In that case, it would be best to set TLSAccept=cert,unencrypted—then, agents would still accept unencrypted connections from our server. Once the certificates are deployed and configured on the Zabbix server, we only have to remove unencrypted from that parameter and restart the Zabbix agents. Let's try this out—change zabbix_agentd.conf again:

TLSAccept=cert,unencrypted

Restart the agent daemon and observe monitoring resuming from the Zabbix server. Now, let's make the server uses its certificate. We'll place the certificate in a place where the Zabbix server can use it. In the directory where the Zabbix server configuration file is located, create a new directory called zabbix_server_certs. Restrict access to it, like this:

# chown zabbix zabbix_server_certs
# chmod 500 zabbix_server_certs

From the directory where we generated the certificates, copy the certificates over to the new directory:

# cp zabbix_ca.crt /path/to/zabbix_server_certs/
# cp zabbix_server.crt /path/to/zabbix_server_certs/
# cp zabbix_server.key /path/to/zabbix_server_certs/

Edit zabbix_server.conf, and modify these parameters:

TLSCAFile=/path/to/zabbix_server_certs/zabbix_ca.crt
TLSCertFile=/path/to/zabbix_server_certs/zabbix_server.crt
TLSKeyFile=/path/to/zabbix_server_certs/zabbix_server.key

Now, restart Zabbix server. Although we have specified the certificates on both agents and the server, passive items still work in unencrypted mode. Let's proceed with making them encrypted. In the Zabbix frontend, navigate to Configuration | Hosts, click on A test host, and switch to the Encryption tab. In the Connections to host selection, choose Certificate, and then click on the Update button. After the server configuration cache has been updated, it will switch to using certificate-based encryption for this host.

Going back to our scenario where we slowly rolled out certificate-based configuration to our agents and added it to the server later, we can now disable unencrypted connections on the agent side. Change zabbix_agentd.conf:

TLSAccept=cert

Restart the agent. If we had followed this process from the very beginning, monitoring would have continued uninterrupted. Let's try to use zabbix_get:

$ zabbix_get -s 127.0.0.1 -k system.cpu.load
zabbix_get [5746]: Check access restrictions in Zabbix agent configuration

That fails because the agent only accepts encrypted connections now. As we did for zabbix_sender, we can specify the certificate—but we must use the Zabbix server certificate now.

Access to the Zabbix server certificate is required for this command:

$ zabbix_get -s 127.0.0.1 -k system.cpu.load 
--tls-connect cert --tls-ca-file /path/to/zabbix_server_certs/zabbix_ca.crt --tls-cert-file /path/to/zabbix_server_certs/zabbix_server.crt --tls-key-file /path/to/zabbix_server_certs/zabbix_server.key
0.030000

Certainly, this results in a more secure environment. It is not enough to spoof the IP address to access this agent. It is not enough to have an account on the Zabbix server to have access to all agents—access to the server certificate is needed, too. On the other hand, it makes debugging a bit more complicated, as we can't query the agent that easily, and sniffing the traffic is much harder, too.

We used PSK and certificate-based encryption with zabbix_sender, zabbix_get, and a passive agent, but the same principles apply for active agents. As an exercise, try to get the active agent items working with encryption, too.