Managing WAL files with OmniPITR

We've stated on several occasions that WAL files are very important. Their role in PostgreSQL crash recovery, backup restoration, and replication gives them a central role in maintaining a high availability cluster. With OmniPITR, we can upgrade communication between servers to ensure that we have logging for every step of a WAL file's movement. This is no small benefit, and we can use it to audit the entire transmission path if we encounter a problem.

Though OmniPITR is a full suite of backup-related tools, we wish to focus on its ability to give us better control of WAL archival and recovery. As a consequence, this recipe will describe usage of the omnipitr-archive command.

Getting ready

This recipe depends on OmniPITR being installed on all servers that need to utilize it. Please follow the Installing and configuring OmniPITR recipe before continuing.

How to do it...

For this procedure, we will continue to use two servers. The backup server will still be named pg-backup, and our primary PostgreSQL server is pg-primary. As usual, the PostgreSQL data directory will be located at /db/pgdata.

Follow these steps to send WAL files from pg-primary to pg-backup:

  1. On the pg-backup server, create a directory writable to the postgres user with the following commands as a root-capable user:
    sudo mkdir /db/pg_archived
    sudo chown postgres:postgres /db/pg_archived
    
  2. Create a file named omnipitr.conf in the /etc directory on pg-primary with the following contents:
    --data-dir /db/pgdata
    --dst-remote postgres@pg-backup:/db/pg_archived
    --log /var/log/postgresql/omnipitr.log
  3. Modify the postgresql.conf file on pg-primary and ensure that the following parameters are set:
    archive_mode = on
    archive_command = 'omnipitr-archive -- 
                       cfg=/etc/omnipitr.conf %p'
  4. Restart the PostgreSQL server with the following command as the postgres user:
    pg_ctl -D /db/pgdata restart
    
  5. Examine the contents of the omnipitr.log logfile with this command as the postgres user:
    tail /var/log/postgresql/omnipitr.log
    

How it works...

We start by ensuring that the postgres user can write to the /db/pg_archived directory on the pg-backup server, which is the location we've set aside to hold WAL files. This is also the only step we perform on the pg-backup server.

One interesting thing to consider about OmniPITR is that it reads configuration files in a similar manner as command-line parameters. With this in mind, and to avoid long and confusing command-lines, we save several in a configuration file for later use.

The first is the path to the PostgreSQL data directory. If this is unset, OmniPITR will assume that the WAL files are local to the data directory in pg_xlog. While this will work, it's better for logging purposes to set this explicitly to /db/pgdata. The second is the remote path to WAL files. As we created the /db/pg_archived directory on pg-backup, we use that same location here. Finally, we'll commit logs to the /var/log/postgresql directory, which should already exist on most Red Hat and Debian-based servers.

Now, we need to ensure PostgreSQL uses OmniPITR to send the files to pg-backup. Once we've confirmed that archive_mode is on, we can set archive_command to invoke omnipitr-archive. Because of our earlier work, we only need to set two parameters. The first is the full path to the configuration file we created, and the second is %p, which represents the full path to the WAL file that PostgreSQL wants to archive. Once PostgreSQL is restarted, it will use OmniPITR to manage its WAL files.

Tip

We should note that we only need to fully restart PostgreSQL if archive_mode was previously set to off. Otherwise, a simple reload will cause PostgreSQL to use the newly defined archive_command value.

Unlike Barman, OmniPITR has no command to verify that it's working properly. To do this, we must examine the logfile. If we look at the end of the omnipitr.log file in /var/log/postgresql/ with tail, we should see something like this:

How it works...

There's more...

Perceptive readers may have noticed that we don't present an analogous situation to pull WAL files from the pg-backup server to a hot-standby. Unfortunately, while the provided omnipitr-restore command will move WAL files to their expected locations and include logging, it can not retrieve these files from a remote server. We are not entirely sure why the authors of OmniPITR would neglect to include this functionality, but it is an issue that we can not overcome.

As such, we do not recommend using OmniPITR to maintain clones or streaming replicas with our suggested architecture. An off-site backup server is invaluable, which means that remote WAL files are an inescapable reality.

This does not imply that OmniPITR is completely unsuited to manage certain elements of larger clusters. If you have time, examine the documentation of each OmniPITR utility and consider how each might be beneficial to your architecture.

See also

  • While OmniPITR does not install manuals locally, we can invoke its tools to learn more about them. To see the full capabilities of omnipitr-archive, use this command:
    Omnipitr-archive --help
    
  • OmniPITR's documentation is also available on PGXN. To view the manual for omnipitr-archive there, please visit http://pgxn.org/dist/omnipitr-archive.html.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.97.216