Verifying a DRBD filesystem

A semi-common maintenance concern regarding synchronized devices is verification. The question we should always ask ourselves in a high-availability scenario is how confident we are that data on both nodes match.

The drbdadm utility provides a parameter specifically for addressing this need. However, there are some caveats to consider when using it, which we will explain in this recipe.

Getting ready

Follow the recipes defined in all previous sections before starting here. At the very least, we need a fully-operational DRBD node pair to follow this recipe.

How to do it...

Follow these steps as the root user on pg1:

  1. Add this block of text inside the resource section defined in /etc/drbd.d/pg.res:
    net {
      verify-alg md5;
    }
  2. Run this command to make DRBD reread its configuration files:
    drbdadm adjust pg
    
  3. Begin verification with this command:
    drbdadm verify pg
    
  4. Monitor /proc/drbd until verification is complete:
    watch cat /proc/drbd
    
  5. Disconnect and reconnect the DRBD resource:
    drbdadm disconnect pg
    drbdadm connect pg
    

How it works...

Our first job is to define what we mean by verify. By default, DRBD is somewhat minimal, and it has no default for the algorithm it should use for checksum comparisons. The verify-alg setting is a network-oriented value and defines how DRBD should compare data segments. We also know md5 as a widely-used checksum algorithm. Thus, we set the verify-alg in a net block within the resource definition for pg.

Afterwards, we need to reread the configuration files so that the verify-alg setting is defined for the verification step. By invoking drbdadm with the adjust parameter, it will read and apply any valid changes we made to /etc/drbd.d/pg.res. When we're ready, we can launch the verification process by calling drbdadm with the verify parameter. Due to the CPU overhead of md5, this will be noticeably slower than a full device synchronization. We can watch its progress by paying attention to /proc/drbd:

How it works...

We can see that our example verification is 26.8% complete, with an estimated completion time of almost 2 minutes. The estimate is produced based on network speed, md5 speed, and the amount of remaining data. These details can fluctuate frequently, as writes to the DRBD device slow down the verification process.

The last step is to disconnect, then reconnect the pg resource from the DRBD network. During verification, DRBD marks blocks that have unmatched md5 checksums, but does not resend them until a new connection is established. We can't speculate about the reason for this step, but it is required to correct errors.

Tip

The last step is only required if any block failed verification. Errors (bad blocks) will be located in the kernel log according to DRBD documentation. We recommend checking for drbd0 messages in /var/log/syslog, /var/log/messages, and /var/log/kern.log, depending on your distribution.

See also

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.217.164.143