Scheduled Tasks

OpenBSD includes scheduled tasks that run once a day, week, and month. These jobs run as root and email the results to root. The daily maintenance script is the most complicated; the monthly script is the simplest.

If a server runs well, I might not log in to it for weeks or even months. In fact, I’ve had a few servers run for more than a year without a human being ever logging in to it. I read the daily reports from the machine and say “Yep, it’s okay,” and get on with my day, confident that what the regular status reports don’t tell me the monitoring system will.

The scheduled maintenance jobs email their results to the local root user, but if no one ever logs in to the machine, no one will see those results. Always set an email alias for root in /etc/aliases, so that these messages go to someone who will actually read them.

Reading every email message from every machine every day is annoying, but much less annoying than finding out that I have a bad service by a user telling me about it. These messages often alert me to system problems before anyone (including me) notices them. Sites with hundreds of machines often write scripts to parse incoming email messages and flag interesting details.

You should send maintenance email to the person ultimately responsible for the system—whoever is most interested in system changes and most likely to be aware of any day-to-day system changes. If you delegate the job of reading maintenance email to a minion who is less aware of the system, he will either annoy you with endless questions about what you did yesterday or learn to ignore anything actually in the status mail messages.

Here, we’ll take a look at how the daily, weekly, and monthly routines work. Complete documentation of the maintenance jobs appears in daily(8).

Daily Maintenance

The daily maintenance job starts by running any custom maintenance jobs (covered at the end of this section) from /etc/daily.local. Daily maintenance includes checking partitions, running the reminder service calendar(1), running rdist(1), removing scratch files, and a few other boring things.

OpenBSD can also do a few other interesting things as part of its daily maintenance:

  • Create a backup root filesystem, /altroot.

  • Perform system security checks.

  • Back up vital system files in /var/backup.

  • Check for changes in vital system files.

  • Check filesystem integrity.

  • Run rdist(1).

I discussed /altroot in Chapter 9 because it requires a dedicated filesystem partition. Each of the other tasks can be configured later.

Security Checks

Some things that go wrong don’t necessarily mean your system has experienced an intrusion, but are nonetheless suspicious. The daily security check looks for a whole slew of misconfigurations and problems that arise from either malice or incompetence. You can read the list of checks in security(8), but they break down into fairly broad categories:

Device node changes and privileges

New device nodes, changed permissions on device nodes, new software packages, and new or altered disks or partitions might indicate malicious activity or might just be normal system management. The security script flags all of these. If you made such changes, you’ll nod and go on with your day. If you requested a minion perform the change and the change doesn’t appear, this is when you ask them what they did all day yesterday. If you didn’t make a change that appears, you want to know about it.

Insecure NFS exports

OpenBSD includes a lot of software to export filesystems and run commands remotely. These services, like printing and NFS, should not allow access from any host just anywhere, but only from hosts you approve. The security job checks for configurations that permit global access.

Misconfigured accounts

Another popular attack route has been the password database and related files. Accounts without passwords, duplicate entries, improperly closed accounts, and so on could all be used to compromise a system. The script checks for these issues.

Permissions

Poor permissions can lead to privilege escalation. For example, is there a new setuid or setgid file on the system? If so, the security script notifies you. If you installed that file with those privileges, you’re okay. If it’s unexpected, you should investigate.

User environment

If you can change a user’s environment, whether that’s root or another account, you can trick him into giving away his authentication credentials or running suspect commands. If an intruder can edit a user’s dotfiles, like .cshrc or .login, he can change which versions of a command he runs. Perhaps his shell is set to run a program that asks the user for his password and sends it to the intruder’s anonymous email account. By having correct permissions on home directories, dotfiles, mail files, and so on, you make this class of attack more difficult. The security script verifies that permissions are set up correctly.

Note that the security check is not an intrusion-detection system. The changes it checks for are the sort that script kiddies and newbie intruders are most likely to make, but skilled intruders familiar with OpenBSD could get around it. They could even replace the security check with a shell script that sends a daily email message that looks like an innocuous security check.

Fortunately, competent intruders are relatively rare. Just keep in mind that receiving a security check with no mention of problems is encouraging, but it’s not proof that your server is secure.

Vital File Backup and Testing

The daily security check tests for changes in the files listed in /etc/changelist and rotates their backups, since these files are generally critical system files, such as /etc/master.passwd, /etc/boot.conf, and /var/cron/tabs/root. It also checks for changes to disk partitioning and mounted filesystems, as well as changes to device nodes.

Look in /var/backups, and you’ll see files like this:

…
etc_fstab.backup
etc_fstab.current
etc_ftpchroot.current
etc_ftpusers.current
…

The files ending in .current are copies of these files as they existed when the daily maintenance job was last run. The files ending in .backup are the previous version of those files.

The first time the security script runs, it copies all of these files to /var/backup. Following that initial setup, the security script checks the original file against the current copy for changes. If the file changes, the previous version of the file is copied to the .backup filename, and the new version is copied to the .current file.

In the preceding example, the list shows that I edited my /etc/fstab at some time, prompting the security script to move its copy of the old filesystem table to a .backup file. I have never edited /etc/ftpchroot or /etc/ftpusers, so there is no .backup version of these files, but only the .current one.

The security script doesn’t copy all of the files that it watches. For example, files containing private keys or that might contain private keys are not copied, but the security script does take a checksum. (Files monitored by checksum have a plus sign before their name in /etc/changelist.) There’s no reason to manually edit /etc/ssh/ssh_host_ecdsa_key, and if the file changes, either you know why or you need to restore from a trusted backup.

/etc/changelist is itself listed in /etc/changelist. This seems recursive, but the system backs up the list of files you want backed up, and also notifies you when someone adds or removes a file in /etc/changelist.

Adding Vital Files

You can add files to the change list and even use wildcards to back up all the files in a directory. But note that if you did use wildcards in /etc/changelist, you won’t be notified when a file is removed.

Consider this example of using wildcards. In Chapter 13, I added the apache2 port to one of my machines. I put the configuration files in /etc/apache2. I could add a line like this to the change list:

/etc/apache2/*

This would automatically copy all files in the apache2 configuration directory to /var/backup and test them for changes. However, I would not be notified when files are removed from this directory. If you’re using a configuration mechanism that says, “include all the .conf files in such-and-such directory,” this might not be desirable. A better option would be to list each file individually and update the list when you add critical files.

One of the most convenient things about the file-integrity check is that it automatically creates a local backup of critical system files. That means that if you decide to learn how to use vipw(8) and utterly trash your user database in doing so, you can grab yesterday’s copy out of /var/backups, install it, and no one will be the wiser. The same applies to every other critical system file.

Filesystem Integrity Checks

You can’t run a full-on Unix File System (UFS) check while a system is in multiuser mode, but you can have fsck(8) perform filesystem integrity checks to try to identify problems before they’re serious. Doing so won’t fix any problems, but it will notify you that they exist so you can schedule downtime for repairs.

To enable these checks, set CHECKFILESYSTEMS to 1 in /etc/daily.local.

Copying Files with rdist

The rdist(1) program is used to copy files to other servers, letting you maintain identical copies of critical files on many servers. If you’re interested in using it, see rdistd(8).

Silencing /etc/daily

Some of us have monitoring systems that track a server’s disk, network, and other basic information. If you don’t need this sort of information to appear in your daily status mail, set VERBOSESTATUS to 0 in /etc/daily.local. This turns off these parts of daily maintenance, reducing the amount you need to read.

If the remaining daily maintenance doesn’t generate any output, the server should not send a status email that day. In environments where you don’t trust the monitoring system, you could use the daily status messages to assure you that the system is running as expected. OpenBSD gives you the choice.

Weekly Maintenance

The weekly script is simpler than the daily script with only three common functions:

  • First, it runs the custom weekly script /etc/weekly.local.

  • Second, it updates the locate(1) database.

  • Finally, it rebuilds the whatis(1) man page database.

Monthly Maintenance

OpenBSD doesn’t need any generic monthly maintenance, but for consistency, the /etc/monthly script runs the custom script /etc/monthly.local.

Custom Maintenance Scripts

Each maintenance script runs a custom script before performing any other tasks. You can put any tasks you need in /etc/daily.local, /etc/weekly.local, and /etc/monthly.local. These commands are run by root, so don’t use them for tasks that should be performed by another user. If your database needs to be backed up, create a separate script, and have the unprivileged user running your database run that script via cron(8).

Some sites use the scheduled maintenance jobs to run complex software that perform site-specific duties. For example, I know of one security firm that collects data from hundreds of machines, and uses the daily jobs to send that data to a central management system. Really, you can use the local scripts any way you choose.

If you have a maintenance task that can run under another user account, but you want to attach it to the scheduled maintenance jobs, you can have the local script call another script. Start that script by using su(1) to switch users and drop privileges.

Custom maintenance scripts may be most useful for altering the way the standard maintenance scripts perform their work. For example, say you have a system with many scratch directories containing temporary files. The weekly maintenance script updates the locate database, but you don’t want these scratch files included in locate results. You could use a custom maintenance script to remove all the scratch files immediately before /etc/weekly creates the new locate database, and schedule this as a separate task. By adding it to /etc/weekly.local, you would know that it will finish before /etc/weekly runs any other tasks.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.117.77.54