We already mentioned three crucial elements that must be backed up in order to have a safeguard against site crashes.
Database backup consists of exporting database structure (tables) and data. Export can be in some proprietary format or as a standard SQL script. Since our focus was on MySQL database we will explain the process for that Relational Database Management System (RDBMS). If you happen to use another RDBMS consult the vendor documentation for the appropriate export procedure.
MySQL comes with variety of command-line tools mostly for administrative purposes. The tool for exporting database is called mysqldump. This is a recommended template command line for exporting a single database from MySQL assuming that the command is being executed on the server where the database is running.
mysqldump -u<username> -p<password> --opt --result-file=<output file> <database name>
Using our values for the Linux, setup export would look like this:
mysqldump -uroot -psomepass --opt result-file=/tmp/moodle-export.sql moodledb
Or in case of Windows:
mysqldump -uroot -psomepass --opt result-file=Z:/temp/moodle-export.sql moodledb
Now that we know how to manually export a database we need to automate this process and order it in a designated manner.
To automate database backup we need to create a script that will export the database and store it in a predefined location on the server. This is done through the usage of shell scripts. Windows and Linux use different shell scripts so we will present separate scripts for each Operating System. Before doing that we need to improve the actual command line for mysqldump. It is considered a security flaw if an administrator uses a command-line tool to access some password protected resource and places a password within the actual the command line. Mysqldump gives us an alternative for specifying options. They can be read from a separate configuration file. That way the only thing we must ensure is that the configuration file itself is well protected against undesired user access. This is the configuration file my.ini which we recommend for backup use with mysqldump.
[client] default-character-set=utf8 add-drop-table=true add-locks=true comments=true complete-insert=true create-options=true disable-keys=true extended-insert=true quick=true quote-names=true set-charset=true single-transaction=true dump-date=true verbose=true #this should be modified by user user=<username> password=<password> #host=<host address> #pipe=true #socket=<socketname>
The section marked by the commentary for modification is the one you should be modifying to adapt to your own needs. If Moodle is installed on the same server where MySQL runs as is the case in this book, you only need to place the appropriate username and password.
Note
Make sure to use a database user that has backup rights. This means that you cannot use a DB user moodle that we created. You should use root account or create a new one and assign appropriate permissions to it.
Server log is a logfile or (several files) automatically created and maintained by system log server. It is used as a central storage for all activity events on the entire server. It is essential to review system logs on a regular basis. That can give you an early warning of any potential issues or offer an insight into a particular problem. In the following section we will give a brief explanation of log systems on Linux and Windows because all scripts in this chapter will write to subsequent system logs.
All Linux and UNIX distributions use Syslog as a standard server log implementation. There are several logfiles on a standard Linux and the one we are interested in is related to the applications. It is located usually at /var/log/messages. To see the latest events from the application system log execute the following command:
/usr/bin/tail /var/log/messages
If you want to filter entries you can use something like this:
/bin/cat /var/log/messages | /bin/grep <filter word(s) >
Every entry in the system log has several elements like facility, event priority, tag, and description message. Facility is a generic category that identifies a source of an event. It can have values like daemon, cron, user, news, etc. Event priority states a level of importance of the event and it can be emergency, alert, critical, error, warning, notice, etc. Tag is a custom application set identifier that can be used to easily identify the entry. It is usually a single word. To add an entry into system log we can use logger command. Format of the command is as follows:
/bin/logger -s -p <facility>.<priority> -t <tag> <description message>
For example, executing this command will add an event entry into log:
/bin/logger -s -p user.err -t Moodle "Test error entry"
To see the actual event let us filter the log using the tag word we have chosen here—Moodle.
/bin/cat /var/log/messages | /bin/grep Moodle
The outcome might look like this:
Nov 21 19:07:28 ip-10-212-155-65 Moodle: Test error entry
Windows has a different concept of system log. It is much more visually oriented. Log entries are called events. To access system events you can use event viewer. Event viewer can be accessed from Start menu | Administrative Tools | Event Viewer.
To create a custom event from a command line we can use eventcreate utility. It has the following format:
eventcreate /l <logname> /t <level> /so <facility> /id <event id> /d <description>
Logname: Represents one of the available logs. By default it offers application, security, setup, system, and forwarded events. The one we plan on using in our scripts is application.
Level: Determines the type of event and can be error, warning, or information.
Facility: Is a custom identifier that describes an event source. It can be any text value.
Event ID: This is a numeric identifier that can have a value between 1 and 1000.
Description: Detailed textual explanation of the event.
Here is a command-line example:
eventcreate /l application /t error /so Moodle /id 1 /d "Test error event"
And this is how the result looks in event viewer:
We can also list events from command line using wevtutil like this:
wevtutil qe application /rd:true /c:1 /f:text
This is the output we might get:
Event[0]:
Log Name: Application
Source: Moodle
Date: 2010-11-21T21:50:56.000
Event ID: 1
Task: N/A
Level: Error
Opcode: Info
Keyword: Classic
User: S-1-5-21-3609673158-3230877478-3424243811-500
User Name: SVR-MOODLEAdministrator
Computer: svr-moodle
Description:
Test error event
To automate database backup we need to create a script that will export the database and add it to the cron execution list. At the same time we want to write the outcome of the execution into a system log. For a start create a new file /etc/cron.daily/moodle_backup.cron and put the following content inside:
#!/bin/sh now=$(date +%d-%m-%Y) backupcmd=/usr/bin/mysqldump dbname=moodle bckupdir=/var/db/backup config=/etc/cron.daily/my.ini archive=$bckupdir/dump_$now logfile=$archive.log backupfile=$archive.sql scriptname=Moodle successmsg=Backup finished OK! failmsg="Unable to backup database! Checkout log file $logfile." $backupcmd --defaults-extra-file=$config -r $backupfile $dbname 2> $logfile rval=$? if [ $rval -eq 0 ] ; then /bin/logger -s -p user.info -t $scriptname "$successmsg" else /bin/logger -s -p user.err -t $scriptname "$failmsg" fi exit $rval
Now create a my.ini file in the same directory and copy the content from the previous section. Do not forget to modify username, password, and other parameters specific to your installation.
We need to mark the new script as executable:
chmod u=rwx,gu= /etc/cron.daily/moodle_backup.cron
To finish the process we need to create a backup destination directory:
mkdir /var/db/backup
chmod u=rw,g=r,o= /var/db/backup
To really understand the backup script you will need a basic knowledge of Linux shell script also known as BASH script. If that is not the case just read on to learn what should be modified in case your distribution or general setup is different from the one presented here.
The first five lines of the script are variables that you may need to modify. It is explained as follows:
Now:Contains the current date as textual value in the format day-month-year. You should change this only if you prefer a different date format. This text will be appended to export filename making it easily distinguishable from the rest.Backupcmd:Contains the location ofmysqldumpprogram. Modify it if your location is different from the current.Dbname:Name of the database you want to export, most likely you will have to change this.Bckupdir:Location where all backups should be stored. If your disk organization and/or preference is different than what we have presented here, modify this value.Config:Location of our custommy.inifile. Since we recommend placing the configuration file in the same location as cron script you should change it only if your cron file location differs from ours.
After every execution a logfile is created and preserved in the root backup directory. This is important for your information. Having an activity log can help you determine and diagnose any problem that might occur. Have in mind that you should periodically delete the old logfiles created by this script.
Batch shell script in Windows is quite different from BASH but it is a required tool for any serious administrator that maintains a server with that OS. Create a directory called backup. In our setup it would look like this:
mkdir Z:ackup
mkdir Z:scripts
Now open a text editor and copy inside the following content:
@echo off @set backupcmd="C:Program FilesMySQLMySQL Server 5.1inmysqldump.exe" @set dbname=moodle @set now=%date:~4,2%-%date:~7,2%-%date:~10,4% @set bckupdir=Z:ackup @set config=%~dp0my.ini @set archive=%bckupdir%dump_%now% @set logfile=%archive%.log @set backupfile=%archive%.sql @set successmsg=Backup finished OK! @set errormsg=Unable to backup database ^ %dbanme%. Check %logfile%. @%backupcmd% --defaults-extra-file=%config% ^ -r %backupfile% %dbname% 2> %logfile% @if %errorlevel% neq 0 goto dberr @echo %successmsg% @goto end :dberr @echo %errormsg% 1>&2 @eventcreate /l application /t error ^ /so Moodle /id 1 /d "%errormsg%" :end
Save it into Z:scriptsmoodle_backup.cmd.
Now in the same directory create a new file called my.ini and save the content presented in database section. Do not forget to modify the username, password, and other parameters specific to your installation. You can see that the basic structure of both scripts is quite similar. Differences are mostly related to the syntax of the two scripts. To see the explanation of the parameters you may need to modify the Linux section since all instructions expressed there also apply here.
As a final step create a scheduled task, which will execute this script every day. From elevated command prompt execute the following command:
schtasks /Create /TN "Moodle Backup"
/TR "Z:scriptsmoodle_backup.cmd" /SC DAILY /NP
Having backups is important but we also should know what to do in case we need to restore the database from the backup. First of all we should create an empty database and then perform the actual import. This is how it can be done from the command line:
mysqladmin -u<user> -p<password> create <database name>
And then the actual data import is:
mysql -u<user> -p<password> <database name> < <backupfile.sql>
In case you want to directly replace the existing database, the commands would look like this:
mysqladmin -u<user> -p<password> drop <database name>
mysqladmin -u<user> -p<password> create <database name>
mysql -u<user> -p<password> <database name> < <backupfile.sql>
To perform any of these tasks you should use administrative root user for MySQL or any other user with the same set of permissions.
This directory serves as file storage for all platform needs. It is essential to have it in order to maintain consistency of LMS.
You might think that backing up a single directory should not be a huge problem but it is actually a bit more complex than it looks. When we make a backup copy of part of the file system we would like to:
- Make sure we copied all files
- Make sure we copied all files correctly
- Make sure we transferred any specific permissions on all files and directories
- Keep the separate log of all items copied during execution
- Log any errors or discrepancies into the same log file
- Report backup outcome to system log for later review
There are many ways to perform a file backup on both Linux and Windows. We will present here the simplest possible approach using only the standard OS tools available. Our implementation will always perform a full copy of Moodledata directory. If that is not acceptable for your case due to disk space or other specific conditions you will have to implement your backup differently.
There are quite a few native tools for copying files on Linux, but we need one that is capable of not only copying files and directories but also that checks the integrity of the copied files. The tool that fits this description is rsync. Rsync is an open source utility that provides fast incremental file transfer. It supports transfers on the same server or between two different servers over network. That fact also presents it as a good choice as you can adapt backup script to a different type of backup easily.
Note
To learn more about rsync type man rsync in the command line or visit the official website http://rsync.samba.org/.
The script itself is structured in a similar fashion as the backup one. Create an empty text file in /etc/cron.daily/backup_files.cron and copy the following content:
#!/bin/sh now=$(date +%d-%m-%Y) source=/var/www/moodledata destinationroot=/var/site/backup destination=$destinationroot/$now logfile=$destinationroot/backup_$now.log successmsg="Backup created in $destination" failmsg="Unable to create backup! Check log file $logfile" direrrmsg="Invalid output directory! Check $destinationroot" scriptname=Moodle if [ -d $destinationroot ] ; then echo "" > /dev/null else /bin/logger -s -p user.err -t $scriptname "$direrrmsg" exit 1 fi /usr/bin/rsync -av $source $destination &> $logfile rval=$? if [ $rval -eq 0 ] ; then /bin/logger -s -p user.info -t $scriptname "$successmsg" else /bin/logger -s -p user.err -t $scriptname "$failmsg" fi exit $rval
The only things you should modify in this script are the values of variable source and destinationroot. Variable source should contain a full path to the Moodledata directory as it is on your system and destinationroot should contain a full path to the location on disk where you plan on storing all your backups. Let us create that directory:
mkdir /var/site/backup
Each backup will be stored in a sub directory named after a current date. We use a format day-month-year. The script first checks if the destinationroot directory exists and if it does not then an error is reported. If the directory check goes well rsync starts a copy of Moodledata directory. A logfile is created in<dirroot>/backup_<date>.log file and it will contain all the output made by rsync during the backup process. You can use this information to review what was copied or not. After backup is finished, the script checks the general status of the execution. The result of that inquiry is reported to the system log. We mark all log entries in the system log with Moodle so you can use this command to filter all the backup entries from the log:
/bin/cat /var/log/messages | /bin/grep Moodle
If you place the script in the /etc/cron.daily directory as we suggest here, it will be executed every day at midnight. If that does not suit your needs you can check the cron documentation and create a custom schedule for your backup task.
Note
A fairly decent documentation about cron can be found on Wikipedia at http://en.wikipedia.org/wiki/Cron.
Although it is possible to set up and use rsync on Windows it is not a straightforward and simple task. Fortunately, Windows has xcopy utility that can copy the entire directory structures, transfer permissions, and verify file integrity.
Create a new empty file Z:scriptsackup_files.cmd and copy the following content:
@echo off @set now=%date:~4,2%-%date:~7,2%-%date:~10,4% @set source=Z:website @set destinationroot=Z:sitebackup @set destination="%destinationroot%\%now%" @set logfile="%destinationroot%ackup_%now%.log" @set successmsg=Backup created in %destination% @set failmsg=^ Unable to create backup! Check log file %logfile% @set direrrmsg=^ Invalid output directory! Check %destinationroot% @set scriptname=Moodle @if not exist "%destinationroot%" goto desterr @xcopy %source% %destination% ^ /e /v /k /x /i /y ^ 1>%logfile% 2>&1 @if %errorlevel% neq 0 goto copyerr @echo %successmsg% @eventcreate /l application /t information ^ /so "%scriptname%" /id 1 /d "%successmsg%" > NUL @goto end :desterr @echo %direrrmsg% 1>&2 @eventcreate /l application /t error ^ /so "%scriptname%" /id 2 /d "%direrrmsg%" > NUL @goto end :copyerr @echo %failmsg% 1>&2 @eventcreate /l application /t error ^ /so "%scriptname%" /id 3 /d "%failmsg%" > NUL :end
Now create a destination directory for the backups:
mkdir Z:sitebackup
As a final step create a scheduled task that will execute this script every day. From the elevated command prompt execute the following command:
schtasks /Create /TN "Moodle site Backup"
/TR "Z:scriptsackup_files.cmd" /SC DAILY /NP
The structure and logic of this script is almost identical to that of Linux. Major differences are related to the actual syntax and specific tool names. To adapt the script to your setup modify the value of variables source and destinationroot. The script checks if destinationroot directory exists and in case it does not then backup is not performed and error is logged in the system event log. The actual backup is performed by xcopy that takes care of all the checks and validations. All execution process log is stored in<destinationroot>ackup_<date>.log file—Z:sitebackupackup_<date>.log in our case. You can use this file for reviewing the actual outcome of the backup. After backup is finished, the script checks the general status of the execution. The result of that inquiry is reported to the system event log. Use the event viewer to check the actual outcome.
There is generally no need to back up Moodle directory every day. You should make a manual backup whenever a new module is added/removed or upgraded or when you update the entire core. Make sure to place Moodle backup with appropriate database backups since quite often during minor versions upgrade database gets updated. If you still want to automate this task you can use the same scripts for Moodledata. Just change their name and modify the source and destination variables.