Re: RAID gotchas!

Justin W wrote:

Ferguson, Michael wrote:

Justin W wrote:

I just backed up my server using a combination of an LVM snapshot, dd to

copy the partition initially, and now it'll be maintained with nightly

rsyncs to a mounted image file. (Note: If anyone is interested, I can post some documentation describing how I set up the backup and the script which

will keep my backup up-to-date).

Justin, Thanks for the offer. Please post, or send me the documentation on

how to.

'preciate it.
Ferg'

I'm currently in the process of rewriting the script and such so that there is not so much manual input. If I succeed, the script should be quite portable across similar setups, but for now, you're going to have to change many of the path names to how they apply to you.

I've completed the update. This should be much more portable, and convenient. The last upgrade I can see is making a small script to do the initial image creation, but I don't think I'm going to be doing that any time soon (too many other projects to get started on right now).

Enjoy!

Justin W

Attachment: backup.xhtml
Description: application/xhtml

#!/bin/bash

# Overview of backup process
#  1) Read and parse configuration file
#  2) Loop through each backup
#  3) Create a FS snapshot using LVM
#  4) Mount the snapshot FS
#  5) Mount our backup file in a loop filesystem (we can guarantee the space
#     will be present this way, and it allows selective recovery of files if
#     need be)
#  6) rsync the differences between the snapshot FS and the backup FS
#  7) Unmount the backup FS
#  8) Unmount the snapshot FS
#  9) Delete the snapshot LVM (leaving it will degrade performance)
# 10) Repeat 3-9 if necessary

# Save the old IFS variable so that we can restore it when we're done
ORIGIFS=$IFS
IFS=$(echo -en "\n\b")

# User where the reports of the backup are sent to
MAILTO=root

# Read from the configuration file any exclude filters for the rsync output
LOG_EXCLUDE=$(cat "$PWD/backup.conf" | egrep "^[ \t]*LOG_EXCLUDE=" |
              cut -d'=' -f 2-)
LOG_FILE=/tmp/backup$$.log

# Get each line from the configuration file one at a time, ignoring anything
# that starts with a pound (#) sign
for i in $(cat "$PWD/backup.conf" | egrep -v "^[ \t]*#.*|^[ \t]*LOG_EXCLUDE=.*"); do
	# Parse out the locations we need
	VOL=$(echo $i | sed "s@^\([^:]*\):\(.*\)@\1@")
	TARG=$(echo $i | sed "s@^\([^:]*\):\(.*\)@\2@")

	# Check to make sure the paths are valid
	[ -b $VOL ]  || { echo "Source volume not valid: $VOL. Skipping";
	                  continue; }
	[ -f $TARG ] || { echo "Target image not valid: $TARG. Skipping";
	                  continue; }

	# Get necessary information about the source volume.
	# Get only the necessary line, and not the headers, and then compress the
	# info so it is only separated by a single space (makes finding fields
	# easier in the next steps). There will be an extra space at the beginning, 
	# so fields are 1 greater than what they seem.
	#
	# Make sure the volume is a LVM Logical Volume
	lvs -a $VOL 2>/dev/null 1>/dev/null || {
		echo "$VOL not an LVM Logical Volume ($?). Skipping."; continue; }
	# Now we actually collect the info
	INFO=$(lvs -a $VOL | tail -n 1 | sed 's/[ ][ ]*/ /g')
	# Field 1: Logical Volume Name
	VOL_NAME=$(echo $INFO | cut -d' ' -f 2)
	# Field 2: Volume Group Name
	GROUP_NAME=$(echo $INFO | cut -d' ' -f 3)
	# Field 3: Attributes - Not needed right now, but may come in handy
	ATTRS=$(echo $INFO | cut -d' ' -f 4)
	# Field 4: Volume Size
	VOL_SIZE=$(echo $INFO | cut -d' ' -f 5)

	# Now we just create a name for the snapshot volume, and then we can
	# create it
	SNAP_NAME=${VOL_NAME}$$
	SNAP_PATH=/dev/${GROUP_NAME}/${SNAP_NAME}
	lvcreate -s -L${VOL_SIZE} -n $SNAP_NAME $VOL >/dev/null || {
		echo "Snapshot of '${GROUP_NAME}/${SNAP_NAME}' failed ($?). Skipping.";
		continue; }
	
	# Mount our filesystems at temporary mount locations
	SNAP_MOUNT=/tmp/backup-$SNAP_NAME
	TARG_MOUNT=/tmp/backup-target$$
	mkdir $SNAP_MOUNT || {
		echo "Failed to create snapshot mount point ($?). Skipping.";
		cleanup; continue; }
	mkdir $TARG_MOUNT || {
		echo "Failed to create target mount point ($?). Skipping.";
		cleanup; continue; }
	mount $SNAP_PATH $SNAP_MOUNT || {
		echo "Mounting snapshot failed ($?). Skipping.";
		cleanup; continue; }
	mount -o loop $TARG $TARG_MOUNT || {
		echo "Mounting target image failed ($?). Skipping.";
		cleanup; continue; }

	# Update the backup file system with files from the snapshot
	# IMPORTANT!!: The trailing slash on the source is necessary for it to
	# work in the expected way, or else all backups will end up within a
	# subdirectory with the name $SNAP_MOUNT
	rsync -aHEAXxv --stats --delete --ignore-errors $SNAP_MOUNT/ $TARG_MOUNT | \
		egrep -v "$LOG_EXCLUDE" >> $LOG_FILE

	# Cleanup stuff is put into a function so that it can be called before
	# an exit call
	function cleanup() {
		# Unmount the backups and remove the temporary dirs
		# Errors may occur here, but we don't really care. Just pipe the errors
		# to /dev/null
		umount $SNAP_MOUNT 2>/dev/null 1>/dev/null
		umount $TARG_MOUNT 2>/dev/null 1>/dev/null
		[ -d $SNAP_MOUNT ] && rmdir $SNAP_MOUNT
		[ -d $TARG_MOUNT ] && rmdir $TARG_MOUNT

		# Delete the snapshot
		[ -b $SNAP_PATH ] && lvremove -f $SNAP_PATH >/dev/null
	}
	# Now call our cleanup routine on normal completions
	cleanup
done

# Mail the log file, and then delete the file
mail -s "Backups on $(hostname)" $MAILTO < $LOG_FILE
rm $LOG_FILE

# Restore the IFS
IFS=$ORIGIFS

# Any files which you don't want to show up in the rsync output (e.g. mail
# spool files which change often, /tmp, etc.), can be filtered out using
# `egrep' compatible regular expressions within a LOG_EXCLUDE variable.
# Logged locations do not begin with the leading slash, so /tmp would be
# filtered with an expression looking for "tmp/".
#
# Note:
# the LOG_EXCLUDE line must follow bash variable requirements, so no space is
# allowed before the equals sign.
#
# Example
#  LOG_EXCLUDE="tmp/|var/spool/mail"
LOG_EXCLUDE="tmp/|var/spool|home/|var/log|var/cache"

# Configuring the script to back up a file system is a very simple process.
# Most of the logic is calculated automatically, so only the original [LVM]
# file system and the target image must be specified
#
# Example:
#  /dev/VolGroup00/LogVol00:/mnt/bigdrive/SysBackup
#
# This would create a backup of the LogVol00 LVM partition (which most of the
# time happens to be the / file system, and it'll save it to the image SysBackup
# which resides on an external hard drive.

/dev/System/fc6:/mnt/smb/userdata/root/FC6-Backup-File