Justin W wrote:
I've completed the update. This should be much more portable, and convenient. The last upgrade I can see is making a small script to do the initial image creation, but I don't think I'm going to be doing that any time soon (too many other projects to get started on right now).Ferguson, Michael wrote:Justin W wrote:I just backed up my server using a combination of an LVM snapshot, dd tocopy the partition initially, and now it'll be maintained with nightlyrsyncs to a mounted image file. (Note: If anyone is interested, I can post some documentation describing how I set up the backup and the script whichwill keep my backup up-to-date).Justin, Thanks for the offer. Please post, or send me the documentation onhow to. 'preciate it. Ferg'I'm currently in the process of rewriting the script and such so that there is not so much manual input. If I succeed, the script should be quite portable across similar setups, but for now, you're going to have to change many of the path names to how they apply to you.
Enjoy! Justin W
Attachment:
backup.xhtml
Description: application/xhtml
#!/bin/bash
# Overview of backup process
# 1) Read and parse configuration file
# 2) Loop through each backup
# 3) Create a FS snapshot using LVM
# 4) Mount the snapshot FS
# 5) Mount our backup file in a loop filesystem (we can guarantee the space
# will be present this way, and it allows selective recovery of files if
# need be)
# 6) rsync the differences between the snapshot FS and the backup FS
# 7) Unmount the backup FS
# 8) Unmount the snapshot FS
# 9) Delete the snapshot LVM (leaving it will degrade performance)
# 10) Repeat 3-9 if necessary
# Save the old IFS variable so that we can restore it when we're done
ORIGIFS=$IFS
IFS=$(echo -en "\n\b")
# User where the reports of the backup are sent to
MAILTO=root
# Read from the configuration file any exclude filters for the rsync output
LOG_EXCLUDE=$(cat "$PWD/backup.conf" | egrep "^[ \t]*LOG_EXCLUDE=" |
cut -d'=' -f 2-)
LOG_FILE=/tmp/backup$$.log
# Get each line from the configuration file one at a time, ignoring anything
# that starts with a pound (#) sign
for i in $(cat "$PWD/backup.conf" | egrep -v "^[ \t]*#.*|^[ \t]*LOG_EXCLUDE=.*"); do
# Parse out the locations we need
VOL=$(echo $i | sed "s@^\([^:]*\):\(.*\)@\1@")
TARG=$(echo $i | sed "s@^\([^:]*\):\(.*\)@\2@")
# Check to make sure the paths are valid
[ -b $VOL ] || { echo "Source volume not valid: $VOL. Skipping";
continue; }
[ -f $TARG ] || { echo "Target image not valid: $TARG. Skipping";
continue; }
# Get necessary information about the source volume.
# Get only the necessary line, and not the headers, and then compress the
# info so it is only separated by a single space (makes finding fields
# easier in the next steps). There will be an extra space at the beginning,
# so fields are 1 greater than what they seem.
#
# Make sure the volume is a LVM Logical Volume
lvs -a $VOL 2>/dev/null 1>/dev/null || {
echo "$VOL not an LVM Logical Volume ($?). Skipping."; continue; }
# Now we actually collect the info
INFO=$(lvs -a $VOL | tail -n 1 | sed 's/[ ][ ]*/ /g')
# Field 1: Logical Volume Name
VOL_NAME=$(echo $INFO | cut -d' ' -f 2)
# Field 2: Volume Group Name
GROUP_NAME=$(echo $INFO | cut -d' ' -f 3)
# Field 3: Attributes - Not needed right now, but may come in handy
ATTRS=$(echo $INFO | cut -d' ' -f 4)
# Field 4: Volume Size
VOL_SIZE=$(echo $INFO | cut -d' ' -f 5)
# Now we just create a name for the snapshot volume, and then we can
# create it
SNAP_NAME=${VOL_NAME}$$
SNAP_PATH=/dev/${GROUP_NAME}/${SNAP_NAME}
lvcreate -s -L${VOL_SIZE} -n $SNAP_NAME $VOL >/dev/null || {
echo "Snapshot of '${GROUP_NAME}/${SNAP_NAME}' failed ($?). Skipping.";
continue; }
# Mount our filesystems at temporary mount locations
SNAP_MOUNT=/tmp/backup-$SNAP_NAME
TARG_MOUNT=/tmp/backup-target$$
mkdir $SNAP_MOUNT || {
echo "Failed to create snapshot mount point ($?). Skipping.";
cleanup; continue; }
mkdir $TARG_MOUNT || {
echo "Failed to create target mount point ($?). Skipping.";
cleanup; continue; }
mount $SNAP_PATH $SNAP_MOUNT || {
echo "Mounting snapshot failed ($?). Skipping.";
cleanup; continue; }
mount -o loop $TARG $TARG_MOUNT || {
echo "Mounting target image failed ($?). Skipping.";
cleanup; continue; }
# Update the backup file system with files from the snapshot
# IMPORTANT!!: The trailing slash on the source is necessary for it to
# work in the expected way, or else all backups will end up within a
# subdirectory with the name $SNAP_MOUNT
rsync -aHEAXxv --stats --delete --ignore-errors $SNAP_MOUNT/ $TARG_MOUNT | \
egrep -v "$LOG_EXCLUDE" >> $LOG_FILE
# Cleanup stuff is put into a function so that it can be called before
# an exit call
function cleanup() {
# Unmount the backups and remove the temporary dirs
# Errors may occur here, but we don't really care. Just pipe the errors
# to /dev/null
umount $SNAP_MOUNT 2>/dev/null 1>/dev/null
umount $TARG_MOUNT 2>/dev/null 1>/dev/null
[ -d $SNAP_MOUNT ] && rmdir $SNAP_MOUNT
[ -d $TARG_MOUNT ] && rmdir $TARG_MOUNT
# Delete the snapshot
[ -b $SNAP_PATH ] && lvremove -f $SNAP_PATH >/dev/null
}
# Now call our cleanup routine on normal completions
cleanup
done
# Mail the log file, and then delete the file
mail -s "Backups on $(hostname)" $MAILTO < $LOG_FILE
rm $LOG_FILE
# Restore the IFS
IFS=$ORIGIFS
# Any files which you don't want to show up in the rsync output (e.g. mail # spool files which change often, /tmp, etc.), can be filtered out using # `egrep' compatible regular expressions within a LOG_EXCLUDE variable. # Logged locations do not begin with the leading slash, so /tmp would be # filtered with an expression looking for "tmp/". # # Note: # the LOG_EXCLUDE line must follow bash variable requirements, so no space is # allowed before the equals sign. # # Example # LOG_EXCLUDE="tmp/|var/spool/mail" LOG_EXCLUDE="tmp/|var/spool|home/|var/log|var/cache" # Configuring the script to back up a file system is a very simple process. # Most of the logic is calculated automatically, so only the original [LVM] # file system and the target image must be specified # # Example: # /dev/VolGroup00/LogVol00:/mnt/bigdrive/SysBackup # # This would create a backup of the LogVol00 LVM partition (which most of the # time happens to be the / file system, and it'll save it to the image SysBackup # which resides on an external hard drive. /dev/System/fc6:/mnt/smb/userdata/root/FC6-Backup-File
