BackupPC with Locking and WOL

Overview of our Setup

BackupPC is a centralized network backup solution. The backuppc daemon monitors network status of workstations, plans incremental and full backups, purges old backups from the repository.
The backup process, along with our other centralized support services (e.g. nightly update of work databases and files for individual developers) is taking advantage of our custom-developed a centralized locking tool. The locks prevent e.g. automatic shutdowns of serviced machines when they are being accessed by other service, or when user is physically working on the machine.
The /etc/backuppc/templates holds configuration templates for basic variants of our workstations/server. Config files for individual machines in /etc/backuppc are mostly symlinks to corresponding templates.
Our intranet web server offers a simple centralized WakeOnLAN service. It takes only a single http request with machine name. Instead of regular ping (parameter $Conf{PingPath}), our backuppc is using a simple script using the central WOL service if the machine is down.

Config Examples

Re-defining ping with WOL capability

$Conf{PingPath} = "/etc/backuppc/bin/wolping.sh";

For the wolping.sh see below.

Limiting tar and rsync to the Current Filesystem

Various machines have various mount points. We want to make sure tar and rsync do not leave the current-level filesystem.

$Conf{TarClientCmd} = '$sshPath -C -q -x -n -l root $host' . ' /usr/bin/env LC_ALL=C $tarPath --ignore-failed-read --one-file-system'-c -v -f - -C $shareName+' . ' --totals'; $Conf{RsyncArgs} = [ # # Do not edit these! # ...... # Add additional arguments here '-D', '--one-file-system', ];

Bandwidth Limiting the SSH Connections

Backup of internet-based servers at full speed would jam the download link. We are using the trickle tool. Only specific servers have this directive in their config file.

# bandwidth limiting: 2000kB=16Mb down, 300kB=2.4Mb up $Conf{TarClientCmd} = '/usr/bin/trickle -s -u 300 -d 2000 $sshPath -C -q -x -n -l root $host' . ' /usr/bin/env LC_ALL=C nice $tarPath --one-file-system -c -v -f - -C $shareName+' . ' --totals';

Limiting I/O Load on Backed-up Machines

Heavily loaded machines cannot afford to hand its full I/O capacity to the backup job. We are using the ionice tool.

$Conf{TarClientCmd} = '$sshPath -C -q -x -n -l root $host' . ' /usr/bin/env LC_ALL=C ionice -c 2 -n 7 $tarPath --ignore-failed-read --one-file-system -c -v -f - -C $shareName+' . ' --totals'; $Conf{RsyncClientCmd} = '$sshPath -q -x -l root $host ionice -c 2 -n 7 $rsyncPath $argList+';

Templating the Configuration Files

We have categorized the backed-up machines based on their purpose and characteristics. Each purpose/characteristics has its specific configuration in /etc/backuppc/templates directory. E.g. common-linux.pl holds configuration forlinux machines in general:

$Conf{XferMethod} = 'tar'; $Conf{TarShareName} = ['/','/boot']; # vetsi timeout... $Conf{ClientTimeout} = 7*3600;


$Conf{BlackoutBadPingLimit} = 72;

$Conf{BlackoutGoodCnt}      = 1;

$Conf{BackupFilesExclude} = { '/' => [ '/proc', '/tmp', '/sys', '/var/run', '/var/lock', '/var/cache/apt/archives', '/cdrom', '/mnt/cd', '/mnt/cdrom', '.In*', '.x*', '.X*', '/var/lib/mysql', '/var/lib/sendmail', '/var/log/mysql', '/home/*/tomcat*/webapp', '*/webapp/target/*',

Our common-devel.pl template provides configuration for our development machines with standardized layout of raid arrays and standardized excluded directories (in fact holding millions of files we do not want to backup)

push @{$Conf{TarShareName}}, '/mnt/raid/'; push @{$Conf{BackupFilesExclude}{'/mnt/raid/'}}, '/mysql*', '/tmp', '/home/*/trash', '/home/*/tomcat*/webapps/', '/home/*/work/data', '*/webapp/target/*', '*.class' ; # larger timeour $Conf{ClientTimeout} = 10*3600;

These features are combined to yield the desired functionality. Regular linux non-development workstations are safely halted after the backup finishes- linux-workstation.pl:

# # Standard linux workstation, only halting


do "/etc/backuppc/templates/common-linux.pl";

$Conf{DumpPostUserCmd} = '$sshPath -q -x -l root $host /opt/ivitera_scripts/bin/halt.sh backup';

Linux development workstations have more specific requirements – linux-devel-workstation.pl:

## config for devel PC - data in /mnt/raid do "/etc/backuppc/templates/linux-workstation.pl"; do "/etc/backuppc/templates/common-devel.pl";

Linux servers are simple. Yet we provide a separate template for the servers which can be modified in the future – linux-server.pl:

# # Standard linux server, only locking

do "/etc/backuppc/templates/common-linux.pl";

Linux development servers build on top of this template, but share the development feature with development workstations – linux-devel-server.pl:

## config for devel server - data in /mnt/raid do "/etc/backuppc/templates/linux-server.pl"; do "/etc/backuppc/templates/common-devel.pl";

We have a windows cygwin workstation too – win-cygwin-workstation.pl:

Conf{XferMethod} = "tar"; $Conf{TarShareName} = ['/cygdrive/c'];


$Conf{BackupFilesExclude} = [

                              '/hiberfil.sys',

                              '/pagefile.sys',

                              '/Program Files',

                              '/RECYCLER',

                              '/System Volume Information',

                              '/Temp',

                              '/WINDOWS',

                              '/Documents and Settings/All Users/DRM',

                              '/Documents and Settings/*/NTUSER.DAT',

                              '/Documents and Settings/*/ntuser.dat.LOG',

                              '/Documents and Settings/*/Local Settings/Data aplikací/Microsoft/Windows/UsrClass.dat',

                              '/Documents and Settings/*/Local Settings/Data aplikací/Microsoft/Windows/UsrClass.dat.LOG',

                              '/Documents and Settings/*/Data aplikací/Mozilla/Firefox/Profiles/*/parent.lock',

                              '/Documents and Settings/*/Data aplikací/Thunderbird/Profiles/*/parent.lock',

                              '/Documents and Settings/*/Data aplikací/Skype'

                            ];
$Conf{ClientTimeout} = 7*3600;

# locks - functional only for linux/cygwin $Conf{DumpPreUserCmd} = '$sshPath -q -x -l root $host /opt/ivitera_scripts/bin/lock.sh backup'; $Conf{DumpPostUserCmd} = '$sshPath -q -x -l root $host /opt/ivitera_scripts/bin/halt.sh backup';

Configuration files for individual machines in /etc/backuppc are either directly symlinks to their corresponding templates in /etc/backuppc/templates, or include the template and add/modify some settings:

orfeus:/etc/backuppc# ls -l .... lrwxrwxrwx 1 root root 49 2010-09-17 12:18 athena.pl -> /etc/backuppc/templates/win-cygwin-workstation.pl -rw-r--r-- 1 root root 69407 2011-05-31 08:23 config.pl -rw-r--r-- 1 root root 27 2011-08-23 14:05 external-control.pl lrwxrwxrwx 1 root root 50 2010-09-17 12:17 hestia.pl -> /etc/backuppc/templates/linux-devel-workstation.pl lrwxrwxrwx 1 root root 44 2010-09-17 12:23 lycos.pl -> /etc/backuppc/templates/linux-workstation.pl -rwxr-xr-x 1 root root 169 2010-11-02 16:22 orion-1000-DumpPreUserCmd.sh -rw-r--r-- 1 pavel www-data 311 2010-11-02 16:21 orion-1000.pl drwxr-xr-x 1 root root 4096 2011-08-23 21:58 templates ...

Where our main devel server orion-1000.pl:

do "/etc/backuppc/templates/linux-devel-server.pl"; # huge timeout... $Conf{ClientTimeout} = 5*24*3600; # plus dumping SQL databases before the backup - must be a single script since backuppc does not accept multiple commands here 🙁 $Conf{DumpPreUserCmd} = '/etc/backuppc/orion-1000-DumpPreUserCmd.sh';

WOL-enabled ping

The following script wolping.sh takes advantage of our central WOL tool

#! /bin/bash


PING=/bin/ping

ARG1=$1

ARG2=$2

WAKEHOST=$3
echo "$1 $2 $3" >> /tmp/wolping.log
source /etc/ivitera/functions.inc
if ! $PING $ARG1 $ARG2 $WAKEHOST; then

        # functions defined in functions.inc

        fwol $WAKEHOST

        if [ "$WOL_RES" = "FAIL" ]; then

                exit 1

        fi

        sleep 5m

        $PING $ARG1 $ARG2 $WAKEHOST || exit 1

fi

exit 0

Where functions.inc:

export WOL_RES function fwol { TO_WAKEUP=$1 if /usr/bin/wget -O - "http://wol.example.com/index.php?machine=$TO_WAKEUP" | grep -q "ERROR" ; then echo "WOL failed" WOL_RES="FAIL" else echo "WOL OK" WOL_RES="OK" fi }

export PING_RES function fping { TO_PING=$1 if /bin/ping -c 10 -l 4 $TO_PING 2>/dev/null >/dev/null ; then echo "$MACHINE running" PING_RES="OK" else echo "$MACHINE not running" PING_RES="FAIL" fi }

Overview of our Setup

Config Examples

Re-defining ping with WOL capability

Limiting tar and rsync to the Current Filesystem

Bandwidth Limiting the SSH Connections

Limiting I/O Load on Backed-up Machines

Templating the Configuration Files

WOL-enabled ping

Archives

Links

BackupPC with Locking and WOL

Overview of our Setup

Config Examples

Re-defining ping with WOL capability

Limiting tar and rsync to the Current Filesystem

Bandwidth Limiting the SSH Connections

Limiting I/O Load on Backed-up Machines

Templating the Configuration Files

WOL-enabled ping

Archives

Tags

Links