Sunday, March 19, 2017

Backing Up NAS to NAS via Raspberry Pi with autofs and rsync

I keep all my home data on a NAS.
Even though it's RAID-mirrored, when drives go, they tend to take the mirror with them —
so I run two NAS units and copy between them periodically.

When the two NAS units are from different manufacturers, or when you only want to sync specific folders,
the built-in network backup feature of a NAS often can't handle it.

So I use a Raspberry Pi to handle backups between the two NAS units.

I rarely touch this setup and keep forgetting how it works, so I'm documenting it here.

Setup Assumptions

  • Both NAS units share folders over Windows file sharing (SMB).
  • One NAS is the active (primary) unit where data is written; the other is standby (replica) for backup only.
  • If the standby NAS fails, buy a new one, promote the old active to standby.
  • If the active NAS fails, buy a new one as the replacement active and restore from the standby.
    (The standby always holds the older unit.)
  • Restarting either NAS should not interrupt the Raspberry Pi backup function.


Assigning Fixed IP Addresses

Assign static IPs to both NAS units.
DHCP might technically work, but if the active and standby accidentally swap addresses, the backup would go the wrong way — and that's a disaster.

Static IP configuration depends on your NAS and router setup — I'll skip that part.

For this guide I'll assume the following addresses:
  • Active NAS IP: {active_nas_ip}
  • Active NAS hostname: active_nas
  • Standby NAS IP: {standby_nas_ip}
  • Standby NAS hostname: standby_nas


Setting Hostnames on the Raspberry Pi

Name the two NAS IP addresses in the Raspberry Pi's hosts file.
SSH into the Raspberry Pi and run:
pi@raspberrypi ~ $ sudo -s
root@raspberrypi /home/pi $ cp /etc/hosts /etc/hosts.`date +%Y%m%d`
root@raspberrypi /home/pi $ echo '{active_nas_ip}    active_nas' >> /etc/hosts
root@raspberrypi /home/pi $ echo '{standby_nas_ip}    standby_nas' >> /etc/hosts
root@raspberrypi /home/pi $ cat /etc/hosts
127.0.0.1 localhost
::1  localhost ip6-localhost ip6-loopback
fe00::0  ip6-localnet
ff00::0  ip6-mcastprefix
ff02::1  ip6-allnodes
ff02::2  ip6-allrouters

127.0.1.1 raspberrypi
{active_nas_ip} active_nas
{standby_nas_ip} standby_nas


Configuring autofs

To mount the NAS shares on the Raspberry Pi, a plain smbmount would require a re-mount every time the NAS reboots.

Instead, use autofs to mount on-demand automatically, which also survives NAS reboots gracefully.
autofs also handles NAS failures without causing persistent mount errors.

Install the required packages:
root@raspberrypi /home/pi $ apt-get install autofs smbclient

Configure autofs for mounting:
root@raspberrypi /home/pi $ cp /etc/auto.master /etc/auto.master.`date +%Y%m%d`
root@raspberrypi /home/pi $ echo '/smb    /etc/auto.smb' >> /etc/auto.master
root@raspberrypi /home/pi $ mkdir /smb
root@raspberrypi /home/pi $ mkdir /etc/creds
root@raspberrypi /home/pi $ echo 'username={nas_username}
password={nas_password}' > /etc/creds/active_nas
root@raspberrypi /home/pi $ chmod 600 /etc/creds/active_nas
root@raspberrypi /home/pi $ echo 'username={nas_username}
password={nas_password}' > /etc/creds/standby_nas
root@raspberrypi /home/pi $ chmod 600 /etc/creds/standby_nas
root@raspberrypi /home/pi $ service autofs restart

Replace {nas_username} and {nas_password} with the credentials for accessing your NAS shares.
If the two NAS units use different accounts, enter each one separately.
Putting credential files in /etc/creds with the hostname as the filename lets autofs pick them up automatically at mount time.
Set permissions to 600 (owner read/write only).

After this, regular users should be able to access the NAS data.
For an I-O Data LANDISK, it would look something like:

root@raspberrypi /home/pi $ exit
pi@raspberrypi ~ $ ls /smb/active_nas
contents  disk  itunes
pi@raspberrypi ~ $ ls /smb/active_nas/disk
TrashBox quickcopy
pi@raspberrypi ~ $ ls /smb/standby_nas
contents  disk  itunes
pi@raspberrypi ~ $ ls /smb/standby_nas/disk
TrashBox quickcopy
pi@raspberrypi ~ $ df
Filesystem      1K-blocks      Used  Available Use% Mounted on
/dev/root        15021296   2428732   11949656  17% /
devtmpfs           437048         0     437048   0% /dev
tmpfs              441384         0     441384   0% /dev/shm
tmpfs              441384     11556     429828   3% /run
tmpfs                5120         4       5116   1% /run/lock
tmpfs              441384         0     441384   0% /sys/fs/cgroup
/dev/mmcblk0p1      61384     21368      40016  35% /boot
//active_nas/disk   1949057984 846123472 1102934512  44% /smb/active_nas/disk
//standby_nas/disk      972305984 839236008  133069976  87% /smb/standby_nas/disk

Wait a bit and run df again — the NAS mounts should have been automatically unmounted.


Running the Backup

Use rsync to sync data.
Swapping source and destination would be catastrophic, so run a dry-run first:
pi@raspberry ~$ /usr/bin/rsync -n -rltv --max-size=100m --size-only /smb/active_nas/disk/ /smb/standby_nas/disk/

For a large NAS, a full checksum diff takes forever —
start with --size-only to only sync files that differ in size.
Review the output; if it looks right, remove -n and run for real.
But let's not delete files yet — just append new/changed ones, and cap at 100MB:

pi@raspberry ~$ /usr/bin/rsync -rltv --max-size=100m --size-only /smb/active_nas/disk/ /smb/standby_nas/disk/

After it runs, confirm the listed files were copied correctly.
If all looks good, run a full sync that also mirrors deletions:

pi@raspberry ~$ /usr/bin/rsync -rltv --delete /smb/active_nas/disk/ /smb/standby_nas/disk/


Scheduling the Backup

Wrap the command in a script and schedule it weekly via crontab:
pi@raspberry ~$ cat rsync.sh
#!/bin/sh

/usr/bin/rsync -rltv --delete $1 $2
pi@raspberry ~$ crontab -e
15 05 * * 1 /root/bin/rsync.sh /smb/active_nas/disk/ /smb/standby_nas/disk/

However, this command has one critical flaw:
if the active NAS fails and the mount shows up as an empty folder,
rsync will happily "backup" that empty folder — wiping all data from the standby.

To guard against this, add sanity checks before allowing any deletion:
#!/bin/bash

THRES=-10
SRC=$1
DST=$2

/bin/ls $SRC
/bin/ls $DST

# Abort if source folder doesn't exist
if [ ! -d $SRC ]
then
  echo "$SRC Not Found" >&2
  exit 1
fi

# Abort if destination folder doesn't exist
if [ ! -d $DST ]
then
  echo "$DST Not Found" >&2
  exit 1
fi

# If /tmp/restore exists (placed manually during a restore operation),
# skip the cron backup to avoid overwriting.
if [ -f /tmp/restore ]
then
  echo "Restoring" >&2
  exit 1
fi

# Check the used size of each mounted share
srcsize=`/bin/df $SRC | /bin/grep / | /usr/bin/awk '{print $3}'`
dstsize=`/bin/df $DST | /bin/grep / | /usr/bin/awk '{print $3}'`
let diff=(${srcsize}-${dstsize})/1048576
if [ $diff -lt $THRES ]
then
  # Abort if source is drastically smaller than destination (likely a failed mount)
  echo "Source data size is too small" >&2
  exit 1
elif [ $diff -lt 0 ]
then
  # If source is slightly smaller, sync without --delete (defensive)
  echo "Sync without delete" >&2
  /usr/bin/rsync -rltv $SRC $DST
else
  /usr/bin/rsync -rltv --delete $SRC $DST
  echo 2
fi


With this in place, the backup is reasonably safe even under partial failure conditions.

No comments:

Post a Comment