Hi all,

SUMMARY:

To backup ntfs-3g shares with rsync_bpc, override the variable
RsyncArgs (and RsyncRestoreArgs), and delete these parameters:

  --links
  --hardlinks 

If not, rsync will keep sending all files over and over again.
This means that if you have a host with ntfs-3g partition and other
partitions for which you want to keep hard links, you must create an
host alias in backup pc for backing up the ntfs-3g shares.

----------------------------------------------------------------------

GORY DETAILS:

I had some trouble configuring BackupPC 4.0 alpha 3 to work correctly
when backing up ntfs-3g partitions on linux.

So, for short, I have a few machines that dual boot Linux / Windows, but
run mostly under Linux, and which I back up with BPC 4.0a3. To backup
the ntfs partition, I just mount them via ntfs-3g, then configure
backuppc to backup these partitions via rsync (using rsync daemon on
host).

But the default bpc configuration does not work: at each backup, rsync
(so rsync_bpc in fact) will systematically send again all files on the
ntfs-3g partitions, to finally report that all files are identical.

The oddity is that standard rsync does not exhibit this behavior. By
experimenting with rsync_bpc, and then looking at the parameters, the
problem was in fact obvious: by default, rsync runs with parameters
"--links" and "--hard-links". This does not work with rsync_bpc on
ntfs-3g. Why it does work with rsync is just a mystery for me yet.


To reproduce the problem, here what I did:

---------------------------
TEST SETUP
---------------------------

- Content of /etc/fstab

UUID=445E1B975E1B813A /c ntfs ro,defaults,nls=utf8,umask=007,gid=46 0
0

- Create a test share in /etc/rsyncd.conf:

max connections = 2
log file = /var/log/rsync.log
max verbosity = 3
transfer logging = yes
log format = %o %i %h [%a] %m (%u) %f %l
timeout = 18000              # 5.2GB/hr, so 20GB files requires ~4hr

read only = yes
list = yes
uid = root
gid = root
auth users = backuppc
secrets file = /etc/rsyncd.secrets

[test1]
comment = Test share 1
path = /c/Boot

- In a separate window, monitor the rsyncd log file: 

   tail -f /var/log/rsync.log

- Create rsync password file: 

   sudo -v; sudo sed -r 's/^.*://' /etc/rsyncd.secrets | sudo
tee /tmp/rsyncd.secrets > /dev/null; sudo chmod 600 /tmp/rsyncd.secrets


------------------------------------------------------
Test case one - PASS - rsync + ntfs-3g
------------------------------------------------------
- Initialize: 

sudo rm -rf /tmp/rsynctest; sudo mkdir /tmp/rsynctest

- Repeat the following step, and observe that files are only txfed the
first time: 

sudo rsync --super --recursive --protect-args --numeric-ids --perms
--owner --group -D --times \
--links --hard-links --delete --partial --log-format='log: %o %i %B %8U,
%8G %9l %f%L' --stats \
--modify-window=5 --one-file-system --password-file=/tmp/rsyncd.secrets
\
--checksum backuppc@griffin::test1 /tmp/rsynctest


------------------------------------------------------
Test case two - FAIL - rsync_bpc + ntfs-3g
------------------------------------------------------

- Initialize: 

sudo rm -rf /tmp/rsynctest; sudo mkdir -p /tmp/rsynctest/cpool; i=0

- Repeat the following step, and observe that files are always txfed: 

cd /tmp/rsynctest; i=$((i+1)); echo backup $i; sudo cp -ar
pc/griffin/$((i-1)) pc/griffin/$i; \
sudo rsync_bpc --bpc-top-dir /tmp/rsynctest --bpc-host-name griffin
--bpc-share-name test1 --bpc-bkup-num $i \
--bpc-bkup-comp 3 --bpc-bkup-prevnum $((i-1)) --bpc-bkup-prevcomp 3
--bpc-log-level 3  --super --recursive \
--protect-args --numeric-ids --perms --owner --group -D --times --links
--hard-links --delete --partial \
--log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats --modify-window=5
--one-file-system \
--password-file=/tmp/rsyncd.secrets backuppc@griffin::test1 /


Increase --bpc-log-level (say set it to 9) to get more debug output. 


------------------------------------------------------
Test case three - PASS - rsync_bpc + ntfs-3g, fixed
------------------------------------------------------
- Initialize: 

sudo rm -rf /tmp/rsynctest; sudo mkdir -p /tmp/rsynctest/cpool; i=0

- Repeat the following step, and observe that files are only txfed the
first time when removing parameters --links --hard-links: 

cd /tmp/rsynctest; i=$((i+1)); echo backup $i; sudo cp -ar
pc/griffin/$((i-1)) pc/griffin/$i; \
sudo rsync_bpc --bpc-top-dir /tmp/rsynctest --bpc-host-name griffin
--bpc-share-name test1 --bpc-bkup-num $i \
--bpc-bkup-comp 3 --bpc-bkup-prevnum $((i-1)) --bpc-bkup-prevcomp 3
--bpc-log-level 3  --super --recursive \
--protect-args --numeric-ids --perms --owner --group -D --times --delete
--partial \
--log-format='log: %o %i %B %8U,%8G %9l %f%L' --stats --modify-window=5
--one-file-system \
--password-file=/tmp/rsyncd.secrets backuppc@griffin::test1 /


--
Michaël




------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors 
network devices and physical & virtual servers, alerts via email & sms 
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/