BackupPC-users

Re: [BackupPC-users] Moving the Pool

2011-05-11 14:34:47
Subject: Re: [BackupPC-users] Moving the Pool
From: "Jeffrey J. Kosowsky" <backuppc AT kosowsky DOT org>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 11 May 2011 14:32:56 -0400
Rob Sheldon wrote at about 10:42:03 -0700 on Wednesday, May 11, 2011:
 >  On Wed, 11 May 2011 10:45:01 -0500, Michael Stowe wrote:
 > > Thought I'd mention that I'm in the process of moving my 1TB backup 
 > > pool
 > > from one jfs array to another on the same system.  Both arrays are 
 > > RAID-5
 > > via mdadm.
 > >
 > > I'm using "rsync -avP -H src dest," which I started Friday, nearly 
 > > five
 > > days ago.
 > >
 > > The data in cpool moved relatively quickly.  On the other hand, rsync 
 > > has
 > > been re-establishing hard links to the pc directories for quite a 
 > > while,
 > > and has quite a ways to go.  On the plus side, it's showing no signs 
 > > of
 > > stopping or running out of resources, so it looks like it will do the 
 > > job
 > > ... eventually.
 > 
 >  I had to do this a while back and, like you, found that rsync didn't 
 >  seem to work well. I ended up using a tarpipe instead, and that worked 
 >  great. I can't find my notes for the exact chant I used for it, but this 
 >  should be a good start: 
 >  
 > http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/
 > 

How big was your pool and pc tree?
The problem is not really a rsync-specific one but rather is due to
the huge number of hard links that must be tracked and cached which
both consumes an increasing amount of memory and also potentially cpu
power to sort/find the right link...
The bottom line is that if you are using a "generic" program like
rsync or tar that doesn't understand the special structure of the pool
and pc hierarchies then essentially the inode and path of each pool
file must be cached and referenced when copying the archive. This does
not scale well as many if not all people have found.

I can believe though that tar might work ok on small archives (just
like rsync can)...

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/