Re: [BackupPC-users] Moving lots of data on a client

On Tue, Aug 20, 2013 at 02:23:38PM -0400, Raman Gupta wrote:
> I have a client on which about 100 GB of data has been moved from one
> directory to another -- otherwise its exactly the same.
> 
> As I understand it, since the data has been moved, BackupPC 3 will
> transfer all the data again (and discard it once it realizes the data
> is already in the pool) i.e. it does not skip the transfer of each
> file even though the checksum is identical to an existing file in the
> pool.

That is also my understanding.
 
> I am using the rsync transfer method.
> 
> Is there a workaround to prevent all 100 GB of data from being
> transferred again?

Your mileage may vary on this, but changing the structure of the data
on the backuppc system to match what is currently present on the
client should work.  Asuuming you moved the data from:

   client1:/location/dir1

to
  client1:/location/dir2

and the share is /location (and directory dir1 and dir2 are
subdirectories).

Find the number for the last full backup and the last incremental
backup for the host in the web interface. (In theory any filled backup
should work in place of the full backup, but you want a tree that has
all the files in their original structure. Regular (unfilled)
incremental backups will have only the file changed since the last
full backup.)

Assuming your backuppc tree containing the subdirectorues cpool, pool,
pc and so on is at BACKUPPC do the following:

 cd BACKUPPC/pc/client1/<full backup number>/f%2flocation

(%2f is an encoded / and the leading f is how backuppc identifies
backed up files/directories rather than metadata like "attrib" and
"backupInfo" files)

  sudo -u backup cp -rl fdir1 ../<incremental backup number>/f%2flocation/fdir2

where backup is the user backuppc runs as. This will create a copy of
the fdir1 tree with links for every file (because of the -l (lower
case l flag)) and do it recursively (the -r flag). The target is
"fdir2" in the last incremental backup. The last incremental backup is
going to be the reference backup for the next full backup.

The copy may fail to links some files if the maximum number of
links to the file is exceeded. If that's the case, the next full
backup should just copy those files into place.

Once this is done, start a new full backup and with luck it will see
the files in the f2 tree and start comparing them with checksums
rather than copying them. You may want to start the full backup using
the BackupPC_dump command directly with verbose mode turned on so you
can get more details on what's happening/transferring.

(
If the move involved directory level changes say from: /location/dir1
to /location2/subdir1/dir2, you will have to create the appropriate
directory structure under the last incremental directory. In this case
it would be BACKUPPC/<last incremental number>/f%2flocation2/fsubdir1
and then use:

  sudo -u backup cp -rl fdir1 ../<incremental backup
number>/f%2flocation2/fsubdir1/fdir2
)

I think I got the essentials here, last time I did this was with data
move between two different hosts and it was a few years ago. IIRC the
attrib file isn't used for determining what directories are in the
reference backup, so it can be ignored for this process.

You may want to wait a day or so to see if anybody else has some
comments, but I claim this should be safe since no data is harmed by
the copy/link 8-).

Standard disclaimers apply.

Let us know how/if it works.

-- 
                                -- rouilj

John Rouillard       System Administrator
Renesys Corporation  603-244-9084 (cell)  603-643-9300 x 111

------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and 
AppDynamics. Performance Central is your source for news, insights, 
analysis and resources for efficient Application Performance Management. 
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/