Hi,

Adam Goryachev wrote on 2008-12-19 10:56:44 +1100 [Re: [BackupPC-users] 
backuppc 3.0.0: another xfs problem?]:
> Jeffrey J. Kosowsky wrote:
> >
> > I don't think that BackupPC_nightly checks for hard link dups between
> > the pc/ and pool/ directories.

I fully agree on that point.

> I would advise that you confirm whether or not your hard links were
> restored properly:
> cd /var/lib/backuppc/pool/3/3/3
> for file in `ls`

Don't you trust shell globbing? ;-)

> do
>       stat $file|grep Links|awk '{print $5" "$6}'
> done

You mean

    cd /var/lib/backuppc/cpool/3/3/3
    perl -e 'foreach (<*>) { $l {(stat $_) [3]} ++; }
             foreach (sort {$a <=> $b} keys %l) {
                 print "$l{$_} files have $_ links\n";
             }'

? ;-)

Actually, if your hard links have *not* been restored correctly, your 700GB
tar file will have been unpacked to occupy significantly more space on your
destination device (at least twice the amount, checking *before the first run
of BackupPC_nightly*). I would almost be surprised if a 700GB pool was, in
fact, restored correctly - see all of the "copying the pool" discussions for
details.

> The other possibility is that xfs is that much slower on your hardware,
> with your mount options, etc... Perhaps look at the backuppc wiki for
> some suggestions on improving performance on xfs.

I can think of one additional point to note. Your files will have been created
in a different order than before (probably the first instance of each inode
encountered is "beneath" a different directory than it was on your old pool).
If a directory references inodes scattered all over the disk and all these
inodes need to be read (such as for determining their link count), this will
be significantly slower than if the inodes are all neatly stored near each
other. Even reading the inodes in numerical order (instead of by appearance
in the directory) speeds up things (IO::Dirent optimization).
So, while your source pool was *not* in the ideal condition (for
BackupPC_nightly) of having the files stored neatly near each other (with
respect to appearance in the pool directories), your current pool may (or may
not) behave significantly worse - though I doubt we're talking of a factor of
22 here. It might be a combination of several things.

- How much memory does your BackupPC server have? How much of that is
  available as disk cache?
- What happens between 01:00:01 and 23:11:42 (the "[...]" in your quote from
  the log file)? Are there backups running during this time? Many? Any error
  messages?
- Is the machine doing anything significant that is unrelated to BackupPC
  during this time?
- Sorry if this is a stupid question: are any read errors reported for the
  disk in question?

You *can* spread out the nightly cleanup process over several nights
($Conf{BackupPCNightlyPeriod}). If you do that, does the time it takes
decrease proportionally? Do all passes take the same time?

Regards,
Holger

------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you.  Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/