|
Re: [Networker] How to speed up this backup (4GB -> 8 hours) ?
2009-03-20 16:28:54
On 21/03/2009, at 04:39 , Howard Martin wrote:
Some statements that Networker is bad with small files/large numbers
of
files are (in my opinion ) misdirected all filesystem based backups
will
have this problem, to avoid it you need to ignore the filesystem and
read
disk blocks directly.
As soon as you hit dense filesystems, where the cost of walking the
filesystem is high, you're going to have backup performance problems
regardless of which backup product you use without a change of tact.
One solution, as Howard suggested, is to eliminate the filesystem and
move to a block level transfer. This can achieve very high speed
backups; the problem however is that if you have a dense filesystem
that also has significant fragmentation, you can instead replace one
problem with another - the backup becomes high speed, but file level
recovery from block level can slow to an absolute crawl as files must
be reconstructed across blocks retrieved from a physically large area
of media. If individual file level recovery is what you need most,
it's likely this is going to be a decision killer.
The other solution is to move towards using much higher levels of
parallelism. This works best when the dense filesystem is hosted on a
well-designed LUN - e.g., sufficient spindles as to allow a high
number of concurrent walks. I.e., while 1 walk/backup may take say, 16
hours to complete, doing 4 or 8 concurrently may allow the backup to
complete in much less time - e.g., 4 hours. This isn't a perfect
solution, BUT, if you do it correctly (I.e., ensure that your
mechanism for selecting subdirectories to backup does not result in
new directories being missed!), it usually works more for more people
than block level backup. Even more so if you're backing up to disk or
VTL, since neither of those devices shoe-shine.
Cheers,
Preston.
--
Preston de Guise
"Enterprise Systems Backup and Recovery: A Corporate Insurance Policy":
http://www.amazon.com/Enterprise-Systems-Backup-Recovery-Corporate/dp/1420076396
http://www.enterprisesystemsbackup.com
NetWorker blog: http://nsrd.wordpress.com
To sign off this list, send email to listserv AT listserv.temple DOT edu and type
"signoff networker" in the body of the email. Please write to networker-request
AT listserv.temple DOT edu if you have any problems with this list. You can access the
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|
|
|
