Networker

Re: [Networker] Incremental backup of large volume takes longer than full

2006-10-27 06:09:03
Subject: Re: [Networker] Incremental backup of large volume takes longer than full
From: Paul Brears <paul AT IFL DOT NET>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Fri, 27 Oct 2006 11:04:16 +0100
Three things to check:



Firstly; when you setup the journal you can specify a size of journal to 
allocate.

On busy file systems the default option may not be big enough. If it's not 
large enough the journal 
will wrap over and Networker will have to calculate the incremental in the 
traditional way.



Secondly if you have lots of directories, Networker looks for a NSR.DIR file in 
each directory as it 
scans (and of course it's unlikely to be present), on large file systems this 
seems to be an 
expensive operation. There is an NT environment variable that allows you to 
tell Networker not to 
look for these files also "save -i" will ignore directive files.  If you run 
sysinternals filemon on 
the system (or preferably an equivalent development platform) you see what it's 
actually doing. On 
one system I saw it was spending 40% of the time looking for files that didn't 
exist.



Thirdly look out for under performing file systems. We had one that we 
'upgraded' from a raid 5 to a 
simple mirror and got a significant  speed improvement.  Look for disk counters 
in performance 
monitor. Turn on write back caching if you can. (Networker's reads don't get 
interrupted by writes)



Paul



----- Original Message ----
From: Kobus Strydom <kstrydom AT MEDIHELP.CO DOT ZA>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Sent: Thursday, October 26, 2006 7:51:26 PM
Subject: [Networker] Incremental backup of large volume takes longer
than full

Please can someone help.

I have a Windows 2003 server HPDL580 4 x 3.2Ghz, 4GB ram, 72GB local
disk, 500GB EMC volume, Gigabit ether, 2GB Fiber for backup purpose.
Networker 7.3.1

On the EMC volume are very roughly about 30mil files, all images.

A full backup of this volume takes roughly 36hours. There are about
10GB's worth of new images added daily, yet the incrementals that I
perform takes forever and seldom completes before 11:am the next
working
day. They start at 18:00 each night. The speed I get out of
incrementals
are in the region of 500KB to 1MB p/s.

I have journal manager enabled on this volume, yet it looks lik all
files are checked for changes?

Any suggestions/help would be greatly appreciated. I have worked with
our local supplier on this issue for months now and still no joy. They
suggested a raw backup, which is great, I get an average speed of 50MB
p/s, but this doesn't help when I need to restore individual files.

Kobus Strydom
Systems Engineer
Medihelp
Mobile +27 82 9090882
Tel +27 12 3342295
Fax +27 12 3342426
e-mail kstrydom AT medihelp.co DOT za














Medihelp is an Authorised Financial Services Provider
---------------------------------------------------------------------------
NOTICE: Please note that this eMail, and the contents thereof,
is subject to the standard Medihelp eMail disclaimer which may be found
at:
http://www.medihelp.co.za/disclaimer.htm
<<<< GWAVASIG >>>>

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems
wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems
wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in 
the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER