AW: Backup taking too long

As i know resourceutil stays in the same filesystem and only starts a new
thread if there is something big to transport to the tsm server. Searching
for changed files isn't multithreaded.

I devided backup of 4 large netapp volumes ~1TByte/4 Mio Files) with 5 nodes
using the following 
include/exclude lists successfully:

for each node i set up a dsm.opt like that:
TCPBUFFSIZE               256
TCPWINDOWSIZE             63
TCPNODELAY                YES
COMPRESSION     YES    

* Managementclass for NAS Backups for each volume
INCLUDE \\sznas01\tsmdata1$\...\*  DATAX
INCLUDE \\sznas01\tsmdata2$\...\*  DATAX
...

* For each volume
* for each node comment one range out
* EXCLUDE.DIR \\sznas01\tsmdata1$\[A-E]*
EXCLUDE.DIR \\sznas01\tsmdata1$\[F-J]*
EXCLUDE.DIR \\sznas01\tsmdata1$\[K-O]*
EXCLUDE.DIR \\sznas01\tsmdata1$\[P-T]*
EXCLUDE.DIR \\sznas01\tsmdata1$\[U-Z0-9$]*
...

The backup itself had a preschedcmd:
create new snapshots for all volumes
create new hidden shares to the root of the volumes

start backup with all 5 nodes at the same time
Netapp FS840 reached about 95 % CPU Util during backup
With one DL380 G3 (1 GB Ethernet) i could do the backup for 12 mio files in
about 20 hours

Stefan Holzwarth


-----Ursprüngliche Nachricht-----
Von: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] Im Auftrag von
Scott Foley
Gesendet: Montag, 1. August 2005 18:19
An: ADSM-L AT VM.MARIST DOT EDU
Betreff: Backup taking too long

I am backing up a large number of files and directories located on a
Network Appliance.  I am using a Windows 2000 client to do this.  It is
taking over 40 hours to do the backup.  I also run out of memory at
times (ANS1028S).   Journaling worked great when the files were on a
Windows system, but journaling works only on a local file system.  I
don't think I can use NDMP because the tape library is connected only to
the Tivoli server.  Most of the time (and memory) is spent determining
what files should be backed up.  There are over 1 million directories.


Total number of objects inspected: 7,819,693
Total number of objects backed up:  260,984
Total number of bytes transferred:    36.22 GB
Data transfer time:                4,579.15 sec
Elapsed processing time:           40:10:58

There are about 36 "root" directories so I could configure the Network
Appliance to have 36 shares and back up each separately if that would
help.  I think this would help with the memory problem.  I would
probably need to run multiple backups simultaneously though.    

I have added the following to the dsm.opt file

RESOURceutilization 14
MEMORYEFFICIENTBACKUP yes

Any suggestions on how to speed the backup up?  


Windows 2000 SP2 with 512 Meg of Memory
Client  Version 5, Release 1, Level 7.0
Server  Version 5, Release 1, Level 9.0

Scott Foley