Slow backup on folders that contain large amount of files

[Thma]

Hi All,

I have an issue with one of my backup client, the backup of the e drive takes forever to complete. I think i have narrowed the proble down to a couple of folders that contain a lot off small files the numbers very from 6000 to 260 000 files in each folder.

Running the backup of one folder containing 129 000 files
with -TESTFLAG=INSTRUMENTETAIL command
reveals that the server spends most of the time qurey server files.

Does anybody know how to speed things up a bit?

I have attached the report file from the command

====================> C O M M E N C E N E W R E P O R T <====================
====================> C O M M E N C E N E W R E P O R T <====================

------------------------------------------------------------------
TSM Client final instrumentation statistics: Tue Mar 15 17:54:07 2011
Instrumentation class: Client detail
Completion status: Success
------------------------------------------------------------------
------------------------------------------------------------------
No instrumented activity reported for thread 7844
------------------------------------------------------------------
Detailed Instrumentation statistics for
Thread: 7420 Elapsed time = 5942.027 sec
Section Actual(sec) Average(msec) Frequency used
------------------------------------------------------------------
Compute 0.000 0.0 1
BeginTxn Verb 0.000 0.0 1
Transaction 0.000 0.0 1
File I/O 0.000 0.0 2
Data Verb 0.000 0.0 1
EndTxn Verb 0.121 121.0 1
Thread Wait 5941.770 5941770.0 1
Other 0.136 0.0 0
------------------------------------------------------------------
Detailed Instrumentation statistics for
Thread: 6700 Elapsed time = 5941.770 sec
Section Actual(sec) Average(msec) Frequency used
------------------------------------------------------------------
Query Server Dirs 0.015 15.0 1
Query Server Files 5901.513 5901513.0 1
Process Dirs 33.110 16555.0 2
Solve Tree 0.000 0.0 1
Sleep 0.062 62.0 1
Thread Wait 0.062 62.0 1
Other 7.008 0.0 0
------------------------------------------------------------------
No instrumented activity reported for thread 7568
------------------------------------------------------------------
No instrumented class associated with thread 4296
Current command:
i \\server\e$\ptc\vault\defaultcachefolder5\ -TESTFLAG=INSTRUMENTETAIL
IBM Tivoli Storage Manager
Command Line Backup-Archive Client Interface
Client Version 6, Release 2, Level 1.0
Client date/time: 03/15/2011 17:54:07
Options settings:
BACKUPREGISTRY: YES
CHANGINGRETRIES: 4
COLLOCATEBYFILESPEC: NO
COMMMETHOD: TCP/IP
COMPRESSALWAYS: YES
COMPRESSION: NO
DISKBUFFSIZE: 32
ENABLELANFREE: NO
ENCRYPTIONTYPE: AES128
FOLLOWSYMBOLIC: 2
IMAGEGAPSIZE: 32
LANFREECOMMMETHOD: NAMED PIPE
MAKESPARSEFILE: YES
MAXCMDRETRIES: 2
MEMORYEFFICIENTBACKUP: NO
PASSWORDACCESS: 1
PRESERVELASTACCESSDATE: NO
PROCESSORUTILIZATION: 0
REPLACE: 1
RESOURCEUTILIZATION: 2
SKIPMIGRATED: NO
SKIPNTPERMISSIONS: NO
SKIPNTSECURITYCRC: NO
SNAPSHOTCACHESIZE: 2
SUBDIR: 2
SUBFILEBACKUP: NO
SUBFILECACHESIZE: 10 MB
TAPEPROMPT: NO
TCPBUFFSIZE: 32 KB
TCPNODELAY: YES
TCPWINDOWSIZE: 64 KB
TXNBYTELIMIT: 25600K
VERBOSE: 4
DEDUPLICATION: NO
ENABLEDEDUPCACHE: YES
-------------------------
Session established with server BCK-03: Windows
Server Version 6, Release 1, Level 2.1
Server date/time: 03/15/2011 17:53:58 Last access: 03/15/2011 17:52:45
Total number of objects inspected: 129,018
Total number of objects backed up: 1
Total number of objects updated: 0
Total number of journal objects: 0
Total number of objects rebound: 0
Total number of objects deleted: 0
Total number of objects expired: 0
Total number of objects failed: 0
Total number of bytes transferred: 274 B
LanFree data bytes: 0 B
Data transfer time: 0.00 sec
Network data transfer rate: 0.00 KB/sec
Aggregate data transfer rate: 0.00 KB/sec
Total number of bytes pre-compress: 251
Total number of bytes post-compress: 251
Objects compressed by: 0%
Total data reduction ratio: 100.00%
Elapsed processing time: 01:39:02
Average file size: 251 B

 

[tariq.kaifi]

You can use Jounalling for E: drive.

 

[kreddy]

Hi Thma

I have been suffering with the same problem. It is on the E: drive too....
Were You able to fix this problem??
Mine is a windows 2000 box on 5.3.6.0