ADSM-L

Performance Large Files vs. Small Files

2001-02-14 10:26:12
Subject: Performance Large Files vs. Small Files
From: "Diana J.Cline" <Diana.Cline AT ROSSNUTRITION DOT COM>
Date: Wed, 14 Feb 2001 10:04:52 -0500
Using an NT Client and an AIX Server

Does anyone have a TECHNICAL reason why I can backup 30GB of 2GB files that are
stored in one directory so much faster than 30GB of 2kb files that are stored
in a bunch of directories?

I know that this is the case, I just would like to find out why.  If the amount
of data is the same and the Network Data Transfer Rate is the same between the
two backups, why does it take the TSM server so much longer to process the
files being sent by the larger amount of files in multiple directories?

I sure would like to have the answer to this.  We are trying to complete an
incremental backup an NT Server with about 3 million small objects (according
to TSM) in many, many folders and it can't even get done in 12 hours.  The
actual amount of data transferred is only about 7GB per night.  We have other
backups that can complete 50GB in 5 hours but they are in one directory and the
# of files is smaller.

Thanks





 Network data transfer rate
 --------------------------
 The average rate at which the network transfers data between
 the TSM client and the TSM server, calculated by dividing the
 total number of bytes transferred by the time to transfer the
 data over the network. The time it takes for TSM to process
 objects is not included in the network transfer rate. Therefore,
 the network transfer rate is higher than the aggregate transfer
 rate.
.
 Aggregate data transfer rate
 ----------------------------
 The average rate at which TSM and the network transfer data
 between the TSM client and the TSM server, calculated by
 dividing the total number of bytes transferred by the time
 that elapses from the beginning to the end of the process.
 Both TSM processing and network time are included in the
 aggregate transfer rate. Therefore, the aggregate transfer
 rate is lower than the network transfer rate.