ADSM-L

Re: Performance Large Files vs. Small Files

2001-02-14 10:52:42
Subject: Re: Performance Large Files vs. Small Files
From: David Longo <David.Longo AT HEALTH-FIRST DOT ORG>
Date: Wed, 14 Feb 2001 10:52:40 -0500
I don't have the fine detail reason but the "in a nutshell" reason is that for 
every file backed up there is a compare between the *SM server DB and the 
client as to file name and date to see if it needs to be backed up or what.  
(That to me is main reason for lot of small files taking longer than a few 
large files.)

This can be improved some with tuning some COMM parameters.  I haven't done 
much in that area, maybe someone could enhance on that.


David B. Longo
System Administrator
Health First, Inc. I/T
3300 Fiske Blvd.
Rockledge, FL 32955-4305
PH      321.434.5536
Pager  321.634.8230
Fax:    321.434.5525
david.longo AT health-first DOT org


>>> Diana.Cline AT ROSSNUTRITION DOT COM 02/14/01 10:04AM >>>
Using an NT Client and an AIX Server

Does anyone have a TECHNICAL reason why I can backup 30GB of 2GB files that are
stored in one directory so much faster than 30GB of 2kb files that are stored
in a bunch of directories?

I know that this is the case, I just would like to find out why.  If the amount
of data is the same and the Network Data Transfer Rate is the same between the
two backups, why does it take the TSM server so much longer to process the
files being sent by the larger amount of files in multiple directories?

I sure would like to have the answer to this.  We are trying to complete an
incremental backup an NT Server with about 3 million small objects (according
to TSM) in many, many folders and it can't even get done in 12 hours.  The
actual amount of data transferred is only about 7GB per night.  We have other
backups that can complete 50GB in 5 hours but they are in one directory and the
# of files is smaller.

Thanks





 Network data transfer rate
 --------------------------
 The average rate at which the network transfers data between
 the TSM client and the TSM server, calculated by dividing the
 total number of bytes transferred by the time to transfer the
 data over the network. The time it takes for TSM to process
 objects is not included in the network transfer rate. Therefore,
 the network transfer rate is higher than the aggregate transfer
 rate.
.
 Aggregate data transfer rate
 ----------------------------
 The average rate at which TSM and the network transfer data
 between the TSM client and the TSM server, calculated by
 dividing the total number of bytes transferred by the time
 that elapses from the beginning to the end of the process.
 Both TSM processing and network time are included in the
 aggregate transfer rate. Therefore, the aggregate transfer
 rate is lower than the network transfer rate.



"MMS <health-first.org>" made the following
 annotations on 02/14/01 10:57:45
------------------------------------------------------------------------------
This message is for the named person's use only.  It may contain confidential, 
proprietary, or legally privileged information.  No confidentiality or 
privilege is waived or lost by any mistransmission.  If you receive this 
message in error, please immediately delete it and all copies of it from your 
system, destroy any hard copies of it, and notify the sender.  You must not, 
directly or indirectly, use, disclose, distribute, print, or copy any part of 
this message if you are not the intended recipient.  Health First reserves the 
right to monitor all e-mail communications through its networks.  Any views or 
opinions expressed in this message are solely those of the individual sender, 
except (1) where the message states such views or opinions are on behalf of a 
particular entity;  and (2) the sender is authorized by the entity to give such 
views or opinions.
This message is for the named person's use only.  It may contain confidential, 
proprietary, or legally privileged information.  No confidentiality or 
privilege is waived or lost by any mistransmission.  If you receive this 
message in error, please immediately delete it and all copies of it from your 
system, destroy any hard copies of it, and notify the sender.  You must not, 
directly or indirectly, use, disclose, distribute, print, or copy any part of 
this message if you are not the intended recipient.  Health First reserves the 
right to monitor all e-mail communications through its networks.  Any views or 
opinions expressed in this message are solely those of the individual sender, 
except (1) where the message states such views or opinions are on behalf of a 
particular entity;  and (2) the sender is authorized by the entity to give such 
views or opinions.

 ===========================================================================