ADSM-L

Re: Long, long, long backup sessions

2001-08-24 11:48:14
Subject: Re: Long, long, long backup sessions
From: Thierry ITTY <thierry.itty AT BESANCON DOT ORG>
Date: Fri, 24 Aug 2001 11:48:14 -0400
as a kind of comparison you can see here the same snippet for our GIS
server, hp9000/j6000 1GB/20GB tsm client 4.1, much smaller but... tsm
server is a rs6k/f50 2x332 MHz 640 MB, network is ATM/155 Mbps

Total number of objects inspected:  333 895
Total number of objects backed up:    1 349
Total number of objects updated:          8
Total number of objects rebound:          0
Total number of objects deleted:          0
Total number of objects expired:        193
Total number of objects failed:           0
Total number of bytes transferred:   609,53 MB
Data transfer time:                   39,74 sec
Network data transfer rate:        15 704,95 KB/sec
Aggregate data transfer rate:        246,10 KB/sec
Objects compressed by:                    0%
Elapsed processing time:           00:42:16   

my network data transfer rate is 15 MB/sec while yours is 520 KB/sec. even
if it is not the only problem, sure it may be a big one sooner or later.

also i see you compress your datas. did you try without compression ?

did you run vmstat, sar, or other such tools to try and find any contention
(you might be IO or CPU bound) ?

did you try multi-threading (heard about it but don't know a lot) ?

run a 1/2 GB archive with a simple pattern (a single FS or a subdir with
some big files) to try and catch contentions. with an archive you'll really
fill the pipes and avoid object inspections which is time consuming.

create a new tsm node and try with an older version client (3.1.0.1 or any
< 3.1.0.6) . compare the behaviour (warning : the first time you'll do a
full backup so it could be a good idea to modify dsm.opt to include only a
part of your disk storage and exclude the remainder !)

btw "my" elapsed processing time _is_ the time the command ran from start
to end (42 minutes). 

well i hope you'll enjoy a nice testing week-end :-)

hth




A 11:24 24/08/01 -0400, vous avez écrit :
>Looking for some guidance to improve backup session times.
>
>The time it takes to backup our e-mail servers (4-way processor H70 AIX
>4.3.3 with 2GB RAM. TSM client is 4.1.2.0) is becoming unacceptable and
>unworkable.
>
>Currently, backup sessions are sometimes taking more than 16-hours (as I
>send this, one session that started yesterday evening is still running),
>yet the total amount of data backed up is only around 2.5-4GB. The current
>session stats shows the first data transfer wasn't until around 3:45 this
>morning, yet the backup session started around 6PM, YESTERDAY.
>
>Yes, I understand the LARGE amount of OBJECTS INSPECTED is an issue. But,
>the current data tells me it took over 8-hours from the start of the
>schedule to when it actually transfered any data from the client to the
>server (OS390 TSM 4.1.3).
>
>Here is a end-of-schedule snippet from a previous days run. Note, it
>started at 6PM and finished at 9AM the next day. When it actually did any
>work, it went pretty fast:
>
>08/23/01   09:00:37 --- SCHEDULEREC STATUS BEGIN
>08/23/01   09:00:37 Total number of objects inspected: 8,521,197
>08/23/01   09:00:37 Total number of objects backed up:  105,231
>08/23/01   09:00:37 Total number of objects updated:          0
>08/23/01   09:00:37 Total number of objects rebound:          0
>08/23/01   09:00:37 Total number of objects deleted:          0
>08/23/01   09:00:37 Total number of objects expired:     32,662
>08/23/01   09:00:37 Total number of objects failed:           7
>08/23/01   09:00:37 Total number of bytes transferred:     2.83 GB
>08/23/01   09:00:37 Data transfer time:                5,685.47 sec
>08/23/01   09:00:37 Network data transfer rate:          522.69 KB/sec
>08/23/01   09:00:37 Aggregate data transfer rate:         55.16 KB/sec
>08/23/01   09:00:37 Objects compressed by:                   51%
>08/23/01   09:00:37 Elapsed processing time:           00:02:48
>08/23/01   09:00:37 --- SCHEDULEREC STATUS END
>08/23/01   09:00:37 --- SCHEDULEREC OBJECT END BACKUP.MAIL 08/22/01
>18:00:00
>
>The "Elapsed processing time" is confusing. Is it trying to tell me it
>only took 3-minutes to actually transfer the 2.83GB ??
>
>What can we do to improve these times ?   Is INCRBYDATE going to make that
>much of a different, considering we will still have to do some kinda full
>backup, anyway ?
>
>Any and all suggestions will be seriously considered !
>
>===========================
>Zoltan Forray
>Virginia Commonwealth University
>University Computing Center
>e-mail: zforray AT vcu DOT edu
>voice: 804-828-4807
>
>
                        - * - * - * - * - * - * -
Si nous avons chacun un objet et que nous les echangeons, 
   nous avons encore chacun un objet.
Si nous avons chacun une idee et que nous les echangeons,
   nous avons alors chacun deux idees.

Thierry ITTY
eMail : Thierry.Itty AT Besancon DOT org               FRANCE