ADSM-L

Re: Slow Performance Restoring Large Files

2005-06-23 09:33:12
Subject: Re: Slow Performance Restoring Large Files
From: "Cahill, Ricky" <Ricky.Cahill AT EQUITAS.CO DOT UK>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Thu, 23 Jun 2005 14:32:46 +0100
Thanks for the comments.

Sadly I can't restore to a different server as only the TSM server and the SQL 
box are on this site purely for DR nothing else is attached. I have restored to 
the TSm box itself and got decent throughput.

Both network ports and switch ports have been chacked and monitored with not 
the slightest error.

The restore I did today was on a single tape, so no tape swapping at all, it 
restored 564gb in 24hrs 37mines

                ........Rikk

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Troy Frank
Sent: 23 June 2005 14:23
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: Slow Performance Restoring Large Files


One thing to check would be whether the restore/file copy goes faster to a 
different server.  This might help narrow down whether the problem's on the tsm 
server side, or the client side.  I would also check the switch ports that both 
machines are plugged into for tx/rx errors.  Could be that one of the nics is 
dying, or has a speed/duplex mismatch.  Most GB nics nowadays seem to only 
allow being set to auto, but if yours has other options, you could try 
hard-coding the switch & server to the same settings.  It might also be helpful 
to know how many tapes the restore data is spread out across.  If the tsm 
server has to load up 15 tapes, and spin each one up to the correct data 
location, the network will be sitting there twiddling it's thumbs a lot.  It's 
also worth noting that 2 LTO2 drives are not even close to capable of 
saturating a 1GB network link.  It is still odd, as you noted, that network 
utilization steadily drops over time.


Troy Frank
Network Services
University of Wisconsin Medical Foundation
608.829.5384

>>> Ricky.Cahill AT EQUITAS.CO DOT UK 6/23/2005 3:14:42 AM >>>
I've totally run out of ideas on this and am seeking some help.

One of the systems I backup is a large SQL server, I backup the diskdumps on a 
daily basis with the largest files being aprox 110gb and 250gb.

On trying to restore large files on our DR system it starts very quickly but 
after aproximatly 10-15gb the network bandwidth slowly drops from 55% to 5% and 
on the 250gb file from 55% to currently 1%

If I copy a file from the TSM server to the SQL server I get a constant 20-25% 
with only a slight dropoff in bandwidth over time.

The nic drivers and HP load balancing software has been stripped and replaced 
with the latest. I've tried the attached storage which is defaulted to raid 5 
as raid 0 and raid 1, raid 0 made no difference to the restore times or network 
bandwidth slowdown but running in raid 1 gave 28% utilisation for aprox 70gb of 
the file before the bandwidth started slowing.

I've tried it with the sql shutdown and also with the /3gb /pae off, no 
difference.

I've also tried changing the cache on the raid card from 50/50 to 75/25 and 
100/0 write/read with now difference in restore perfomance.

The restores are coming off tape and not disk.


Specs
TSM Server      HP DL380 2gb ram 
                2xcpu 
                STK L40 with 2x LTO2 drives
                TSM 5.2.3.1
                OS WIN2k3

SQL server      HP ML570
                16gb ram
                4 x cpu
                TSM client 5.2.3.1
                OS WIN2k3


Systems connected with with load balanced 1gb network.

It looks to me like it's the raid controller and storage attached to this box 
which is slowing it down but cna't see why a copy goes trhough fine but slower 
and the restore dies over time.

Thanks in advance for any help that can be given as I'm running out of ideas.

         ...........Rikk







                


****************************************************************************************************
Equitas Limited, 33 St Mary Axe, London EC3A 8LL, UK
NOTICE: This message is intended only for use by the named addressee
and may contain privileged and/or confidential information.  If you are
not the named addressee you should not disseminate, copy or take any
action in reliance on it.  If you have received this message in error
please notify postmaster AT equitas.co DOT uk and delete the message and any
attachments accompanying it immediately.

Equitas reserve the right to monitor and/or record emails, (including the
contents thereof) sent and received via its network for any lawful business
purpose to the extent permitted by applicable law

Registered in England: Registered no. 3173352 Registered address above
****************************************************************************************************

Confidentiality Notice follows:

The information in this message (and the documents attached to it, if any)
is confidential and may be legally privileged. It is intended solely for
the addressee. Access to this message by anyone else is unauthorized. If
you are not the intended recipient, any disclosure, copying, distribution
or any action taken, or omitted to be taken in reliance on it is
prohibited and may be unlawful. If you have received this message in
error, please delete all electronic copies of this message (and the
documents attached to it, if any), destroy any hard copies you may have
created and notify me immediately by replying to this email. Thank you.


****************************************************************************************************
Equitas Limited, 33 St Mary Axe, London EC3A 8LL, UK
NOTICE: This message is intended only for use by the named addressee
and may contain privileged and/or confidential information.  If you are
not the named addressee you should not disseminate, copy or take any
action in reliance on it.  If you have received this message in error
please notify postmaster AT equitas.co DOT uk and delete the message and any
attachments accompanying it immediately.

Equitas reserve the right to monitor and/or record emails, (including the
contents thereof) sent and received via its network for any lawful business
purpose to the extent permitted by applicable law

Registered in England: Registered no. 3173352 Registered address above
****************************************************************************************************

<Prev in Thread] Current Thread [Next in Thread>