How to Improve Aggregate Data Transfer Rate in the Backup?

[shahzeb]

I optimized most performance tuning parameter in the TSM Clients, which are running on Linux, and my TSM Server on MVS. While I run backup, why my aggregate data transfer rate is so slow. Please advise how can I increase the Aggregate data transfer rate.



There were no other backup ran during this time, and network is GiGi network full duplex.



Here is the sample output below.



Total number of bytes transferred: 1.58 GB

Data transfer time: 123.40 sec

Network data transfer rate: 13,456.46 KB/sec

Aggregate data transfer rate: 698.98 KB/sec

Objects compressed by: 42%

Elapsed processing time: 00:39:35



Thanks in advance.

 

[sergio.morales]

WHat does your dsm.sys look like? Are you backing up to tape or to diskpools?



sERGE

 

[shahzeb]

My backups are going to the tape directly. Here is the dsm.sys file.



SErvername xxxxxxxxx

NODename xxxxxxxxx



COMMmethod TCPip

SCHEDMode POLLING



TCPPort 1500

TCPClientport 1501

HTTPPort 1581



TCPServeraddress xxxxxxx

TCPClientaddress xxxxxxxxx



PASSWordaccess Generate



COMPression YES

RETRYPERIOD 1

CHANGINGRETRIES 1

LARGECOM YES



TCPW 1024

TXNB 2097152



SCHEDLOGR 5D

ERRORLOGR 5D



Schedlogname /opt/tivoli/tsm/client/ba/bin/dsmsched.log

Errorlogname /opt/tivoli/tsm/client/ba/bin/dsmerror.log



RESOURCEUTILIZATION 10



*MANAGEDSERVICES WEBCLIENT SCHEDULE

 

[sergio.morales]

Hey! I think you may not have enough tapes in the scratchpool that TSM is writing to. How many tapes are there, and how many tape drives do you have available to it? As a side note, I would lower the resourceutilization setting to something like 6 . . .



sERGE

 

[shahzeb]

I have 400 Gen. 2 tape, and six drives available. I will make that change for RESOURCEUTILIZATION. Thanks.

 

[cheffern]

Make sure you take a look at this also:



"Resourceutilization and maxxnummp (max mount points) work together."

IE, if you change one, you should be checking the other's value as well.

 

[robbenleaf]

Also, compression on the client side can slow things down - and if your tape drives do hardware compression, it's not really necessary.



You might want to try it with compression turned off, and see how much that improves performance.

 

[shahzeb]

I have turned off the compression, but still no change. I also change other parameters as suggested. Here is the latest output.



Total number of objects inspected: 165,579

Total number of objects backed up: 70

Total number of objects updated: 0

Total number of objects rebound: 0

Total number of objects deleted: 0

Total number of objects expired: 1

Total number of objects failed: 0

Total number of bytes transferred: 5.37 MB

Data transfer time: 0.76 sec

Network data transfer rate: 7,218.70 KB/sec

Aggregate data transfer rate: 59.69 KB/sec

Objects compressed by: 87%

Elapsed processing time: 00:01:32

tsm>



This should have take fraction of seconds, but no improvement. Open for suggestion. Thanks.

 

[may]

Objects compressed by: 87%



==> you still have compression on your Client,

are you sur you set it to "NO"



have a look if this is not force by your "Client OptionSet " parameter on your TSM server.



try also with the:



TCPNODELAY YES

 

[AndyRH]

I have one question:

You scanned 165,679 objectes in 1 minute and 32 seconds and you are not happy with the performance? Maybe it can be done in half the time, but you save only 4.66 hours per year...



Seriously, your system scanned 165K objects and sent 70 (5.37mb) of them in 1:32. The aggregate rate is based on total bytes backed up divided by total backup time.



Andy

 

[shansell]

We have a lot of clients on 10/100 lan's, and they look just like your Network Transfer Rate. Is the switch this nic is plugged into running at an "AUTO" setting? And if it is, what setting do you have on your nic. Seems like you are not getting gig speed.. :-o



However, I don't know that would increase this particular backup by much, maybe 20-25%

 

[shahzeb]

The whole backup network is running on dedicated Gigi network, and configured as full duplex in both end.



I have configured for other clients, where they backed up million files in a minute with the exeact same configuration for the TSM Clients, and on an average others have backed up giga bytes of data per minute in a dedicated gigi network. I know larger files do faster backup. Here is a latest result from another TSM client's full system backup. Please advise. Thanks



06/03/05 23:05:30 Total number of objects inspected: 180,242

06/03/05 23:05:30 Total number of objects backed up: 180,221

06/03/05 23:05:30 Total number of objects updated: 0

06/03/05 23:05:30 Total number of objects rebound: 0

06/03/05 23:05:30 Total number of objects deleted: 0

06/03/05 23:05:30 Total number of objects expired: 0

06/03/05 23:05:30 Total number of objects failed: 0

06/03/05 23:05:30 Total number of bytes transferred: 1.75 GB

06/03/05 23:05:30 Data transfer time: 405.50 sec

06/03/05 23:05:30 Network data transfer rate: 4,528.79 KB/sec

06/03/05 23:05:30 Aggregate data transfer rate: 495.74 KB/sec

06/03/05 23:05:30 Objects compressed by: 39%

06/03/05 23:05:30 Elapsed processing time: 01:01:44

 

[sergio.morales]

Have you attempted a smaller tcpwindowsize? 1024 seems rather large . . . try going to 64 and going up from there . . .



sERGE

 

[nbs03]

Zeb,

Is this slowness system wide or isolated to a few clients?

If it is system wide, have you looked at the TSM Server performance? It could be that the database lookups are slow due misconfigured disks.

If this is only a few clients, run a netstat and look for network problems. I recently had a Solaris 10 client which was set to NOT autonegotiate and forced to 100/FDX It ran fine until it was moved to a new switch and then started having errors. Setting it to autonegotiate solved the problem. You may also want to run traceroute to see how many routers you are going through to get from the client to the server. One of the routers may be misconfigured.



cheers,

neil

 

[klee]

Your backups must be getting compressed by client. The Objects compressed will always be 0% if you are not using compression.



In your example where 165k files are inspected and only 70 are actually backed up everything looks fine to me. The whole backup only takes 1.5 mins and the network transfer time is less than a second. The extra 1.5 mins is probably the TSM server having to inspect another 165k files to see if they need to be backed up.



I would be more concerned about your example that inspects and backs up 180k files. The elapsed time is over an hour and the network takes only 8 minutes or so. Since you are backing up all of the files I would expect the elapsed time to be closer to the network transfer time. It states that compression is 39%, if this is correct the client will be compressing over a Gb of data before it is transferred over the network and backed up. If the client does not have sufficient processor resources (i.e speed and number of processors) and has to compete with other applications on the same server then compression can take some time.



The network transfer speed is not too bad in either of these cases. What type of tape drives are you using? You might have gbit network but the backups will only be as fast as the tape drives since you are not using disk backup pools . When backup is started how long before tape is mounted?. It takes about 45 secs using our tape library. So if i'm only backing up a single 50k file to tape it will still take up to a minute.



I would seriously look at the server and client settings for compression.

 

[Bartacus]

One other thing to consider since your on MVS like us: is your TCP/IP stack shared by other apps? If you don't have a TCP/IP stack solely for TSM, or don't have TSM on its own LPAR, maybe another app is competing with TSM for TCP/IP cycles?



And definetly kill that client-side data compression.



Bart

 

[mome0407]

I already have the same, I don't know what to do.
but there is a note, when i take the backup as incremental the aggregate raise.

 

[sadmemories20]

I would be more concerned about your example that inspects and backs up 180k files. The elapsed time is over an hour and the network takes only 8 minutes or so. Since you are backing up all of the files I would expect the elapsed time to be closer to the network transfer time. It states that compression is 39%, if this is correct the client will be compressing over a Gb of data before it is transferred over the network and backed up. If the client does not have sufficient processor resources (i.e speed and number of processors) and has to compete with other applications on the same server then compression can take some time.


__________________
watch online movies