• Please help support our sponsors by considering their products and services.
    Our sponsors enable us to serve you with this high-speed Internet connection and fast webservers you are currently using at ADSM.ORG.
    They support this free flow of information and knowledge exchange service at no cost to you.

    Please welcome our latest sponsor Tectrade . We can show our appreciation by learning more about Tectrade Solutions
  • Community Tip: Please Give Thanks to Those Sharing Their Knowledge.

    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.

  • Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)

    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

Large File Backup in Windows

Andrew21210

ADSM.ORG Member
#1
I have a Windows cluster that backs up some large flat file database backups. There are several files that are anywhere from 500GB to 1TB in size with the typical total size of the backup being 2 to 3TB. Is there any way that I could reduce the backup time for this cluster? The resourceutilization is set to 10 and the maxnummp is set to 8 but these don't come into play when there is a single 1TB file being backed up. Could I get better performance by playing with tcpwindowsize, txnbytelimit and/or tcpbuffsize parms on the client?
 

marclant

ADSM.ORG Moderator
#2
Could I get better performance by playing with tcpwindowsize, txnbytelimit and/or tcpbuffsize parms on the client?
Possibly, but probably not to the point where you need it to be. I'd just try to tweak the tcpwindow size, not as much to gain with the others.

If you're backing up to a container pool, I'd consider client-side dedup and compression, that would cut down the amount of data to send to the server.
 

Andrew21210

ADSM.ORG Member
#3
Possibly, but probably not to the point where you need it to be. I'd just try to tweak the tcpwindow size, not as much to gain with the others.

If you're backing up to a container pool, I'd consider client-side dedup and compression, that would cut down the amount of data to send to the server.

Backing up to VTL so no dedup at the moment. The cluster is Windows Server 2012 which has tcp window scaling set to auto by default. I've set the tcpwindowsize to 256 on the client which, I believe, will override the OS with regards to the backup.
 

marclant

ADSM.ORG Moderator
#4
Backing up to VTL so no dedup at the moment. The cluster is Windows Server 2012 which has tcp window scaling set to auto by default. I've set the tcpwindowsize to 256 on the client which, I believe, will override the OS with regards to the backup.
Regarding TCPWindowsize, a value of 0 will use the OS value, a value that is not 0 will use what you specify.
 

RecoveryOne

ADSM.ORG Senior Member
#6
Are you trying to do client side compression? A few of our large flat files (2TB or so) turned off compression for them. That way its just sending the data down the pipe instead of chewing cpu cycles and time trying to compress the data before sending it. That took my backup window from 14 hours down to 9 hours. Not a lot but certainly helps.

How much bandwidth do you have coming into the TSM server? How much is available on the client? How much of either is being used during the backup window of this client? No matter how you slice it, if you have a 2TB file and only a 1GB Ethernet connection and assuming you hit the absolute max transfer speed of 1GB you are looking around about 5 hours of transfer time. Now if you have 10GB and your disk's (source and TSM) can handle the iops, you should be able to transfer 2TB in about 25 minutes give or take. (All napkin math above by the way)

Also, you may want to bump your maxnummp to 12 if the resourceutilization is at 10. I always found setting maxnummp two values higher than resourceutilization helps prevent 'This node has exceeded its maximum number of mount points'. That said, I've noticed the biggest performance improvement with resourceutilization is you have many different filesystems. If its just two or three, you might not go above 6ish.
 

Advertise at ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

UpCloud high performance VPS at $5/month

Get started with $25 in credits on Cloud Servers. You must use link below to receive the credit. Use the promo to get upto 5 month of FREE Linux VPS.

The Spectrum Protect TLA (Three-Letter Acronym): ISP or something else?

  • Every product needs a TLA, Let's call it ISP (IBM Spectrum Protect).

    Votes: 8 21.6%
  • Keep using TSM for Spectrum Protect.

    Votes: 19 51.4%
  • Let's be formal and just say Spectrum Protect

    Votes: 6 16.2%
  • Other (please comement)

    Votes: 4 10.8%

Forum statistics

Threads
30,969
Messages
131,783
Members
21,226
Latest member
rocaflqu
Top