1. Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING) Click the link to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This message will disappear after you have made at least 12 posts. Thank you for your cooperation.

Exchange 2010 DAG - TSM backups too slow

Discussion in 'Backup / Archive Discussion' started by seckin1983, Apr 11, 2012.

  1. seckin1983

    seckin1983 New Member

    Joined:
    Apr 11, 2012
    Messages:
    5
    Likes Received:
    0
    Hi everyone,

    we have 2 exchange 2010 SP1 with Rollup Update 5 configured for DAG running on Windows 2008 R2 SP1.

    When we try to backup Exchange any node (passive or active) with TSM Data Protection for MS Exchange we see backup throughput is approximately 10MB/second.
    Our TSM Server version 5.5 level 4.3 and DPM is 6.1.2

    I also wanted to see backup performance on different system which is configured for our test platform. on that system exchange server is configured on single server like production exchange version which is exc 2010 SP1 with Rollup Update 5 running on Windows 2008 R2 SP1. Also i updated DPM level to 6.1.3.1 on this test server. But backup performance is not changed which is same like production 10 or 12 MB/second .

    Test system works on Vmware ESXi and production systems works on different Vmware ESX hosts. When i try copy file from/to exchange servers on network, I see 50MB/second throughput which is we expected. So we say there is no network problem with Exchange nodes. Also we see better backup performances from other servers on TSM. so we can say also there is no performance issue with TSM server.

    I changed TCPWindowsize to 63 and TCPBuffSize to 32 in dsm.opt file. However nothing was changed.

    Because of this backup performance issues our backup times takes 14 hours. What should i changed? Where is this performance issue caused by? Is this problem caused by windows 2008 R2 or Exchange 2010 or DPM?

    Thanks for your suggestions.
    Regards.
     
  2.  
  3. moon-buddy

    moon-buddy Moderator

    Joined:
    Aug 24, 2005
    Messages:
    6,263
    Likes Received:
    282
    Occupation:
    Electronics Engineer, Security Professional
    Location:
    Somewhere in the US
    I am not clear on this: Are the PROD servers on physical boxes or VM Ware guests?
     
  4. seckin1983

    seckin1983 New Member

    Joined:
    Apr 11, 2012
    Messages:
    5
    Likes Received:
    0
    All of them vmware guests;
    - Production servers (2 nodes in DAG) on Vmware ESX platform
    - Test server (single server, no dag configured) on Vmware ESX-i platform
     
  5. moon-buddy

    moon-buddy Moderator

    Joined:
    Aug 24, 2005
    Messages:
    6,263
    Likes Received:
    282
    Occupation:
    Electronics Engineer, Security Professional
    Location:
    Somewhere in the US
    I have not tested Exchange on VM Ware since I don't 'believe' on virtual environments for heavy I/O Apps or Systems users like Exchange.

    However, see if you can have the Exchange servers the only user of the VM Ware platform for the duration of the test - no other systems running. See if there is an improvement.

    There are parameter settings called BUFFERS and BUFFERSIZE on dsm.opt. BUFFERS is from 2 to 8, and BUFFERSIZE is from 64 to 8192 kilobytes with 1024 as default. Experiment on the values.

    Also, read this or reference it: http://publib.boulder.ibm.com/infoc...itsm.mail.exc.doc/b_dp_mail_exc_guide_win.pdf
     
    Last edited: Apr 11, 2012
  6. seckin1983

    seckin1983 New Member

    Joined:
    Apr 11, 2012
    Messages:
    5
    Likes Received:
    0

    I dont think its an I/O problem, because the result never changes during the work day or midnight.
    Also backup throughput exactly same on test system which is running on located on local disks in vmware esxi host and production is running on vmware esx located on SAN. however i will examine this again.

    also i will change BUFFERS and BUFFERSIZE settings and observe backup performance.

    Thanks for your suggestions. i will inform you with results.
     
  7. gimlet

    gimlet New Member

    Joined:
    Apr 25, 2008
    Messages:
    154
    Likes Received:
    4
    Occupation:
    Infrastructure Monkey
    Location:
    St. Louis, Missouri
    It's hard to say without knowing more details of your environment. You mention the software and platforms, but not network connectivity, storage configuration, other VMs, other workloads/sessions on the TSM server. So it's going to be difficult to debug via forum.

    That said...

    Are you talking data transfer rate or aggregate transfer rate? In our environment, the data transfer rate from Exchange 2010 to TSM 5.5 is ~100MB/s, with an aggregate rate of ~22MB/s.

    How fast does the BA client on the same node(s) backup? Do all other nodes backup much more quickly? Same performance whether full or incremental backups?
     
  8. emerman

    emerman New Member

    Joined:
    Apr 10, 2009
    Messages:
    31
    Likes Received:
    0
    I am also investigating improving backup performance with Exchange 2010. I'm currently at 45MB's per second but could use some improvement. (My issue may be disk though). Anyway, saw this and wanted to chime in that BUFFERS and BUFFERSIZE only apply to legacy backup which isn't supported on Exchange 2010 according to the TDP manual.
    I would start by looking at storage speed. How fast is a file copy (CIFS) from your Exchange mailbox server to your TSM server? It should be around 40-50MB's per second. If not, your storage at either end may be the issue.
     
  9. olis

    olis New Member

    Joined:
    Nov 30, 2010
    Messages:
    16
    Likes Received:
    0
    Location:
    Germany
    Hi,

    i saw some kind of network trouble with 2010DAG an Citrix Xen Server. I did some network speed testing with jperf and saw heavy errors while using the same TCPWindowsize and Buffer like tsm would do. Maximal speed was < 800 MB/h...

    Finally we activated compression on the client and traffic wend up to >70 MB/s. Maybe you could try out these steps.

    Greetings
     
  10. seckin1983

    seckin1983 New Member

    Joined:
    Apr 11, 2012
    Messages:
    5
    Likes Received:
    0
    Hi,

    our exchange server dag nodes, tsm server are connected to network 1 Gbps. tsm server is connected to library via fiber.

    we are still using exchange 2003 in our organization for some mailboxes and during backup jobs of exch 2003 with tsm we see 2-3 GB per minute which is acceptable i think. also we see better backup performance values from other clients.

    On the other hand after emerman suggestion i tested the file copy throughput between exchange server to tsm server. i saw average 10 MB per second and i thought bottleneck is caused by network issue. i dont understand that file copy between tsm server and exchange 2003 is better than exchange 2010 with old driver installed tsm server.
    immediately i upgraded the network driver of tsm server. now during file copy between exchange 2010 server to tsm server i see 20-30 MBs per second, however backup performance has not changed it is still 8-12 MBs per second.

    Olis where can i activate the compression, could yo clarify?

    Thanks
     
  11. seckin1983

    seckin1983 New Member

    Joined:
    Apr 11, 2012
    Messages:
    5
    Likes Received:
    0
    additionally our tsm server is windows 2003 standard sp2 32 bit located on physical server with Broadcom NeXtreme Gigabit network adapter.
     
  12. olis

    olis New Member

    Joined:
    Nov 30, 2010
    Messages:
    16
    Likes Received:
    0
    Location:
    Germany
    Hey seckin1983,

    you could add the compression statemend to your tdp dsm.opt:

    compression yes

    How did you test the throughput? Copy some files or some real test software? Give jperf or iperf a try.
     
    Last edited: Apr 17, 2012
  13. unpaired

    unpaired New Member

    Joined:
    Jun 17, 2010
    Messages:
    84
    Likes Received:
    1
    Hi All,

    Anybody find a solitions? I have a same problem but i did'n find reason yet.

    Could you please share your experience?
    Thank you.
     
  14. unpaired

    unpaired New Member

    Joined:
    Jun 17, 2010
    Messages:
    84
    Likes Received:
    1
    our exchange team did backup test with windows backup same database and backup time and speed look pretty good for 1GBit/sec network interfaces.
    It doesn’t look problem with windows backup.

    it seems problem TDP problem but what?
    Thank you for your advice.
     
  15. emerman

    emerman New Member

    Joined:
    Apr 10, 2009
    Messages:
    31
    Likes Received:
    0
    TSM backs up each database one at a time. The trick for me was to separate the backups into different jobs and then run them side by side. I did this by writing a custom powershell script.
     
  16. unpaired

    unpaired New Member

    Joined:
    Jun 17, 2010
    Messages:
    84
    Likes Received:
    1
    Hello emerman,
    Thank you so much your fastly answer.
    I have one db which is 10 GB. I run test only one db. result so slowly too. ?
     

Share This Page