Veritas-bu

[Veritas-bu] Improving Duplication Performance from DataDomain Restorers

2009-04-20 13:50:53
Subject: [Veritas-bu] Improving Duplication Performance from DataDomain Restorers
From: "JAJA (Jamie Jamison)" <jamisonj AT zgi DOT com>
To: "veritas-bu AT mailman.eng.auburn DOT edu" <veritas-bu AT mailman.eng.auburn DOT edu>
Date: Mon, 20 Apr 2009 10:47:29 -0700
Here's my set up. I have a Sparc T2000 with 16 1.2Ghz cores and 16Gb of RAM running Solaris 10. I have three DataDomain restorers, two DD460s and a DD565. These are connected to a dedicated backup gigabit backup network and jumbo frames are implemented on the DataDomains and the T2000 interfaces that talk to the backup subnet. I write all of my backups to disk storage units mounted from the DataDomains or to a VTL on the DD565. I keep these for four weeks. I then duplicate the images from the DataDomains to LTO4 tape with retention levels on the duplicates ranging from two months to infinity (for certain archival data) and offsite them with Iron Mountain. I've implemented the system tuning recommendations in the DataDomain white paper on best practices with DataDomain restorers and NetBackups. I've tuned the NetBackup parameters as follows:
 
NET_BUFFER_SZ=262144
NUMBER_DATA_BUFFERS=256
NUMBER_DATA_BUFFERS_DISK=256
SIZE_DATA_BUFFERS=262144
SIZE_DATA_BUFFERS_DISK=262144
 
To take advantage of the increased value of NET_BUFFER_SZ I've increased tcp_recv_hiwat and tcp_xmit_hiwat to 262144 bytes from the default Solaris 10 value of 49152 bytes.
 
The result of this is that when things are going well, as they are this morning, I can get between 120 and 150 megabytes per second throughput to two LTO4 drives, when things aren't going well I can see drive throughput drop to kilobytes per second (as measured by iostat). The latter condition seems to be caused by duplicating smaller backup images, bpdm has to be called, it opens the backup image, reads it, sends the data to bptm and bptm writes it to tape. When I'm duplicating the smaller images I can watch the logs and see each bpdm process being spawned and then each bptm being spawned and I can watch my poor LTO4 tape drives idling at 4 kilobytes per second, which kills me and makes me wonder if PETLTO4TD (People for the Ethical Treatment of LTO4 Tape Drives) are going to come and take my tape drives away.
 
Since all of my primary backups are written to a disk based storage unit multiplexing is set to 1. Is there a way to multiplex duplication from disk storage units? Failing that is there some way of writing to tape asynchronously so that the the buffers on the tape drives could be filled before the write actually happen. Are there tuning steps that I haven't taken that could improve the duplication performance? Any help will be, as it has been in the past, greatly appreciated.
 
Thank You,
 
Jamie Jamison
Network Systems Administrator
ZymoGenetics
 
 
 
 
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
<Prev in Thread] Current Thread [Next in Thread>
  • [Veritas-bu] Improving Duplication Performance from DataDomain Restorers, JAJA (Jamie Jamison) <=