Just a follow-up question...
In my environment, I'm going to be given access to some SAN storage
for use as staging areas. I have a tape library with 4 drives and I'm
backing up 13TB per week. It is tough getting this done within the
backup windows with the 4 tape drives so I want the DSSU space to ease
the load a bit. It seems to me that having the DSSU's should
essentially extend my backup window since all data written to DSSUs
can be de-staged outside of the backup window.
So my question is how best to configure the DSSUs with the goal of
optimized de-staging. I will have 6TB to configure as desired on the
backup server. If I understand correctly, the more concurrent streams
allowed to the DSSUs, the slower the de-staging because of interleaved
backup streams. So, for the 6TB of SAN storage it would seem to be
more efficient to create several smaller volumes, each with its own
filesystem and mountpoint rather than one big 6TB volume with 1
filesystem. Am I correct to assume that the interleaving blocks from
multiple streams is occurring at the filesystem level? Since the SAN
provisioned LUN is virtual and made up of several disks in a stripe
configuration I expect disk reads to be reasonably quick.
Thanks,
Vic
On Mon, Apr 26, 2010 at 4:34 PM, Martin, Jonathan <JMARTI05 AT intersil DOT
com> wrote:
>
>
> I started with a calculation like that, but I also incorporate destaging
> speed. I've never used anything higher than 16 concurrent writes, and that
> is for incremental backups. With incremental backups, I don’t care as much
> because even if the destaging speed is poor, the amount of data is so small
> that it will still be written to tape quickly.
>
>
>
> However for full backups, I’ve found that the more streams you have, the
> slower your destaging will be. I created the graphic below to help explain
> it to management. The issue isn’t fragmentation as much as it is sequential
> blocks of data. If you were to write one sequential stream to a raid group
> then reading it back would be a sequential operation. And in my case, SATA
> disks read sequential data very quickly. But as soon as you multistream, the
> data “fragments” or becomes non-sequential (really, the file sizes are so
> large every file is fragmented).
>
>
>
> Image removed due to size limitation.
>
>
>
> So to find your “sweet” spot you will have to test. I started by building
> my environment and then testing with a single stream, from client to disk,
> then disk to tape. This was my best case scenario, and I was able to drive
> LTO3 with it.
>
>
>
> My configuration: Dell MD1000 w/ 12 500GB DATA disks in a Raid 5. PERC 5/e
> card with Adaptive Read Ahead and Write Back enabled. GPT Partition table w/
> 64KB block sizes.
>
>
>
> The 64KB block size and adaptive read ahead settings were key to my
> performance. I was going to test the same hardware with a linux ext2 file
> system and 4KB block sizes for comparison, but I never got the chance.
>
>
>
> Once you have a baseline you can run more streams. You also need to consider
> job length. For example, I backup an NFS server with 16 streams to a DSSU
> that allows all 16 streams because 12 of those streams take less than an
> hour each. I’ve also tested with 1TB SATA Disks and 300GB 15K SAS Disks. I
> still rely on the 500GB SATA as my work horse because I can drive LTO3 if I
> need to. Most of my DSSUs are 8 streams and push 70MB/sec to tape.
>
>
>
> -Jonathan
>
>
>
>
>
>
>
> -----Original Message-----
> From: Victor Engle [mailto:victor.engle AT gmail DOT com]
> Sent: Monday, April 26, 2010 3:39 PM
> To: Martin, Jonathan
> Cc: veritas-bu AT mailman.eng.auburn DOT edu
> Subject: Re: [Veritas-bu] Architectural question (staging)
>
>
>
> Thanks Martin. Good info. What criteria do you use to determine the
>
> number of concurrent jobs for a DSSU? Is it reasonable to determine
>
> the concurrent DSSU sessions based on the speed of the clients? For
>
> example, If I have a disk to which I can write 100MB/s and clients
>
> that can write 5MB/s then I would use 20 concurrent sessions?
>
>
>
> Thanks all for your responses!
>
>
>
> Vic
>
>
>
>
>
> On Mon, Apr 26, 2010 at 10:14 AM, Martin, Jonathan
>
> <JMARTI05 AT intersil DOT com> wrote:
>
>> We use Disk Stage Storage Units (DSSU) almost exclusively for our backups.
>
>> As someone has already mentioned, you can stream many slow clients to a
>> DSSU
>
>> without impacting your speed significantly. To do this with tape you would
>
>> have to use multiplexing, which is a real performance killer come restore
>
>> time. The DSSUs essentially allow you to “multiplex” to disk, then single
>
>> stream to tape. Regarding Ed’s speed issue below, I’ve got data here that
>
>> directly correlates number of concurrent streams written to a DSSU with
>> the
>
>> performance to tape. You’ll need to balance this when setting this on the
>
>> DSSU, but we get away with 8-12 streams on 14x500GB SATA Disks in a Raid-5
>
>> and still drive LTO3.
>
>>
>
>>
>
>>
>
>> Here we were also able to purchase a much smaller tape library. I
>> replaced
>
>> a library with 8 x SDLT220 drives with a library with 2 x LTO3 drives (and
>
>> disk storage.) The disk storage is far more reliable than tapes and
>
>> libraries. As Ed noted below you also benefit from having your backups on
>
>> disk which makes restores very snappy.
>
>>
>
>>
>
>>
>
>> -Jonathan
>
>>
>
>>
>
>>
>
>> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
>
>> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Ed
>> Wilts
>
>> Sent: Sunday, April 25, 2010 3:45 PM
>
>> To: Victor Engle
>
>> Cc: veritas-bu AT mailman.eng.auburn DOT edu
>
>> Subject: Re: [Veritas-bu] Architectural question (staging)
>
>>
>
>>
>
>>
>
>> On Sun, Apr 25, 2010 at 2:14 PM, Victor Engle <victor.engle AT gmail DOT com>
>
>> wrote:
>
>>
>
>> Just wanted to get some opinions about whether disk staging units are
>
>> worthwhile. My backup server has two BasicDisk staging units with the
>
>> storage units configured such that the data goes to disk and is then
>
>> moved to tape. I have a tape library with four LTO-3 drives connected
>
>> via FC. So what I'm wondering is, since the LTO drives are reasonably
>
>> fast, and since I'm writing the data ultimately to tape anyway, would
>
>> it be better to just write directly to tape. The disk is just old
>
>> fashioned spinning disk with no de-duplication so there are
>
>> operational costs for the disks. All tape and disk storage units are
>
>> local to the backup server. I'm thinking it would be better to add LTO
>
>> drives and eliminate the disk for now and maybe later add a
>
>> de-duplicating disk unit.
>
>>
>
>> We worked with disk staging units for at least a year before we mostly
>> gave
>
>> up. The biggest challenge we ran into was that destaging was too slow.
>> Even
>
>> though we proved to Symantec that we could read from those disk drives at
>
>> over 100MB/sec, we could never destage even half that fast. We had an
>> open
>
>> case with Symantec for a VERY long time before we agreed that it wasn't
>
>> going to get fixed.
>
>>
>
>> Under what circumstances does it make sense to stage data on disk. I
>
>> would appreciate hearing what your thoughts and experiences are with
>
>> regard to disk staging.
>
>>
>
>> There are times when DSSUs make sense. 1. If you don't have a tape drive
>
>> free but want to do a backup anyway - we still use DSSUs for things like
>
>> small backups of Oracle archive logs. 2. If you need to throttle your
>
>> backups, especially across things like a bunch of virtual servers on the
>
>> same physical server. NBU only allows you to set the maximum jobs per
>
>> client name, not per client. DSSUs make an acceptable choke point for
>
>> clusters. 3. If you have small backups, but don't have a lot of them at
>
>> once, multiplexing may not buy you enough performance boosts. Use DSSUs
>> to
>
>> write those little jobs to disk and then destage them at once.
>
>>
>
>> If you currently multiplex, realize that your restores are going to be
>
>> slower than if you don't multiplex. All tapes created from a DSSU destage
>
>> are non-multiplexed so your restores can go faster.
>
>>
>
>> DSSUs also give you a staging area for restores. If your tapes go
>> offsite,
>
>> you may still be able to do a restore from the staging unit the next day
>> (or
>
>> longer) depending on how big your stagig units are. NBU is smart enough
>> to
>
>> realize that if the same data is on both disk and tape and you kick off a
>
>> restore, the restore will automatically come from disk.
>
>>
>
>> In general, I'd say that there is a place for DSSUs but it's not the great
>
>> benefit we thought it was going to be.
>
>>
>
>> .../Ed
>
>>
>
>> _______________________________________________
>
>> Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
>
>> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
>>
>
>>
>
> _______________________________________________
> Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
>
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
|
