Amanda-Users

Re: hardware vs software compression (was Re: amflush/amcheck not in sync?)

2003-04-24 14:56:00
Subject: Re: hardware vs software compression (was Re: amflush/amcheck not in sync?)
From: Russell Adams <RLAdams AT kelsey-seybold DOT com>
To: amanda-users AT amanda DOT org
Date: Thu, 24 Apr 2003 13:47:59 -0500
Perhaps I should clarify.
      
The DLT's are Compaq TZ89 (DLT7000) units in a TL892 10 tape
library. I have a dedicated UW-SCSI bus for each pair of TZ89's. There
are three pair.

The disks being backed up are on a dedicated UW-SCSI bus to a large HD
array (Compaq HSZ80 w/ dual ports & dual controllers). The disks are
_not_ in use in production, they are a broken mirror copy of my
production disks (disk to disk hot-backup). During the disk to disk
copy, the disks peak at about 25-30MB/s throughput.

The system these are tied into is a Quad EV67/833mhz Alphaserver ES40,
w/ 6 GB of ram.

The host runs OpenVMS, and the only reason this is relevant is because
I'm constantly wishing I could run AMANDA on this platform. VMS's
BACKUP utility doesn't support compression, or anything else for that
matter. Our backups are done by custom scripts. If I ran Tru64 or
Linux on these, I'd be using AMANDA.

Because of the benchmarked performance with these tape drives, I've
often wanted to use client side compression due to the amount of
horsepower available. I cannot due to software constraints.

Quite frustrating when the disk to disk copy takes 4 hours, then it
takes 14 hours to back it all up to tape.

In uncompressed mode I get 4-5MB/s, in compressed its 800KB/s. Sad
really.

Russell

On Thu, Apr 24, 2003 at 11:45:02AM -0400, Mitch Collinsworth wrote:
> 
> On Thu, 24 Apr 2003, Jon LaBadie wrote:
> 
> > On Thu, Apr 24, 2003 at 09:26:12AM -0500, Russell Adams wrote:
> > > > Of course hardware compression has one big advantage: speed.  If you
> > > > have to fit lots of data in a small backup window at night, and your
> > > > cpu's are not fast enough, or interfere with the nightly cpu-intensive
> > > > jobs, then hardware compression is the way to go.
> > >
> > >
> > > I have to take issue with this. My experience with hardware
> > > compression on DDS and DLT drives is that when enabled it makes things
> > > incredibly slow.
> > >
> > > My DLT IV (er, 7000?) drives can do 4.9 MB/s uncompressed, and only
> > > 800KB/s in compressed mode.
> > >
> >
> > This is of course counter to advertising claims :))
> >
> > Are you sure there was not some other factor involved.  A possibility
> > that comes to mind is that the no-hw-compression came from data already
> > in a single file or local to the server while the with-hw-compression
> > test data had to be "found" and that procedure caused the slow down?
> 
> Yes, I agree.  This sounds suspicious.  The slowdown described sounds
> like exactly what happens with DLT4000 and DLT7000 when you can't feed
> them data fast enough to keep them streaming.  Supposedly this was fixed
> somewhat in DLT8000 but I never bought one to verify that claim.  I've
> never heard of anyone claiming this also happens with compression.
> 
> John Jackson was (is?) running several DLT drives and said he eventually
> made the switch from software to hardware compression just for the
> reasons already stated here.  The bottom line is you have to get the
> data onto the tapes and he was able to do it much faster with h/w
> compression.
> 
> -Mitch

<Prev in Thread] Current Thread [Next in Thread>