Another product to consider is Legato SnapImage. It takes a snapshot of the
filesystem then does a block-level backup. It's effectively the
snapshot/block-level functionality from Celestra, without the serverless
functionality.
-----Original Message-----
From: Terry Lemons [mailto:lemons_terry AT EMC DOT COM]
Sent: Tuesday, September 10, 2002 2:40 PM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] Large filesystem backup revisited
I'll echo Jason's suggestion to try a block-level backup. Legato Celestra
has been available for several years. Though it doesn't appear to have been
widely implemented, it does support Solaris and integrates with NetWorker.
Celestra requires you to use an additional Solaris system, which will be the
'data mover'. If you have multiple Solaris systems, they can share this
single data mover.
You can find a very readable, but quite technical, description of Celestra
at http://www.legato.com/resources/whitepapers/W064.pdf. Celestra uses both
SCSI Extended Copy (XCOPY) and NDMP and, unlike other XCOPY implementations,
it does create file level metadata, and does support full and non-full
backups.
Software cost (used to be VERY expensive) and dedicated data mover system
requirement are two reasons that, IMHO, Celestra hasn't prospered; it isn't
for every environment. But you may have an ideal situation for its use.
See if Legato can help you with a test drive (the kit is on the latest
Legato CD set, but I didn't find a temporary license; Celestra used to be
available via Legato Professional Services only).
Good luck!
tl
Terry Lemons
CLARiiON Application Solutions Engineering
EMC²
where information lives
4400 Computer Drive, MS D239
Westboro MA 01580
Phone: 508 898 7312
Email: Lemons_Terry AT emc DOT com
-----Original Message-----
From: Jason Stege [mailto:jrstege AT yahoo DOT com]
Sent: Monday, September 09, 2002 1:51 PM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] Large filesystem backup revisited
A journalling file system is almost always a better idea and will
help with this, but probably won't solve all your problems. VXFS is
a good choice, but I like the LSCI products like QFS as well. Sun
has just recently aquired LSCI earlier this year.
The real problem I see is that if you do suceed in a backup, you will
have a really hard time restoring this data if you loose the whole
filesystem. The inode creation is relatively slow and will take
forever if at all for this number of small files. A block level
backup is a much better solution and would bypass the small file
problem caused by file system overhead.
A full block level backup with the incrementals handled with some
variant of snapshot technology would probably be the most efficient,
but you're then relying on the RAID technology for the incrementals.
I believe there are block level incremental backups out there, but
I'm not sure if Legato is one of the vendors.
If this sounds like something your looking for, let me know and I'll
get the product info for you.
--Jason Stege
Independant Consultant
--- Stuart Lamble <Stuart.Lamble AT ITS.MONASH.EDU DOT AU> wrote:
> Since there seems to be a certain amount of interest on the list,
> here's a list of things we have (and have not :) tried to date.
> First,
> a quick rundown on the filesystem:
>
> * 522 GB hardware RAID partition.
> * Approximately 50 GB in 2,995,971 files (average of 16 kB/file)
> * Fibre channel 1 Gbps interface.
>
> General consensus was that we are being hit by the "many small
> files"
> problem, which certainly fits in with our situation.
>
> Things we've tried to date:
>
> * Increased the DNLC and inode cache (see
>
> http://www.princeton.edu/~unix/Solaris/troubleshoot/kerntune.html
> --
> very useful web page)
>
> Slight improvement from this, but Networker is still timing
> out.
>
> * Running uasm over the whole filesystem, piped to /dev/null to
> eliminate the network as a bottleneck -- I aborted this after
> eight hours (considering that we're "only" using 60 GB of space
> at
> peak, and less than 50 GB after the processes using the disk
> clean
> up after themselves, eight hours _should_ be adequate.)
>
> * Removing the temporary working area from the backup -- again,
> maybe
> a slight improvement, but it doesn't resolve the underlying
> issue.
>
> Executive summary: we've managed to get slight improvements, but
> Networker
> still likes to time out whilst backing up this filesystem. Sigh.
> Too many
> small files.
>
> At the moment, we're preparing to evaluate Sun's QFS as a possible
> alternative to UFS for this particular situation. Does anybody have
> any
> knowledge of how Networker interacts with QFS? Sun's representative
> is
> saying that our situation seems ideal for QFS, but then, I'd be
> inclined
> to take his words with a grain of salt :-) The biggest problem from
> my
> point of view is that Legato's compatibility matrix only lists UFS
> and
> VxFS; VxFS is an option, but it would be nice (if only from a
> bargaining
> point of view ;-) to have QFS as an option as well.
>
> As an aside, I'll be going on leave at the end of this week for a
> six
> week holiday, so the testing will be somewhat abbreviated (we have
> a
> thirty day evaluation license) ... I'll be picking up the ball once
> I get
> back, in late October.
>
> Ta muchly,
>
> Stuart.
>
> --
> Note: To sign off this list, send a "signoff" command via email
> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> http://listmail.temple.edu/archives/networker.html where you can
> also view and post messages to the list.
> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
__________________________________________________
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes
http://finance.yahoo.com
--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
|
