[Veritas-bu] extremely large number of files backup problem (time)

2001-12-20 13:31:34
Subject: [Veritas-bu] extremely large number of files backup problem (time)
From: jim AT DOT edu (Jim Ennis)
Date: Thu, 20 Dec 2001 13:31:34 -0500 (EST)

We are running Netbackup 3.2 on an E450 as the master server.  We use
DLT7000 drives on a Sun L1800 tape library for backup media and the backup
client in question is connected via private gigabit ethernet.

We are using webct for a large number of classes and the package produces
millions of little (<2K for the most part) files in one filesystems
(Solaris 7, UFS, Veritas Volume Manage, Sun A5200 for storage).

A full backup runs close to 30 hours and the main
bottleneck seems to be the sheer volume of files (total disk space is
about 30GB, number of files is 6.2 million).

I am looking at three options to handle this better (cut down the backup
window and have less user impact due to I/O performance):

1) Change the backup schedule to do differential backups on a daily basis,
culmative backup on weekly basis and the full backup only once a month
(currently the full backup runs weekly).  This would add recovery time but
decrease the backup windows to hopefully run between 1am and 7am.

2) Look at using Veritas Flashback or Sun's Instant Image to capture a
snapshot of the system and then backup the associated backup image data
which would not impact the end-users significantly and cut the 'backup'
time to minutes.  I am not sure how file level restores work with these

3) Split the current 2-way mirror to a 3-way mirror and take one of the
mirrors offline so that I can back it up (use the pre and post script
abilities of postscript to control this).  This should work, but needs
testing on my end.

Any comments or recommendations about which way to go.  We are trying to
get the Sun Instant Image software onsite for testing and I am trying to
check with Veritas about Flashback.

Thanks in advance.

Jim Ennis                        | jim AT DOT edu
Systems Administrator            | (407) 823-1701  |  Fax: (407) 823-5476
University of Central Florida    | Murphy's paradox:
                                 | Doing it the hard way is always easier.

<Prev in Thread] Current Thread [Next in Thread>