Networker

Re: [Networker] Very large filesystems backup

2008-11-10 12:34:39
Subject: Re: [Networker] Very large filesystems backup
From: Bruce Breidall <Bruce.Breidall AT CONCUR DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Mon, 10 Nov 2008 09:32:34 -0800
Some long term suggestions.

Get this data on a storage platform that gives you more ability to
manage the backups. Images need to be archived, not backed up. A CAS
solution would be the most appropriate direction, but that requires
additional effort at the application level to be able to move the data
from primary to secondary storage (unless you have a lot of money to
insert a third party data movement engine that may or may not work with
the number of files you have). NAS is your next choice, giving you the
same approximate price point on the storage as CAS.

NW fails miserably in being able to easily break up your savesets to
create streams. The scripting tricks provided to you from other
technicians are a workaround for an extremely limited ability to break
up your backups, but they do nothing to help you on the restore side,
and in fact breaking up the savesets can hurt you even more on the
restore side in a DR scenario.

It would also be a good idea to discuss with the application folks a
better way to organize the data, so that you have more options available
to you on managing the backups. Application folks rarely consider the
file system layout in terms of backups. Don't have single directories
with millions of files. You need to have multiple directories with maybe
a couple of hundred thousand files to eliminate those scenarios. This
will speed up everything, guaranteed.

For us, we have the ability to redeploy the application to a new tree on
a NAS file system. This allows us to keep the trees at a manageable size
for backups, and not have to mess with all these scripting ways to get
around what a backup product should be able to do, easily. I create file
systems on the NAS filer at around 1 TB, with 3 trees.

We have a total of 4 TBs on NAS, with many trees. The next logical
improvement there is to implement NDMP to shrink the backup window.

These are all future suggestions, but the key is to start preparing for
the inevitable by getting your company to start thinking about the
future, and get out of the reactive mode you are in today.
 
-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Browning, David
Sent: Monday, November 10, 2008 8:54 AM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: [Networker] Very large filesystems backup

Just curious as to what everyone else does out their for their VERY
large filesystem backups.  

Our document imaging file server has gradually been increasing in size
over the past year, and is now up to 21+ million files.   Data size is
under 1TB, so size isn't an issue, it's simply the 21 million files - it
takes 48 hours to backup. 

We have a couple of other file servers that are large (3 - 5 million),
but nothing this size. 

Are people using some kind of snapshot system, or something else? 

TIA

David M. Browning Jr.
IT Project Coordinator Enterprise Backups and Help Desk

 

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER