Networker

Re: [Networker] recommendation to backup 1.5 million small files?

2008-01-31 07:58:14
Subject: Re: [Networker] recommendation to backup 1.5 million small files?
From: Vincent Lin <vlinx AT YAHOO DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Thu, 31 Jan 2008 04:55:57 -0800
NetWorker SnapImage ModuleEMC NetWorker SnapImage
Module creates a block-level snapshot of an organization's entire file
system, then periodically captures incremental data changes, ...

http://software.emc.com/products/software_az/networker_snapimage_module.htm

----- Original Message ----
From: Craig Worthington <craig.worthington AT NRM.QLD.GOV DOT AU>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Sent: Thursday, January 31, 2008 12:21:53 PM
Subject: Re: [Networker] recommendation to backup 1.5 million small files?


yes, 
that's 
1 
one 
directory. 
It 
takes 
days 
to 
backup, 
even 
3 
million
files 
takes 
30+ 
hours 
to 
backup. 
I 
know 
it's 
the 
worst 
way 
to 
organise
data 
but 
it 
was 
given 
to 
us 
by 
a 
external 
programmer.

so 
is 
there 
a 
way 
using 
legato 
to 
bypass 
the 
filesystem 
layer 
?

CraigW

On 
Wed, 
2008-01-30 
at 
03:35, 
Tim 
Mooney 
wrote:
> 
In 
regard 
to: 
Re: 
[Networker] 
recommendation 
to 
backup 
1.5 
million 
small...:
> 
> 
> 
I'm 
having 
the 
same 
problem 
with 
2 
directories 
on 
UNIX 
servers 
(SGI 
and
> 
> 
Solaris), 
1 
is 
3million 
and 
the 
other 
15million 
files.
> 
> 
Wait 
-- 
you're 
not 
just 
talking 
about 
15 
million 
files 
on 
one 
filesystem,
> 
you're 
talking 
about 
15 
million 
files 
in 
one 
directory?  
What's 
the
> 
filesystem 
type?  
How 
long 
does
> 
>  
    
time 
ls 
> 
/dev/null
> 
> 
in 
that 
directory 
take?
> 
> 
Very 
few 
filesystems 
handle 
large 
numbers 
of 
files 
in 
one 
directory 
very
> 
well.  
If 
standard 
commands 
take 
a 
long 
time 
to 
process 
the 
directory,
> 
NetWorker 
will 
do 
no 
better.  
You 
will 
have 
to 
bypass 
the 
filesystem 
layer
> 
to 
get 
any 
kind 
of 
decent 
performance.
> 
> 
Tim
> 
-- 
> 
Tim 
Mooney  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Tim.Mooney AT ndsu DOT edu
> 
Information 
Technology 
Services  
  
  
  
  
  
  
  
  
 
(701) 
231-1076 
(Voice)
> 
Room 
242-J6, 
IACC 
Building  
  
  
  
  
  
  
  
  
  
  
  
(701) 
231-8541 
(Fax)
> 
North 
Dakota 
State 
University, 
Fargo, 
ND 
58105-5164
> 
> 
To 
sign 
off 
this 
list, 
send 
email 
to 
listserv AT listserv.temple DOT edu 
and 
type 
"signoff 
networker" 
in 
the 
body 
of 
the 
email. 
Please 
write 
to 
networker-request AT listserv.temple DOT edu 
if 
you 
have 
any 
problems 
with 
this 
list. 
You 
can 
access 
the 
archives 
at 
http://listserv.temple.edu/archives/networker.html 
or
> 
via 
RSS 
at 
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER


************************************************************************
The 
information 
in 
this 
email 
together 
with 
any 
attachments 
is
intended 
only 
for 
the 
person 
or 
entity 
to 
which 
it 
is 
addressed
and 
may 
contain 
confidential 
and/or 
privileged 
material.
Any 
form 
of 
review, 
disclosure, 
modification, 
distribution
and/or 
publication 
of 
this 
email 
message 
is 
prohibited, 
unless
as 
a 
necessary 
part 
of 
Departmental 
business.
If 
you 
have 
received 
this 
message 
in 
error, 
you 
are 
asked 
to
inform 
the 
sender 
as 
quickly 
as 
possible 
and 
delete 
this 
message
and 
any 
copies 
of 
this 
message 
from 
your 
computer 
and/or 
your
computer 
system 
network.
************************************************************************

To 
sign 
off 
this 
list, 
send 
email 
to 
listserv AT listserv.temple DOT edu 
and 
type 
"signoff 
networker" 
in 
the 
body 
of 
the 
email. 
Please 
write 
to 
networker-request AT listserv.temple DOT edu 
if 
you 
have 
any 
problems 
with 
this 
list. 
You 
can 
access 
the 
archives 
at 
http://listserv.temple.edu/archives/networker.html 
or
via 
RSS 
at 
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER






      
____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER