When NetWorker runs an incremental also passes the dat etime stamp of the
last backup. It uses this date time stamp as it performs the incremental.

It then walks the file system

1) Find file
2) read file info
3) Has it been modified since date/time stamp?
4a) yes - back it up
4b) No - skip the file
5) go back to 1

Thus in a file system that has millions of small files it spends more time
walking the file system than actually moving data.
This is where technologies such as SnapImage and Change Journal can help.
But they have their issues too.

What would be good would be if NetWorker were to become multithreaded and
use some of the technology other OS', such as VMS use, which is to have a
process that walks the file system and pass a list of files onto another
process that does the backup, thus keeping it fed with data.

Siobhan


On 28/10/07 11:20 AM, "George Sinclair" <George.Sinclair AT NOAA DOT GOV> wrote:

> If a given save set has a large number of files, say many tiny files,
> does this affect the time it will take a full to actually start running?
> 
> I guess where I'm going with this is that I'm trying to understand how
> Legato is affected by a large number of inodes and in what way. For
> example, I have this directory that takes hours to run a 'du' command on
> because it has a huge number of files, but it's only about 700 GB, but I
> might have another path that's over 1 TB, and it only takes 5 minutes to
> run a 'du'. Just curious what an incremental does before it actually
> starts backing up any data versus what a full would do. Obviously, both
> will update the client's index while the data is being backed up, so if
> the save set has a lot of files then the full backup will force the
> index to grow even larger, but it seems that while the full will
> undoubtedly cause the index to grow larger than the incremental, I would
> think the incremental would take longer to figure out what, if anything,
> needs to be backed up since it has to read the index from the last
> backup and traverse the inode information on the data path to determine
> what to actually back up, right? What about a full?
> 
> Thanks.
> 
> george


Siobhán

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER