Veritas-bu

[Veritas-bu] FW: Slow performance during restore

2004-01-28 11:25:24
Subject: [Veritas-bu] FW: Slow performance during restore
From: John.Sokol AT reuters DOT com (John Sokol)
Date: Wed, 28 Jan 2004 16:25:24 +0000
Not sure if this has been answered already, I know it's a bit old.

Throughput is limited restoring lots of small files, simply because
there are a lot of small updates (and reads) to do for each file,
creating directory entries, updating inodes etc, causing much more time
to be spent head positioning.  There may be other factors affecting
restores, but that is certainly one of them.





-----Original Message-----
From: Mark Scheufele [mailto:mark.scheufele AT diasemi DOT com] 
Sent: 23 January 2004 13:15
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] FW: Slow performance during restore


Hi,

I've to administer several Netbackup installations. Most of them are
running NetBackup Version 4.5 FP3. All the installations are configured
the same way. I'm backing up a directly attached vxfs filesystem with a
size ranging from 600gb to 1tb. Media and master server are on the same
machine. The backup is configured that four datastreams can be written
in parallel to the tape drive. This means that I'm able to backup 
four toplevel directories of my filesystems in parallel. The netbackup
catalog backups reside on the rootdisk of the server.

One day I had to do a restore of a complete filesystem. The restore
itself took ages (49h for about 250 gigs) and I noticed the following:

        - when a big single file was being restored the I/O speed which
was used to read
          from tape was between 3-7mb/s. 
        - when a directory with many little files was being restored the
restore speed  from       tape was about 300kb/s and I noticed that the
%b column of iostat showed 99% busy 
          time for my rootdisks. So my guess is that it has something to
do with reading the 
          netbackup catalogs when doing a restore. I already asked a
Veritas consultant about 
          that but he said that this behaviour would be completely to
him.

So here are my questions:

- how does reading the netbackup catalogs affect restore performance
when I want to do a 
  complete restore of a filesystem that may contain several million
files? I'm not sure
  abou the procedure how netbackup gathers all necessary information
needed to restore a
  file?
- is the high I/O while doing a restore caused by reading the catalog
backups?
- might it be possible that doing such a big restore may consume a lot
of memory so that 
  the disk I/O is caused by extensive paging/swapping (sorry I cannot
remember about
  memory consumption of the restore processes anymore).
- I found the document http://seer.support.veritas.com/docs/230823.htm
at Veritas'
  support site which also deals with the problem of long restore times.
The following
  section is not completely clear to me. It would be great if someone
could shed some
  light onto it.

1) Specifically request the file to be restored. 
The inclusion of directories causes NetBackup to evaluate each and every
file deeper in the directory structure. Where each file search
represents 'x' number of system cycles, the specification by directory
expands this search time of the NetBackup database by a multiple of x,
equal to the number of entries deeper in the directory structure.
Typically, this additional search time is insignificant, but in the case
of large file systems or strained systems, it can impact performance.

Many thanks in advance for all your help,

mark

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

Get closer to the financial markets with Reuters Messaging - for more
information and to register, visit http://www.reuters.com/messaging

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.



<Prev in Thread] Current Thread [Next in Thread>