Veritas-bu

Re: [Veritas-bu] Backup hangs on the directory with 1.5 million files in it (syntax for the exclude_list)

2010-10-06 11:58:25
Subject: Re: [Veritas-bu] Backup hangs on the directory with 1.5 million files in it (syntax for the exclude_list)
From: Nate Sanders <sandersn AT dmotorworks DOT com>
To: Aleksandr Nepomnyashchiy <anepomn AT gmail DOT com>
Date: Wed, 6 Oct 2010 10:58:11 -0500
Why not just do /my/huge/dir

No trailing slash or wildcard. It ignores the directory, not the files
in it.

On 10/06/2010 10:56 AM, Aleksandr Nepomnyashchiy wrote:
> Dear Gurus,
> Apologies for the beginner's question...
>
> I have noticed from looking at the bpbkar log that the backup hangs on
> the directory which has 1.5 million files in it (client is RHEL 5.2
> with NetBackup 6.5, server is RHEL 5.5 with NetBackup 6.5.6) . "ls -al
> /my/huge/dir" takes 3-4 hours, the file system is GFS which makes it
> even slower. No surprise that NetBackup is having trouble.
>
>
> I am trying to exclude the problematic directory from the backup and
> added "/my/huge/dir/*" to /usr/openv/netbackup/exclude_list. I'd like
> to avoid reading this huge list of files but this doesn't achive the
> purpose. Please see the last lines from bpbkar log :
>
> 12:28:41.917 [3560] <4> bpbkar: INF - Excluded /my/huge/dir/file_aaa
> by exclude_list entry /my/huge/dir/*
> 12:28:41.933 [3560] <4> bpbkar: INF - Excluded /my/huge/dir/file_aaan
> by exclude_list entry /my/huge/dir/*
> 12:28:41.943 [3560] <4> bpbkar: INF - Excluded /my/huge/dir/file_aaab
> by exclude_list entry /my/huge/dir/*
> 12:28:41.948 [3560] <4> bpbkar: INF - Excluded /my/huge/dir/file_aaac
> by exclude_list entry /my/huge/dir/*
> 12:28:41.959 [3560] <4> bpbkar: INF - Excluded /my/huge/dir/file_aaaz
> by exclude_list entry /my/huge/dir/*
> 12:28:41.969 [3560] <4> bpbkar: INF - Excluded /my/huge/dir/file_aaag
> by exclude_list entry /my/huge/dir/*
> 12:28:55.288 [3560] <16> bpbkar: ERR - bpbkar killed by SIGPIPE
> 12:28:55.289 [3560] <16> bpbkar: ERR - bpbkar FATAL exit status = 40:
> network connection broken
> 12:28:55.289 [3560] <4> bpbkar: INF - EXIT STATUS 40: network connection 
> broken
> 12:28:55.289 [3560] <4> bpbkar: INF - setenv FINISHED=0
>
> It looks like bpbkar is trying to go through each and every file in
> "/my/huge/dir" and trying to make a decision on each file
> individually.
>
>
> QUESTION: Is it possible to avoid the "/my/huge/dir/" as a whole and
> avoid reading the directory contents?
>
>
> Thank you,
> Aleksandr Nepomnyashchiy
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> .
>
>   

-- 
Nate Sanders            Digital Motorworks
System Administrator      (512) 692 - 1038




This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

<Prev in Thread] Current Thread [Next in Thread>