Re: [Bacula-users] Backup of many many files
2008-09-03 07:28:47
On Wed, Sep 03, 2008 at 12:20:14PM +0100, Chris Howells wrote:
> Frank Sweetser wrote:
>
> > That difference strongly suggests to me that a big piece of your performance
> > problem is inserting attributes into the catalog. I'd suggest you look into
> > upgrading to 2.4 with the batch insert feature enabled, which should give
> > you
> > a big performance boost on inserting attributes.
>
> I wouldn't be so sure. 16 million files is rather a lot for a single job
> - it's an average of ~66KB per file for that job. (Although I back up
> about 8 million in 9 hours using multilple concurrent jobs, reading from
> 28 disks concurrently). Backing up small files is inefficient as the
> disk has to seek all over the place.
how does concurency work for tape libraries? can i use spooling and concurrency?
would that help? i've already got spooling job (2GB segments) and attributes.
>
> I suggest that you try this first:
>
> time tar cf /dev/null /the/directory/you/are/backing/up
i know it takes a long time to work thru that many files. i'm really looking
for ways to optimize the process.
thanks!
-- michael
>
> Then you should be able to work out whether its the Netapp limiting the
> back up speed, rather than bacula.
>
> --
> Chris Howells <chris AT transitive DOT com>
> Senior SysAdmin - Transitive Ltd.
> http://www.transitive.com
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
- [Bacula-users] Backup of many many files, Daniel Betz
- Re: [Bacula-users] Backup of many many files, Jose Ildefonso Camargo Tolosa
- Re: [Bacula-users] Backup of many many files, Daniel Betz
- Re: [Bacula-users] Backup of many many files, Michael Galloway
- Re: [Bacula-users] Backup of many many files, Michael Galloway
- Re: [Bacula-users] Backup of many many files, Frank Sweetser
- Re: [Bacula-users] Backup of many many files, Michael Galloway
- Re: [Bacula-users] Backup of many many files, Chris Howells
- Re: [Bacula-users] Backup of many many files,
Michael Galloway <=
- Re: [Bacula-users] Backup of many many files, Chris Howells
- Re: [Bacula-users] Backup of many many files, Ryan Novosielski
- Re: [Bacula-users] Backup of many many files, Michael Galloway
- Re: [Bacula-users] Backup of many many files, Ryan Novosielski
- Re: [Bacula-users] Backup of many many files, Hemant Shah
- Re: [Bacula-users] Backup of many many files, Arno Lehmann
- Re: [Bacula-users] Backup of many many files, Jose Ildefonso Camargo Tolosa
- Re: [Bacula-users] Backup of many many files, Ryan Novosielski
- Re: [Bacula-users] Backup of many many files, Hemant Shah
- Re: [Bacula-users] Backup of many many files, Jesper Krogh
|
|
|