Re: [Bacula-users] Performance options for single large (100TB) server b

Initial thoughts on this would be one of two ways (or both):

All in the fileset resource:
   As a fileset option something like:
         StreamPerFS
            which would kick off a stream for every FS in the fileset.   More of an 'automated' method to improve performance for those who don't want to manually tune it.

Or something like a new token to indicate what to back up as a new stream.    I.e.
      StreamFile =

            which would act just like "File = " but would kick off a new stream for that location.

This would need to tie into the SD to tell it to also spool direct dump each one seperately just list it does with multiple concurrant jobs.    Likewise tell the FD to spawn off a new thread for each directed (StreamFile) or dynamic 'StreamPerFS' point).

I don't know how much work that would be in the code?

-----Original Message-----
From: Eric Bollengier [mailto:eric.bollengier AT baculasystems DOT com]
Sent: Wednesday, July 6, 2011 11:20 AM
To: bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] Performance options for single large (100TB) server backup?
Hello,

On 07/06/2011 04:20 PM, Florian Heigl wrote:
> Saving multiple streams is something that has been proven as a
> solution for many years, and where that is still too slow NDMP comes
> into place. (in case of ZFS NDMP is still at a unusable stage)
>
> 100TB is a lot, but I wonder if everyone agrees the "right" solution
> would be saving multiple streams instead of splitting up the source
> system (will be fun to do a restore of such a split client)...
>
> Hopefully some of the larger Bacula customers will fund these features
> some day, as both have the mix of being very important, complex and
> elemental changes :))

I would like also to see such feature in Bacula :-) (even if a
workaround already exists)

Have you any idea on how Bacula would choose when it should start a new
"backup stream" (can be automatic, by fileset configuration, etc..) ?

I think that it would be nice the spread the load between physical
disks, and not only trying to read the same FS with many threads.

Bye

--
Need professional help and support for Bacula ?
Visit http://www.baculasystems.com

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users