Re: [Bacula-users] How to handle long, time consuming backups
2008-12-31 05:13:04
Hi all,
I have the same question: how to backup a remote samba server with 60GB
of file system space?
I get an average throughput of 220Kb/s but the job always fails with
network error (Connection reset). In the middle there is a firewall and
I asked to people guys to raise tcp conn timeout..
Splitting the job into 5 subjobs doesn't help.. after 2 or 3 hours the
connection is broken..
After going through docs and forums I tried to add spooling and
concurrency, so that other jobs must not wait for the samba one to
complete.. no way! I can't see any concurrency at all
Here it is the config
Director {
...
Maximum Concurrent Jobs = 4
}
JobDefs {
...
SpoolData = yes
}
---
Storage {
...
Maximum Concurrent Jobs = 20
}
Device {
...
Spool Directory = /data/spool;
Maximum Spool Size = 40 GB ;
Maximum Job Spool Size = 1 GB;
...
}
I get spooling but jobs don't run concurrently.. may because jobs use
different pool? I also run interactively 3 jobs associated to the same
pool..
spooling itself is a serial process: mount tape, write data to spool
until it reaches the max size, destage to tape, restart writing to
spool...
I started using 10gb max size but I always get broken pipe error
message, now with 1GB it's working (22 seconds to despool 45MB/s)
I want to solve the problem in bacula way, not rsyncing data between the
client and the server.. do you think setting up a file storage daemon on
the client and then migrate volume to main storage is reasonable?
Thanks a lot!
Massimo
In the middle there is a checkpoint firewall
On Tue, 2008-12-30 at 19:38 -0500, John Drescher wrote:
> > A large, mostly static data set, and low bandwidth connection: this is
> > rsync's specialty! If you are not familiar with it, look here:
> >
> > http://samba.anu.edu.au/rsync/
> >
> > I suggest you use rsync to sync your Windows client to a copy at your main
> > site (costs you 20GB of disk space somewhere) then do your full Bacula
> > backup from there.
> >
> > There will no doubt be some issues with timing/scheduling, and possibly
> > permissions, to work out. And restores will be two-stage. But I think it
> > would be worth it.
> >
>
> I do that for a remote backup of a few GB and it works great.
>
> John
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
|
|