BackupPC-users

Re: [BackupPC-users] Backup through slow line?

2008-09-03 00:14:04
Subject: Re: [BackupPC-users] Backup through slow line?
From: dan <dandenson AT gmail DOT com>
To: "Adam Goryachev" <mailinglists AT websitemanagers.com DOT au>
Date: Tue, 2 Sep 2008 22:13:57 -0600
wow.  id have to say that your cant trust that backup!  partial backups every night?  no consistent time which there would actually be a full backup?  *maybe* every now and then?  This is as good as not having a backup.

consider
1) do you need to backup all of the files on that system?  are there some large files or mp3 or video files that can be skipped?  consider narrowing the scope of the backup to just appropriate files.
2) consider compressing the data.  With such a small pipe you will be able to realize some benefit to using maximum compression with rsync.
3) consider bumping up the available bandwidth.  I buy DSL lines in many markets for my company and have never seen a situation where getting a 256Kb upload cost substantially more than a 128Kb link.
3a) sounds like you might be using ADSL, see if SDSL is available or see if there is a plan with more upload and the same download capacity.
4) consider changing the full backup script in backuppc to NOT use the no-times option that causes all of the files to be transfered.  This would make this essentially an incremental backup that sticks around.  I have great success in doing 1 full backup and a month worth of incrementals.  Saves a TON of bandwidth and shortens backups considerably.  I can backup a server with 4GB of data that changes infrequently in about 10Minutes via incrementals vs 3 Hours for a full over a 256Kb link.  The vast majority of the time is search for and transfering the file list because the server has email which is so small.  This server has nearly 1 million files which gives rsync a workout.

good luck


On Tue, Sep 2, 2008 at 5:03 PM, Adam Goryachev <mailinglists AT websitemanagers.com DOT au> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Les Mikesell wrote:
> Christian Völker wrote:
>> Now I want to backup these servers. Obviously it takes more than 24
>> hours to perform a full backup, so the running backup fails after 24hours.
>>
>> BackupPC now schedules the next full backup....which runs for 24h and
>> fails...and so on.

At some time, you need the daily admin process to run, depending on what
version of backuppc you are using, but I think backups are not run
during this time...

>> My question now is:
>> If the full backup fails, does it start from scratch every time or are
>> some files already stored in the backup and used during the next try, so
>> it'll finish some day?

You didn't send your host config file, or any log files, or any relevant
information, so we can't conclusively answer you...

> If you are using rsync as the transfer method it will continue
> approximately where it stopped.

However, you did later agree that you are using rsync, hence it will
continue pretty much where it stopped last time.

>> Meanwhile, the full backup runs for 5 days, interrupted five times after
>> 24hours...I don't know if it will finish some day.

Quite possibly not, given that the default configuration is configured
to remove a partial backup after it is 3 days old. That means (I may be
wrong) once it has been re-attempted three times, it would be removed
and the next backup would start from scratch....

>> Next question:
>> Assume, the full backup is finished after two weeks- will the next full
>> backup take the same amount of time?
>
> If you are using rsync it will be much faster next time, sending only
> the changes.

Hopefully this one has been answered for you by now...

Regards,
Adam
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIvcZVGyoxogrTyiURAt/7AJ4wDc0b92QJ1sAQ7dx6k/ERhflbOwCgm+Gx
avdBoJ8mURHvnn4GiwX1ikA=
=0zMN
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url="">
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/