BackupPC-users

[BackupPC-users] My cpool and pool dirs are empty

2009-03-11 16:56:55
Subject: [BackupPC-users] My cpool and pool dirs are empty
From: "Wayne L Andersen" <waynea AT clima-tech DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Wed, 11 Mar 2009 14:48:55 -0600
Yes, I did move the directories, but it was immediately after install and 
before I had performed any of the other setup steps.

I guess from the question that that was a mistake.

So how do I fix it?

On Wed, 11 Mar 2009 19:55:24 +0000
  backuppc-users-request AT lists.sourceforge DOT net wrote:
> Send BackupPC-users mailing list submissions to
>       backuppc-users AT lists.sourceforge DOT net
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>       https://lists.sourceforge.net/lists/listinfo/backuppc-users
> or, via email, send a message with subject or body 'help' to
>       backuppc-users-request AT lists.sourceforge DOT net
> 
> You can reach the person managing the list at
>       backuppc-users-owner AT lists.sourceforge DOT net
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of BackupPC-users digest..."
> 
> 
> Today's Topics:
> 
>   1. Re: Resuming backup after losing connection to   client
>      (Adam Goryachev)
>   2. Re: Resuming backup after losing connection to client
>      (Christian G. von Busse)
>   3. Re: Resuming backup after losing connection to   client
>      (Adam Goryachev)
>   4. My cpool and pool dirs are empty (Wayne L Andersen)
>   5. Re: My cpool and pool dirs are empty (Les Mikesell)
>   6. Re: Backup faiures, how to troublehoot (Andrew Libby)
>   7. Re: Backup faiures, how to troublehoot (Les Mikesell)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 11 Mar 2009 17:08:04 +1100
>From: Adam Goryachev <mailinglists AT websitemanagers.com DOT au>
> Subject: Re: [BackupPC-users] Resuming backup after losing 
>connection
>       to      client
> To: "General list for user discussion,        questions and support"
>       <backuppc-users AT lists.sourceforge DOT net>
> Message-ID: <49B75544.1020002 AT websitemanagers.com DOT au>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Christian G. von Busse wrote:
>>> Just go to the home page for that computer and hit "start full
>>> backup" and off it should go...
>>
>> But there is no way to have backuppc do this automatically if the
>> backup terminated because the network connection was lost? I had
>> hoped that backuppc could run unattended.
>>
>> The log looks like this:
>>
>> [full backup running] [here the network connection was lost:]
>> 2009-03-10 03:05:23 Backup failed on server1 (Child exited
>> prematurely) 2009-03-10 03:05:23 Running BackupPC_link server1
>> (pid=13284) 2009-03-10 03:06:03 Finished server1 (BackupPC_link
>> server1) 2009-03-10 04:00:00 Next wakeup is 2009-03-10 05:00:00
>> 2009-03-10 04:00:02 Started full backup on server1 (pid=14953,
>> share=Backup) 2009-03-10 04:08:09 Backup failed on server1 (Child
>> exited prematurely) 2009-03-10 05:00:00 Next wakeup is 2009-03-10
>> 06:00:00 2009-03-10 05:00:02 Started full backup on server1
>> (pid=17309, share=Backup) 2009-03-10 05:08:52 Backup failed on
>> server1 (Child exited prematurely) 2009-03-10 06:00:00 Next wakeup
>> is 2009-03-10 07:00:00 2009-03-10 06:00:03 Started full backup on
>> server1 (pid=28117, share=Backup) 2009-03-10 06:08:22 Backup failed
>> on server1 (Child exited prematurely) 2009-03-10 07:00:00 Next
>> wakeup is 2009-03-10 08:00:00 2009-03-10 07:00:02 Started full
>> backup on server1 (pid=29979, share=Backup) 2009-03-10 07:08:33
>> Backup failed on server1 (Child exited prematurely)
>>
>> As you can see, after the network connection was lost, full backup
>> was not resumed automatically with the error "Child exited
>> prematurely".
> Actually, you can see that backuppc has restarted the backup
> automatically every hour which is the default wakeupschedule.
> You could set the schedule to run every 10 minutes, which means 
>within
> 10mins of the failure backuppc will retry. Of course, if backuppc
> isn't actually making any progress, then it will never complete a
> successful backup....
> 
> Regards,
> Adam
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkm3VT0ACgkQGyoxogrTyiUcywCePXOQsw6Pff6WZxqEiul0rUMM
> 9QAAmwQGaAL6bSGQFtJp1u4agox+Rtyi
> =m2le
> -----END PGP SIGNATURE-----
> 
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Wed, 11 Mar 2009 07:19:44 +0100 (CET)
>From: "Christian G. von Busse" <CvB AT kruemel DOT org>
> Subject: Re: [BackupPC-users] Resuming backup after losing 
>connection
>       to client
> To: "General list for user discussion,        questions and support"
>       <backuppc-users AT lists.sourceforge DOT net>
> Message-ID: <3877.192.168.1.150.1236752384.squirrel AT www.kruemel DOT org>
> Content-Type: text/plain;charset=iso-8859-1
> 
> 
>>> That's good to know, thanks. So the question is (only), how do I 
>>>tell
>>> backuppc to immediately start a new/queue (full or partial) backup, 
>>>if
>>> the prior one was terminated accidentally/abnormally.
>>> Backkuppc did not start the full backup here again, yet - although 
>>>there
>>> was no blackout period and wake up points..
>> Check your wakeupschedule, backuppc will only re-start or consider
>> trying to do a needed backup once at each wakeupschedule.
> 
> If I understand the log correctly which I just posted to this list,
> backuppc did wake up again, but always terminated with some kind of 
>error
> Backup failed on server1 (Child exited prematurely)...
> 
> Christian
> 
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Wed, 11 Mar 2009 22:52:08 +1100
>From: Adam Goryachev <mailinglists AT websitemanagers.com DOT au>
> Subject: Re: [BackupPC-users] Resuming backup after losing 
>connection
>       to      client
> To: "General list for user discussion,        questions and support"
>       <backuppc-users AT lists.sourceforge DOT net>
> Message-ID: <49B7A5E8.4060502 AT websitemanagers.com DOT au>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Christian G. von Busse wrote:
>>>> That's good to know, thanks. So the question is (only), how do I 
>>>>tell
>>>> backuppc to immediately start a new/queue (full or partial) backup, 
>>>>if
>>>> the prior one was terminated accidentally/abnormally.
>>>> Backkuppc did not start the full backup here again, yet - although 
>>>>there
>>>> was no blackout period and wake up points..
>>> Check your wakeupschedule, backuppc will only re-start or consider
>>> trying to do a needed backup once at each wakeupschedule.
>> 
>> If I understand the log correctly which I just posted to this list,
>> backuppc did wake up again, but always terminated with some kind of 
>>error
>> Backup failed on server1 (Child exited prematurely)...
> 
> Correct, the backup process was terminated due to "Child exited
> prematurely" every time backuppc attempted to do the backup. You 
>should
> look at the host Log file for details of what happened during each
> backup attempt. Possibly none of the backups even started due to 
>some
> error, possibly each backup ran for some period of time and 
>transferred
> some data before failing. We don't know from the limited information 
>you
> have provided.
> 
> Perhaps you should try the following:
> 1) login to the backuppc server
> 2) su - backuppc (or whatever your backuppc user is)
> 3) /path/to/backuppc/bin/BackupPC_dump -v -f server1 (or whatever 
>the
> name of your server is)
> 
> This will show you exactly what is happening when backuppc attempts 
>to
> do a backup.
> 
> You should probably tell the backuppc web interface to not run any
> backups for this host for a couple of hours before you do the above, 
>as
> I've noticed (at least on the older 2.1 version) that backuppc will
> launch a new backup while the command line backup is running, and 
>they
> will "step on each others toes" as far as both writing to the same
> filenames etc...
> 
> Regards,
> Adam
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkm3peIACgkQGyoxogrTyiVWswCfaFf2SUQplKVORaa9BKX8WDep
> g+4AniBINX0O7u7w2eiKy8CLmtKc2cTJ
> =yHES
> -----END PGP SIGNATURE-----
> 
> 
> 
> ------------------------------
> 
> Message: 4
> Date: Wed, 11 Mar 2009 11:33:50 -0600
>From: "Wayne L Andersen" <waynea AT clima-tech DOT com>
> Subject: [BackupPC-users] My cpool and pool dirs are empty
> To: backuppc-users AT lists.sourceforge DOT net
> Message-ID: <ximss-3949748 AT clima-tech DOT com>
> Content-Type: text/plain;charset="utf-8"; format="flowed"
> 
> There must be something wrong with my installation, I am assuming 
>the 
> required hard links did not get created correctly.
> 
> My backups and restores work great but the pooling is not 
>functioning, my 
> status page has these lines in the summary at the top.
> 
> Pool is 0.00GB comprising 0 files and 1 directories (as of 3/11 
>01:00),
> Pool hashing gives 0 repeated files with longest chain 0,
> Nightly cleanup removed 0 files of size 0.00GB (around 3/11 01:00),
> Pool file system was recently at 46% (3/11 11:21), today's max is 
>46% (3/11 
> 01:00) and yesterday's max was 46%.
> 
> 
> 
> ------------------------------
> 
> Message: 5
> Date: Wed, 11 Mar 2009 13:45:10 -0500
>From: Les Mikesell <les AT futuresource DOT com>
> Subject: Re: [BackupPC-users] My cpool and pool dirs are empty
> To: "General list for user discussion,        questions and support"
>       <backuppc-users AT lists.sourceforge DOT net>
> Message-ID: <49B806B6.1000807 AT futuresource DOT com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Wayne L Andersen wrote:
>> There must be something wrong with my installation, I am assuming 
>>the 
>> required hard links did not get created correctly.
>> 
>> My backups and restores work great but the pooling is not 
>>functioning, my 
>> status page has these lines in the summary at the top.
>> 
>> Pool is 0.00GB comprising 0 files and 1 directories (as of 3/11 
>>01:00),
>> Pool hashing gives 0 repeated files with longest chain 0,
>> Nightly cleanup removed 0 files of size 0.00GB (around 3/11 01:00),
>> Pool file system was recently at 46% (3/11 11:21), today's max is 
>>46% (3/11 
>> 01:00) and yesterday's max was 46%.
> 
> Did you change the archive location after installing?  Is everything 
>on 
> the same filesystem?  Does the backuppc user have write access to 
> pool/cpool?
> 
> -- 
>   Les Mikesell
>    lesmikesell AT gmail DOT com
> 
> 
> 
> 
> 
> 
> ------------------------------
> 
> Message: 6
> Date: Wed, 11 Mar 2009 15:15:01 -0400
>From: Andrew Libby <alibby AT xforty DOT com>
> Subject: Re: [BackupPC-users] Backup faiures, how to troublehoot
> To: "General list for user discussion,        questions and support"
>       <backuppc-users AT lists.sourceforge DOT net>
> Message-ID: <49B80DB5.2020803 AT xforty DOT com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> Andrew Libby wrote:
>> Les Mikesell wrote:
>>> Andrew Libby wrote:
>>>>> There are a lot of files on the system, and backups take
>>>> quite a while so I upped the timeout to 40 hrs, but
>>>> backups seem to be failing now in under 20 hrs when they
>>>> were failing at the 20 hr mark.
>>>>
>>>> Any hints or suggestions are greatly apprecaiated.  Other
>>>> info that might be helpful:
>>>>
>>>> Both client and server are ubuntu 8.04.
>>>> Transport is rsync.  Client has about 6.5 million files.
>>> One possibility is a corrupted file system. Have you done an fsck 
>>> recently?  That's also a lot of files to process in one run.  Rsync 
>>> loads the whole directory tree in RAM and becomes extremely slow if 
>>>you 
>>> go into swap.
>>>
>> 
>> Hi Les,
>> 
>> So we calculated it and the ram comes to under a gig.  We've
>> got 4G of ram.  The system does not seem to swap when
>> processing a backup.   fsck also seems to be in good standing.
>> 
>> Thanks for the suggestions,
>> 
>> Andy
> 
> Okay, so I've got some more information.  Seems that the
> log in /var/lib/backuppc/log called LOG has
> 
> 2009-03-11 10:12:23 shyamalan: Out of memory!
> 2009-03-11 10:12:35 Backup failed on shyamalan (Child exited
> prematurely)
> 
> 
> I thought out of memory errors were to happen on the client,
> rather than the server.  I don't know why I would think that
> though.
> 
> The server here has 4G, and as I said, the ram required for
> the 6M files should be under a gig 6512257 * 100 == 621M
> unless my math stinks (and it frequently does).
> 
> My sar stats clearly show the backuppc server running out
> of memory, but swap utilization remains low.
> 
> Thanks for any further insight.
> 
> Andy
> 
> 
> 
> ------------------------------
> 
> Message: 7
> Date: Wed, 11 Mar 2009 14:55:07 -0500
>From: Les Mikesell <lesmikesell AT gmail DOT com>
> Subject: Re: [BackupPC-users] Backup faiures, how to troublehoot
> To: "General list for user discussion,        questions and support"
>       <backuppc-users AT lists.sourceforge DOT net>
> Message-ID: <49B8171B.2000501 AT gmail DOT com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Andrew Libby wrote:
>> 
>>>
>>> So we calculated it and the ram comes to under a gig.  We've
>>> got 4G of ram.  The system does not seem to swap when
>>> processing a backup.   fsck also seems to be in good standing.
>>>
>>> Thanks for the suggestions,
>>>
>>> Andy
>> 
>> Okay, so I've got some more information.  Seems that the
>> log in /var/lib/backuppc/log called LOG has
>> 
>> 2009-03-11 10:12:23 shyamalan: Out of memory!
>> 2009-03-11 10:12:35 Backup failed on shyamalan (Child exited
>> prematurely)
>> 
>> 
>> I thought out of memory errors were to happen on the client,
>> rather than the server.  I don't know why I would think that
>> though.
> 
> The client sends the directory list to the server, then the server 
>walks 
> though it comparing against the previous full.
> 
>> The server here has 4G, and as I said, the ram required for
>> the 6M files should be under a gig 6512257 * 100 == 621M
>> unless my math stinks (and it frequently does).
> 
> Where did you find the '100' factor?  I'd have guessed bigger - and 
>I 
> think some versions of 64 bit perl have bugs that make it consume 
>more 
> memory.  It might work better to stay 32-bit on a 4 gig machine - or 
>add 
> more RAM if you are 64-bit.
> 
>> My sar stats clearly show the backuppc server running out
>> of memory, but swap utilization remains low.
> 
> Sar takes snapshots at 10 minute intervals - you might easily miss 
> peaks.  If you are running more than one backup concurrently it 
>might 
> help to cut back.
> 
> -- 
>   Les Mikesell
>    lesmikesell AT gmail DOT com
> 
> 
> 
> 
> 
> ------------------------------
> 
> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) 
>are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly 
>and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based 
>development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> 
> ------------------------------
> 
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users AT lists.sourceforge DOT net
> List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:    http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
> 
> 
> End of BackupPC-users Digest, Vol 35, Issue 13
> **********************************************

Wayne L Andersen
System Administrator
Clima-Tech Corporation
Main 208-375-9755
Direct 208-947-1849
Cell 208-440-2912

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/