BackupPC-users

Re: [BackupPC-users] BackupPC_dump memory usage

2013-03-11 06:01:37
Subject: Re: [BackupPC-users] BackupPC_dump memory usage
From: ashka <shellgratuit AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Mon, 11 Mar 2013 10:59:04 +0100

Le 11.03.2013 02:17, backuppc AT kosowsky DOT org a écrit :
Holger Parplies wrote at about 02:00:45 +0100 on Monday, March 11, 2013:
 > Hi,
 > 
 > ashka wrote on 2013-03-10 18:55:38 +0100 [Re: [BackupPC-users] BackupPC_dump memory usage]:
 > > Le 10.03.2013 18:16, Les Mikesell a écrit :
 > > > On Sun, Mar 10, 2013 at 4:01 AM, ashka <shellgratuit AT gmail DOT com> wrote:
 > > >> [...]
 > > >> I'm using BackupPC 3.2.1 and I'm having a lot of trouble with a server :
 > > >> the BackupPC_dump process takes something close to 10Gb of RAM.
 > > >> In addition, the backup process keeps dying like that :
 > 
 > 10 GB, really? That means you are running a 64-bit system and you have *a lot*
 > of RAM. How much *do* you have, and which column of 'ps' output are you
 > quoting? I wouldn't be surprised at all if the cause of the process dying is
 > an 'out of memory'. Is your system swapping? How much swap does it have, how
 > much is used?
 > 
 > > > If you are using rsync, both ends will hold the directory contents of
 > > > the whole tree in RAM during the transfer for comparison.  If you have
 > > > millions of files you may not have enough ram for that.
 > 
 > Absolutely correct, and the *first* thing to check. Honestly, the fact that
 > you don't have an answer to this comment before asking your question below
 > irritates me.
 > 
 > > > Also, if you haven't already checked you might want to be sure that no
 > > > other backups are running at the same time as the one having problems.
 > > 
 > > Is there a nice way to achieve that with BackupPC ?
 > 
 > That has been answered, but really, 10 GB for one backup process feels wrong.
 > Having concurrent backups running would make things even *worse*, but *not*
 > having them running doesn't fix it.
 > 

Seems wrong to me too considering that I back up about a dozen
machines using a plugcomputer with a single ARM core running at 800MHz
with just 512MB of RAM and no swap. Heck, I have for kicks got it
running on a DNS-323 NAS with I think just 64MB or RAM (though I do
have about 512MB in swap) and a 500MHz ARM core. I even run 2-3 dumps
at once.

The point is that BackupPC typically isn't such a resource a hog by at
least an order of magnitude... unless you perhaps have a very
different situation from ordinary pc-type backups...

------------------------------------------------------------------------------
Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester  
Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the  
endpoint security space. For insight on selecting the right partner to 
tackle endpoint security challenges, access the full report. 
http://p.sf.net/sfu/symantec-dev2dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Hello,

(FYI, I did kind of solve the problem by excluding tons of file, but I'd like to keep them anyway)
 > 10 GB, really? That means you are running a 64-bit system and you have *a lot*
 > of RAM. How much *do* you have, and which column of 'ps' output are you
 > quoting? I wouldn't be surprised at all if the cause of the process dying is
 > an 'out of memory'. Is your system swapping? How much swap does it have, how
 > much is used?
I'm using htop for simplicity purposes. It uses sometimes about 10Gb, it swaps when two backups are running at the same time.
 > > > If you are using rsync, both ends will hold the directory contents of
 > > > the whole tree in RAM during the transfer for comparison.  If you have
 > > > millions of files you may not have enough ram for that.
 > 
 > Absolutely correct, and the *first* thing to check. Honestly, the fact that
 > you don't have an answer to this comment before asking your question below
 > irritates me.
I do have millions of files, I can't really get them out the backup though. Those files are really important, and I excluded them temporarily just to see how it was working out.

--
ashka
------------------------------------------------------------------------------
Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester  
Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the  
endpoint security space. For insight on selecting the right partner to 
tackle endpoint security challenges, access the full report. 
http://p.sf.net/sfu/symantec-dev2dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/