BackupPC-users

Re: [BackupPC-users] FTP, large files, and memory use

2011-04-19 14:55:04
Subject: Re: [BackupPC-users] FTP, large files, and memory use
From: John Sellens <jsellens AT syonex DOT com>
To: mstowe AT chicago.us.mensa DOT org, "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Tue, 19 Apr 2011 14:37:14 -0400
On Fri, Apr 15, 2011 at 07:27:29PM -0500, Michael Stowe wrote:
| > I don't think Windows is relevant here, I think the problem is because
| > it's a large file and backup is via FTP.
| 
| Find out!  See if you can ftp the file from a command line client.

The strace I included shows the file being read from the the remote
side and more memory being allocated on the server side.  Eventually
BackupPC_dump hits its limit and can't allocate memory, gives up, and
writes "Out of memory!" to the LOG file.

I'll have to look into the FTP libraries and see if I can find an
alternate approach that perhaps FTPs the file into a local tmp file.
The current approach for FTP in BackupPC clearly doesn't work for
larger files.


| > (I'm using FTP because my Windows rsync of choice - DeltaCopy - seemed
| > to be confused by Windows 7's multitude of links/shortcuts, and we
| > don't want to use a share on portable machines, out of a belief that
| > we can control FTP better than we can control shares.)
| 
| An intriguing belief, as is the concept that rsync is confused by links. 
| Mine isn't, but I don't use DeltaCopy.

I didn't say rsync was confused by links, I said DeltaCopy ran into
problems on Windows 7.  Though I did use the wrong terminology - it's the
Windows "junctions" that it has trouble with.  My guess might be that
DeltaCopy (or the underlying windows built rsync) needs to treat
a junction the same as a symbolic link.  Some others ran into the
same problem and mentioned it here:
    http://forums.aboutmyx.com/forums/posts/list/176.page

Cheers!

John

------------------------------------------------------------------------------
Benefiting from Server Virtualization: Beyond Initial Workload 
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve 
application availability and disaster protection. Learn more about boosting 
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>