On Fri, 2011-12-16 at 10:42 +0100, Jean Spirat wrote:
> hi,
> 
>   I use backuppc to save a webserver. The issue is that the application 
> used on it is making thousand of little files used for a game to create 
> maps and various things. The issue is that we are now at 100GB of data 
> and 8.030.000 files so the backups takes 48H and more (to help the files 
> are on NFS share). I think i come to the point where file backup is at 
> it's limit.

> ps: backuppc server and the web server are debian linux,  i use rysnc 
> method and backup  the NFS that i mount localy on the backuppc server.

I have a backup with a similar number of files in and I have found that
tar is much better than rsync. Your issues are:

1. rsync will take a very long time and a very large amount of memory to
build the file tree, especially over NFS

2. NFS isn't really a high performance filesystem, you are better off
working locally on the server being backed up via ssh.

I would suggest you try the following: 

Move to tar over ssh on the remote webserver, the first full backup
might well take a long time but the following ones should be faster.

tar+ssh backups however use more bandwidth but as you are already using
nfs I am assuming you are on a local network of some sort.

-- 
Tim Fletcher <tim AT night-shade.org DOT uk>


------------------------------------------------------------------------------
Learn Windows Azure Live!  Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for 
developers. It will provide a great way to learn Windows Azure and what it 
provides. You can attend the event by watching it streamed LIVE online.  
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/