Re: [BackupPC-users] 8.030.000, Too much files to backup ?

you may try to use a rsyncd directly on the server. This may speed up things.
another thing is to split the large backup into several smaller ones. I've an email cluster with 8TB and millions of small files (I'm using dovecot), theres also a san involved. in order to use all the bandwidth available I configured backup to run from username starting in a to e, f to j and so on, then they all run at the same time. incremental take about 1 hour and full about 5.
cheers
pedro

Sent from my galaxy nexus.
www.linux-geex. com

On Dec 16, 2011 9:47 AM, "Jean Spirat" <jean.spirat AT squirk DOT org> wrote:

hi,

I use backuppc to save a webserver. The issue is that the application
used on it is making thousand of little files used for a game to create
maps and various things. The issue is that we are now at 100GB of data
and 8.030.000 files so the backups takes 48H and more (to help the files
are on NFS share). I think i come to the point where file backup is at
it's limit.

Is any of you reached this kind of issue, how do you solve that ?
going for a different way of backuping the files by using
block/partition backup system etc..

the issue is that so manyfiles make the file by file process very slow.
I was thinking about block backup but i do not even know tools that does
it apart R1backup that is a commercial one.

If anyone here met the same issue and give some pointers it would be
great even perhaps if someone found a way to continue using backuppc in
that extreme situation.

ps: backuppc server and the web server are debian linux, i use rysnc
method and backup the NFS that i mount localy on the backuppc server.

regards,
Jean.

------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

------------------------------------------------------------------------------
Learn Windows Azure Live!  Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for 
developers. It will provide a great way to learn Windows Azure and what it 
provides. You can attend the event by watching it streamed LIVE online.  
Learn more at http://p.sf.net/sfu/ms-windowsazure

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/