BackupPC-users

Re: [BackupPC-users] Too many links : where is the problem ?

2011-08-09 10:47:31
Subject: Re: [BackupPC-users] Too many links : where is the problem ?
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Tue, 09 Aug 2011 09:46:04 -0500
On 8/9/2011 8:21 AM, friedmann wrote:
>
> it returned, within 1-2 sec :
> "Cant create 32001th link: Trop de liens"
> So yes, notice that i translated the error msg, for communication and 
> googling purposes : "Trop de liens" is the actual message, and now i realize 
> the importance of that detail ...
>
> I launched one BackupPC_dump this night, with the limit moved to 100000 : it 
> failed the same way, as expected.
>
> [Question] How can i check that "the number of links on one or more pool 
> files exceeds
> HardLinkMax", and figure out what is special/unique about such a file ? 
> [Question]
>
> > > [Arrow] mkdir 
> > > /var/lib/backuppc/pc/192.168.2.105/new//f%2fmedia%2fRhenovia/fVBox_Data_until_January_2011/fSBML_221010_IP3R_ForArnaud/fsbml2java_distrib/fDATE27octobre2010TIME09h17_huge9/fSim9997-singleInput1193.777e-9.0-J0_k188586.679e-3.0-IP33.8881551803E7e-9.0:
> > >  Trop de liens at /usr/share/backuppc/lib/BackupPC/Xfer/RsyncFileIO.pm 
> > > line 641
> Parent read EOF from child: fatal error!
> Done: 0 files, 0 bytes
> [Arrow] Got fatal error during xfer (Child exited prematurely)

That is making a directory that fails, not a pool link.  It makes sense 
for that to be a fatal error.

> And, maybe more insightful, i have 32002 subdirectories in the same 
> directory, i.e. in the "most populated" one (the one on which the backup 
> process fails : 
> /var/lib/backuppc/pc/192.168.2.105/new//f%2fmedia%2fRhenovia/fVBox_Data_until_January_2011/fSBML_221010_IP3R_ForArnaud/fsbml2java_distrib/fDATE27octobre2010TIME09h17_huge9)

Every subdirectory contains a ".." link to its parent, so the same limit 
will apply.

> It actually fails after 31998 created subdirectories on the ext3 partition
> tested on the host's ntfs partition : the test is passed. I even tried with a 
> value of 500000 : I stopped it after 296000 subdirectories were created.

That can't possibly be an efficient way to store things on the source 
filesystem.  Is there any chance of modifying the application creating 
that mess to use a deeper tree?

> [Idea] So i consider the problem as identified : It is relatively not related 
> to backuppc itself, but deals with a sort of filesystems incompatibility. 
> [Idea]
> [Question] And finally, the only question that lasts is : how can i increase 
> the number of subdirectories tolerated by ext3 ? [Question]

I don't think it is possible to change in ext3.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com

------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model 
configuration take the hassle out of deploying and managing Subversion and 
the tools developers use with it. Learn more about uberSVN and get a free 
download at:  http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/