BackupPC-users

Re: [BackupPC-users] Another jLib/fixLinks issue.

2010-12-07 14:29:38
Subject: Re: [BackupPC-users] Another jLib/fixLinks issue.
From: "Jeffrey J. Kosowsky" <backuppc AT kosowsky DOT org>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Tue, 07 Dec 2010 14:27:46 -0500
Robin Lee Powell wrote at about 11:05:46 -0800 on Tuesday, December 7, 2010:
 > On Tue, Dec 07, 2010 at 01:58:28PM -0500, Jeffrey J. Kosowsky wrote:
 > > Robin Lee Powell wrote at about 15:40:04 -0800 on Monday, December 6, 2010:
 > >  > This is *fascinating*.
 > >  > 
 > >  > >From the actually-fixing-stuff part of the run, I get:
 > >  > 
 > >  >   ERROR: 
 > > "tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg"
 > >  - Too many links if added to "59c43b51dbdd9031ba54971e359cdcec"
 > >  > 
 > >  > to which I say "lolwut?" and investigate.
 > >  > 
 > >  > $ ls -li /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec*
 > >  > 2159521202 -rw-r----- 31999 backuppc backuppc 76046 Oct  7 08:29 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec
 > >  > 2670969865 -rw-r----- 31999 backuppc backuppc 76046 Oct 16 15:15 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_0
 > >  >   79561977 -rw-r----- 31999 backuppc backuppc 76046 Oct 22 22:07 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_1
 > >  >  156369809 -rw-r----- 31999 backuppc backuppc 76046 Oct 31 09:06 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_2
 > >  > 3389777838 -rw-r----- 31999 backuppc backuppc 76046 Nov  7 09:10 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_3
 > >  >  106188559 -rw-r----- 31999 backuppc backuppc 76046 Nov 13 15:10 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_4
 > >  >  247044591 -rw-r----- 31999 backuppc backuppc 76046 Nov 19 17:20 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_5
 > >  >  293083240 -rw-r----- 31999 backuppc backuppc 76046 Nov 26 06:14 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_6
 > >  >  513555136 -rw-r----- 31999 backuppc backuppc 76046 Dec  1 19:37 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_7
 > >  >   52908307 -rw-r-----  7767 backuppc backuppc 76046 Dec  4 10:37 
 > > /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_8
 > >  > $ ls -li 
 > > /backups/pc/tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg
 > >             374791856 -rw-r----- 1 backuppc backuppc 76046 Dec  4 08:03 
 > > /backups/pc/tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg
 > >  > 
 > >  > That's a bunch of files with *thirty two thousand* hard links.
 > >  > Apparently that's a limit of some kind.  BackupPC handles this by
 > >  > adding new copies, a hack that BackupPC_fixLinks is apparently
 > >  > unaware of.
 > > 
 > > BackupPC_fixLinks does know about the limit and in fact is careful
 > > not to exceed it (using the same hack) when it combines/rewrites
 > > links. Other than that, I'm not sure where you think
 > > BackupPC_fixLinks needs to be aware of it?
 > 
 > I would expect it to not emit an ERROR there?  :)  Shouldn't it move
 > to the next file, and the next, and so on, until it finds one it
 > *can* link to?
 > 
 > It emitted thousands of such ERROR lines; surely that's not good
 > behaviour.

Well, it was designed (and tested) for the use case where this was a
*rare* event so that it would be interesting to signal it. Perhaps
even then  "WARN" or "NOTICE" would have been better than "ERROR."
Indeed, that would be a good change (and you could always 'grep -v' it
out of your results).

My thinking was that in the case of a messed-up pool knowing that some
files had 32000 links would be worthy of notice.... of course, it
seems like for you this is a non note-worthy occurrence.

Now per my comments in the code, this doesn't break anything, it only
means that the links can't be combined and so pool usage can't be
freed up for that file. Specifically the code says:

                        return -1; #Note, this still leaves everything OK, 
since the file is still linked to the pool
                       #just means we can't free up an extra pool entry (should 
rarely happen anyway since
                                   #we have already checked this...


You may want to think of whether there is a boundary case in which
some other relinking could still be helpful. But certainly it does no
harm and the pool is as optimized as it can be given the inode hard link
limitation (again unless there is a boundary case I am not
considering).


------------------------------------------------------------------------------
What happens now with your Lotus Notes apps - do you make another costly 
upgrade, or settle for being marooned without product support? Time to move
off Lotus Notes and onto the cloud with Force.com, apps are easier to build,
use, and manage than apps on traditional platforms. Sign up for the Lotus 
Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/