BackupPC-users

Re: [BackupPC-users] Another jLib/fixLinks issue.

2010-12-09 18:37:00
Subject: Re: [BackupPC-users] Another jLib/fixLinks issue.
From: Robin Lee Powell <rlpowell AT digitalkingdom DOT org>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 9 Dec 2010 15:35:26 -0800
On Tue, Dec 07, 2010 at 02:27:46PM -0500, Jeffrey J. Kosowsky wrote:
> Robin Lee Powell wrote at about 11:05:46 -0800 on Tuesday, December 7, 2010:
>  > On Tue, Dec 07, 2010 at 01:58:28PM -0500, Jeffrey J. Kosowsky wrote:
>  > > Robin Lee Powell wrote at about 15:40:04 -0800 on Monday, December 6, 
> 2010:
>  > >  > This is *fascinating*.
>  > >  > 
>  > >  > >From the actually-fixing-stuff part of the run, I get:
>  > >  > 
>  > >  >   ERROR: 
> "tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg"
>  - Too many links if added to "59c43b51dbdd9031ba54971e359cdcec"
>  > >  > 
>  > >  > to which I say "lolwut?" and investigate.
>  > >  > 
>  > >  > $ ls -li /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec*
>  > >  > 2159521202 -rw-r----- 31999 backuppc backuppc 76046 Oct  7 08:29 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec
>  > >  > 2670969865 -rw-r----- 31999 backuppc backuppc 76046 Oct 16 15:15 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_0
>  > >  >   79561977 -rw-r----- 31999 backuppc backuppc 76046 Oct 22 22:07 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_1
>  > >  >  156369809 -rw-r----- 31999 backuppc backuppc 76046 Oct 31 09:06 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_2
>  > >  > 3389777838 -rw-r----- 31999 backuppc backuppc 76046 Nov  7 09:10 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_3
>  > >  >  106188559 -rw-r----- 31999 backuppc backuppc 76046 Nov 13 15:10 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_4
>  > >  >  247044591 -rw-r----- 31999 backuppc backuppc 76046 Nov 19 17:20 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_5
>  > >  >  293083240 -rw-r----- 31999 backuppc backuppc 76046 Nov 26 06:14 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_6
>  > >  >  513555136 -rw-r----- 31999 backuppc backuppc 76046 Dec  1 19:37 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_7
>  > >  >   52908307 -rw-r-----  7767 backuppc backuppc 76046 Dec  4 10:37 
> /backups/cpool/5/9/c/59c43b51dbdd9031ba54971e359cdcec_8
>  > >  > $ ls -li 
> /backups/pc/tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg
>             374791856 -rw-r----- 1 backuppc backuppc 76046 Dec  4 08:03 
> /backups/pc/tm50-s00292__nfs/68/f%2f/fshared/fthepoint/fsite_images/f0042/f4097/fMULTI_medium.jpg
>  > >  > 
>  > >  > That's a bunch of files with *thirty two thousand* hard links.
>  > >  > Apparently that's a limit of some kind.  BackupPC handles this by
>  > >  > adding new copies, a hack that BackupPC_fixLinks is apparently
>  > >  > unaware of.
>  > > 
>  > > BackupPC_fixLinks does know about the limit and in fact is careful
>  > > not to exceed it (using the same hack) when it combines/rewrites
>  > > links. Other than that, I'm not sure where you think
>  > > BackupPC_fixLinks needs to be aware of it?
>  > 
>  > I would expect it to not emit an ERROR there?  :)  Shouldn't it move
>  > to the next file, and the next, and so on, until it finds one it
>  > *can* link to?
>  > 
>  > It emitted thousands of such ERROR lines; surely that's not good
>  > behaviour.
> 
> Well, it was designed (and tested) for the use case where this was
> a *rare* event so that it would be interesting to signal it.
> Perhaps even then  "WARN" or "NOTICE" would have been better than
> "ERROR." Indeed, that would be a good change (and you could always
> 'grep -v' it out of your results).
> 
> My thinking was that in the case of a messed-up pool knowing that
> some files had 32000 links would be worthy of notice.... of
> course, it seems like for you this is a non note-worthy
> occurrence.
> 
> Now per my comments in the code, this doesn't break anything, it
> only means that the links can't be combined and so pool usage
> can't be freed up for that file. 

I'm worried we're talking past each other, so be gentle if I'm
confused.  :)

If I have thousands of such files, each copy takes up the usual
amount of space.  They *should* be linked into the pool, so as to
take up 32k times less space.  The reason I ran it in the first
place was to link unlinked files like this into the pool; in this
case, unless I'm missing something, they stayed unlinked.

Since my goal was to free up space, it's important to me.

I agree it's something of an edge case, though, and if you don't
want to fix it I'd totally understand.

-Robin

-- 
http://singinst.org/ :  Our last, best hope for a fantastic future.
Lojban (http://www.lojban.org/): The language in which "this parrot
is dead" is "ti poi spitaki cu morsi", but "this sentence is false"
is "na nei".   My personal page: http://www.digitalkingdom.org/rlp/

------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/