Holger Parplies wrote at about 04:48:10 +0200 on Monday, October 3, 2011:
 > You might want to use parts of either Jeffrey's or my script. Jeffrey builds 
 > a
 > "pool" of information which files use which inode. I build a file with pretty
 > much the same information. Both don't need to be stored on the pool FS they
 > refer to (because they're not links to content, they're files with
 > information). That way, you could iterate over the pool *once* and reuse the
 > information multiple times.
 > 
 > My method has the downside that you need to sort a huge file (but the 'sort'
 > command handles huge files rather well). Jeffrey's method has the downside
 > that you have an individual file per inode with typically probably only a few
 > hundred bytes of content, which might end up occupying 4K each - depending on
 > file system. Also, traversing the tree should take longer, because each file
 > is opened and closed multiple times - once per link to the inode it 
 > describes.
 > Actually, a single big file has a further advantage. It's rather fast to look
 > for something (like all non-pooled files) with a Perl script. Traversing an
 > "ipool" is bound to take a similar amount of time as traversing pool or cpool
 > will.

Holger, have you ever compared the time on actual data?

Just one nit, I do allow for caching the inode pool so frequently
referenced pool files do not require the corresponding inode pool file
to be opened repeatedly.

Also, I would think that with a large pool, that the file you
construct would take up a fair bit of memory and that the O(n log n)
to search it might take more time then referencing a hierarchically
structured pool, especially if the file is paged to disk.  Of course,
the above would depend on things like size of your memory, efficiency
of file system access, cpu speed vs. disk access. Still would be
curious though...

Finally, did you ever post a working version of your script. I have
heard you mention it (and we both discussed the approach several years
ago), but I don't remember seeing any actual code...

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/