Re: [BackupPC-users] merge host from one pool into another pool

If you want to de-duplicate a live filesystem you would need to modify the filesystem code. The problem with deduplication and creating hardlinks/softlinks is that if you write to any one of the deduplicated files, all of its links would also be writted too which is not a good idea...

It would be better to add some functionality to your filesystem of choice to deduplicate files.

I wish that I had the coding skills to start a full deduplcating open source filesystem project.

On Tue, Nov 25, 2008 at 3:31 PM, Rich Rauenzahn <rich AT shroop DOT net> wrote:

Msquared wrote:
I guess the simple question is, then: is there a tool you can run that
will de-dupe and/or re-merge with the pool?

  
How about a linux tool like either of these?   There is a risk of linking two files that are the same, but are semantically different... like an attrib file or something.

[rrauenza@tendo ~]$ rpm -qi hardlink fdupe
Name        : hardlink                     Relocations: (not relocatable)
Version     : 1.0                               Vendor: Fedora Project
Release     : 5.fc8                         Build Date: Wed 22 Aug 2007 11:42:52 PM PDT
Install Date: Sun 16 Dec 2007 10:23:46 PM PST      Build Host: xenbuilder4.fedora.phx.redhat.com
Group       : System Environment/Base       Source RPM: hardlink-1.0-5.fc8.src.rpm
Size        : 10639                            License: GPL+
Signature   : DSA/SHA1, Wed 24 Oct 2007 08:07:11 PM PDT, Key ID b44269d04f2a6fd2
Packager    : Fedora Project
URL         : http://cvs.fedora.redhat.com/viewcvs/devel/hardlink/
Summary     : Create a tree of hardlinks
Description :
hardlink is used to create a tree of hard links.
It's used by kernel installation to dramatically reduce the
amount of diskspace used by each kernel package installed.
package fdupe is not installed

[rrauenza@tendo ~]$ rpm -qi fdupes
Name        : fdupes                       Relocations: (not relocatable)
Version     : 1.40                              Vendor: Fedora Project
Release     : 10.fc8                        Build Date: Wed 28 Nov 2007 12:29:44 PM PST
Install Date: Fri 07 Nov 2008 02:44:54 PM PST      Build Host: xenbuilder1.fedora.redhat.com
Group       : Applications/File             Source RPM: fdupes-1.40-10.fc8.src.rpm
Size        : 23464                            License: MIT
Signature   : DSA/SHA1, Thu 28 Aug 2008 09:08:59 PM PDT, Key ID 62aec3dc6df2196f
Packager    : Fedora Project
URL         : http://netdial.caribe.net/~adrian2/fdupes.html
Summary     : Finds duplicate files in a given set of directories
Description :
FDUPES is a program for identifying duplicate files residing within specified
directories.
[rrauenza@tendo ~]$
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url="">
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/