i think you missed the point on sending zfs snapshots.  *if* the remote 
filesystem is in sync with the snapshot you have then it should work, 
but what if that is not the case?  if the remote filesystem is not 
synced with any specific snapshot, then there is no base level target to 
snapshot from so the new smaller snapshot cannot be created because 
common sync point is not known.  you would first have to make sure that 
the remote filesystem matched a snapshot on the local system and then 
proceed if it does, if it does not then what?  run a 100+GB complete 
sync over a 1.5Mb link?  there is the problem, what the remote system 
falls out of sync because of a deleted snapshot or missed sync?  how 
will you check to see if the remote filesystem and local snapshot are 
the same?

I think the intent of the zfs send is just to migrate a filesystem from 
one system to another.  I know that this is along those lines but it is 
just different enough that i think zfs is a bit lacking for remote 
syncs.  I'm sure it can be done but I'm not convinced that it would be 
very reliable or headache free because of a big potential for a very big 
bandwidth penalty when the systems drift out of sync.

as far as rsync in concerned, i think that you need to have a ton of ram 
and a fast CPU to make large fileset transfers work, which I have.  I 
doubt a 1GB p3 1Ghz is going to cut it.

I have another idea I will post in a new message as I think it is off of 
this subject, please read on about BITTORRENT.  thanks

Les Mikesell wrote:
> dan wrote:
>> the problem with zfs snapshots is that you will have to be very 
>> careful about organizing the data.  you are taking a snapshot of a 
>> live filesystem but what does it depend on? 
>
> If I understand the process right, you make a zfs snapshot first, so 
> what you send isn't from a live filesystem.  Then you send that and at 
> the other end receive it into a snapshot that you could promote to the 
> filesystem if you wanted (but you don't have to).
>
>> and when you send it to the other machine is that dependancy met?  
>> what if you miss sending a snapshot?
>
> The only thing I'd expect to be a problem would be if you failed 
> partway through the send.  But, you should be able to snapshot, 
> receive to that, and revert on a failure.
>
>> The problem here is automating the process.
>
> Someone must have done this already.  If it is sent over ssh, you 
> should know if it completes or not.
>
>> I have toyed with the idea of doing a script that would do an md5 and 
>> timestamp on each file, write that to a log and then do the same on 
>> the remote system.  then compare the two systems and use rsync on 
>> each item.
>
> I don't think you need any of that - just snapshot and retry until the 
> send of that snapshot succeeds.
>
>> I am currently syncing over 240GB with rsync without issues.  the 
>> sync takes about 1-1.5 hours with most of that time being spent 
>> making the file list.  this does work well, you just have to be 
>> patient and avoid the temptation to kill rsync because you think it 
>> is dead.
>
> I've let rsync run for days and never got the first copy completed.
>

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/