BackupPC-users

Re: [BackupPC-users] Concrete proposal for feature extension for pooling

2010-03-14 14:35:54
Subject: Re: [BackupPC-users] Concrete proposal for feature extension for pooling
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Sun, 14 Mar 2010 13:34:02 -0500
Saturn2888 wrote:
> I'm still confused on what you meant about the migration system and the 
> different digests available. I'd suggest some kind of converter if you are 
> allowing us to change the digest as that could cause a lot of issues and some 
> people might do it the md5 way and then get a hardware-based SHA processor 
> and want to change it for performance reasons.

Old files would only need to have the digest recomputed once if you are still 
backing them up - and it would be done with the new method.

> To me, BackupPC's compression is much better for storage reasons than lzjb, 
> but I'd like to see some hard numbers before anything comes around. The fact 
> that there are multiple pools based on compressed data really does this in. 
> There has to be a way to move backups between compressed and uncompressed and 
> more/less compressed pools or else the benefit of filesystem migration or 
> features would be non-existent.

The numbers will vary wildly with the data. Large growing logfiles or databases 
with small changes will almost certainly be handled much better by block-level 
de-dup.  Widely distributed copies of the same file won't matter and might have 
less overhead with file level handling.

> Frankly, I'd love to keep my backups forever, but there's no great way to do 
> this yet. There's really no way to synthesize old fulls with the incrementals 
> up to the next full so you will end up losing data in the process. Sure, it's 
> nice to see the pool size go down for a change, but it's also nice to be able 
> to say "here's everything and now I've fused it in a package. Maybe have it 
> so just the way backups are viewed, there is a way to access them from full 
> to full such as the full and incrementals from 1-19-10 to 2-18-10, and then 
> the full and incrementals from 2-19-10 to 3-18-10. I don't know how best to 
> put it.

I'm not sure I understand this.  You either have room to keep them or you 
don't. 
     That's not going to change, although I suppose making incrementals 
backwards deltas from the more current full might help.  But, without a good 
method for content searches the backuppc format probably isn't a good way to 
store things where you might want to find any version of changing files 
forever. 
  You should probably use a version control system for that sort of thing and 
just keep the current version of the repository backed up since it will contain 
the history in a more accessible form.

> I've begun moving all of my Windows machines over to Rsyncd and because of 
> the emulation of GNU using Cygwin, I'd have to say the performance is far 
> below that of Samba on so many fronts. I have had more issues using Rsyncd in 
> Windows with BackupPC than I ever had with Samba. My question is, with 4.x, 
> is the FileSys::SmbClient support going to be added in? If so, what will be 
> gained by using it instead of the standard SmbClient approach?

For most people, the performance of any single client isn't critical.  Rsync 
will do much better over remote connections with lower bandwidth.  And 
performance will likely improve if the backuppc version is updated to match the 
rsync 3.x scheme where it doesn't have to send the whole directory tree before 
starting.

> Are there plans to maybe create the BackupPC client again (I would love this 
> idea)?

I'm not sure it is necessary with the vss schemes that let you access open 
files.  But I always thought the right way to do this if you want one would be 
to teach backuppc to talk to the bacula agent.

> What about adding support for Unison or creating a way to make Unison 
> worthwhile? Frankly, I don't think Unison is very feature-worthy at the 
> moment, but I'm sure if people already had Unison servers, having BackupPC 
> manage those with its snapshop methodology would be greatly appreciated 
> without having to setup Rsyncd on the server.

Unison is about tracking changes on either side - not something that usually 
happens with backups.

> Is there any hope for customization of Rsyncd such as adding in rsync -z 
> compression for wireless or VPN'd clients, allowing passwords to be sent 
> encrypted somehow, or for connecting to the Rsyncd client over SSH? If 
> there's no benefit to doing that, then oops on my part.

You can use ssh port-forwarding to connect to rsyncd - and enable compression 
in 
ssh.

> What are the changes, if any, for adding an array of IPs and hostnames and 
> priorities on each and even being able to specify if a machine is on IP 
> 12.34.56.78, that means it's not on the local network and will probably have 
> a slower connection to be soft to random disconnections or fluctuations in 
> connection speed.

If you have enough of these to matter, you should probaly set up some sort of 
dynamic DNS to hand out the best address for any sort of connection.

> As a side note, I'd like to say I don't condone of using the terminology 
> WinXX for a number of reasons:
> 1. Unnecessary: Win or Windows machines makes more sense than WinXX
> 2. Redundant: All the Windows up to Windows Vista were XX so by saying WinXX, 
> that is not necessarily specifying anything except the exclusion of 3.11 and 
> below versions which are probably out of the scope of the documentation 
> anyway.
> 3. Exclusionary: Windows Vista and Windows 7 do not abide by the XX 
> specification making it seem like it's only Win95 to WinXP that is supported 
> by BackupPC.
> 4. Confusing: Not everyone would understand what WinXX means because they 
> might think it's a subset of WinXP, a random typo, or a concrete exclusion of 
> the Vista+ line of Windows.

So what is the correct way to refer to something that is windows-specific and 
may need specific app versions to match different version of windows?

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com



------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/