Re: [BackupPC-users] BackupPC 4.0.0 released
2017-03-04 16:03:24
On 2017-03-04 01:12 PM, Craig Barratt
wrote:
BackupPC community,
I'm happy to announce that BackupPC 4.0.0 has been released
on Github
and SourceForge.
BackupPC 4.0.0 is a significant improvement over 3.x in
terms of performance and storage efficiency. It is backward
compatible with 3.x storage, so it can be used to upgrade an
existing V3 installation as well as for brand new installs.
After installing those two packages, BackupPC 4.0.0 can
be installed from the tar ball with:
tar zxf
BackupPC-4.0.0.tar.gz
cd BackupPC-4.0.0
See the README.md, ChangeLog and doc/BackupPC.html
files for more information. An overview of the major
changes is below.
Thanks for to everyone who provided fixes and reported
issues with the alpha and github versions.
Enjoy!!
Craig
Here's a summary of the most significant changes in V4:
- No use of hardlinks (except temporarily to do atomic
renames). Reference counting is handled at the
application level in a batch manner (hardlinks will
still remain for any legacy V3 backups).
- Backups are stored as "reverse deltas" - the most
recent backup is always filled and older backups are
reconstituted by merging all the deltas starting with
the nearest future filled backup and working backwards.
- This is the opposite of V3 where incrementals are
stored as "forward deltas" to a prior backup (typically
the last full backup or prior lower-level incremental
backup, or the last full in the case of rsync).
- Since the most recent backup is filled,
viewing/restoring that backup (which is the most common
backup used) doesn't require merging any deltas from
other backups.
- The concepts of incr/full backups and unfilled/filled
storage are decoupled. The most recent backup is always
filled. By default, for the remaining backups, full
backups are filled and incremental backups are unfilled,
but that is configurable.
- Uses full-file MD5 digests, which are stored in the
directory attrib files. Each backup directory only
contains an empty attrib file whose name includes its
own MD5 digest, which is used to look up the attrib
file's contents in the pool. In turn, that file
contains the metadata for every file in that directory,
including each files's MD5 digest.
- The Pool layout still supports chains to handle md5
collisions. While collisions can be constructed and are
now well-known, they are highly unlikely in the wild.
Pool files are never renamed or moved, unlike V3.
- Any backup can be deleted (deltas are merged into next
older backup if it is not filled).
- The reverse deltas allow "infinite incrementals" - no
need for a full backup if you are willing to trade speed
for the risk that a file change will not be detected if
the metadata (eg, mtime or size) doesn't change.
- An rsync "full" backup now uses --checksum (instead of
--ignore-times), which is much more efficient on the
server side - the server just needs to check the
full-file checksum computed by the client, together with
the mtime, nlinks, size attributes, to see if the file
has changed. If you want a more conservative approach,
you can change it back to --ignore-times, which requires
the server to send block checksums to the client.
- The use of rsync --checksum allows BackupPC to guess a
potential match anywhere in the pool, even on a
first-time backup. In that case, the usual rsync block
checksums are still exchanged to make sure the complete
file is identical.
- Uses a modified rsync called rsync_bpc (currently
based on rsync-3.0.9) on the server side (in place of
File::RsyncP), with a C code interface to the BackupPC
storage. So the whole data path for rsync is now in
compiled C code, which is much faster than perl.
- Due to the use of rsync-3.X, acls and xattrs are
supported, and many other useful options (but not all)
are supported. Rsync protocol 30 supports the efficient
incremental file list, which significantly improves
memory usage and startup time. It also supports MD5
full-file checksums, which match BackupPC's new digest.
That allows a full-file digest to be checked as easily
as an mtime on the server side.
- Significant portions of the BackupPC code are now
compiled C code in a new module called BackupPC::XS that
is dynamically linked to perl.
- V3 migration: nothing specific is needed. V4 can
browse/view/restore V3 backups. When you install V4, no
changes are made to any V3 backups. If you are upgrading
from V3, be sure to set $Conf{PoolV3Enabled} to 1 so the
old V3 pool is searched for matching files.
- When you install V4, it will notice that the V3 pool
exists. Running configure.pl should
set $Conf{PoolV3Enabled} to 1 in that case, but you
should be sure to check that.
- When a V4 backup is first done,
BackupPC_backupDuplicate is run to duplicate the most
recent V3 backup to create a new V4 backup. A "filled"
view of the most recent V3 backup is used to create a
"filled" V4 backup tree.
- This step could be time consuming, since every file
needs to be read (as a V3 file) and written as a V4
file. However, the V4 pooling code knows about the V3
pool, so it will move the V3 pool file into the V4
pool. So this duplication process doesn't burn a lot
of pool storage space, but every file still needs to
be read (to compute the MD5 digest) and "written"
(really just matching/linking).
- Expiry: all the V3 + V4 backups are considered on a
combined basis for expiry checking.
- On a clean new V4 install, the steps of computing
and checking V3 digests is eliminated.
--
This is great news. Now to figure out how to upgrade my Ubuntu 14.04
box.
|
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
|
|