I am listing the observations and complaints of mine after using BackupPC for 5
months. My system is running Ubuntu Hardy and has been kept up with the latest
updates and patches. I upgraded from 3.0.0 to 3.1.0 within my first few months
and have kept this computer solely a BackupPC machine. I've since gone through
a few hard drive configurations which I will list here:
- 3x640GB in hardware RAID3 using an IDE RAID card with SATA ports
- 3x250GB in Linux software RAID5 using mdadm and onboard SATA ports
- 1x2TB using onboard SATA ports
All of those configurations have assured me it is not the speed of the drives
themselves which have been giving me any issues. The machine itself has a
3.4GHz P4HT which I have equally tested with and without HyperThreading to no
apparent change in usage. The Ethernet NIC is a gigabit broadcom one, and I
have both switched the cables the switches with no apparent change in
performance as far as I can remember.
Through all my reading of the documentation and many articles about BackupPC
which I found using Google, I was able to reconfigure my configuration files to
fit the needs of the machines I had and their backup schedules. I was
constantly tweaking and changing setting files, but I want to note, I did not
see a change in performance, just less error messages such as excluding /dev
for instance.
Please feel free, from here on in, to question or assert my obervations and
complaints so that we may all get a well-rounded understanding of the situation.
./01\.
The first thing I noticed when using the program was that the defaults seemed a
bit extreme for my system. I began my setup with two always-on Windows machines
configured with Samba. I let one full backup go one night and the other the
next. From here, I noticed the load of my computer consistently at or over 4,
and I don't think I've ever seen the processor usage go under 100% unless
BackupPC isn't doing anything. Either way, these backups seemed to go very
smoothly and did not cause me any problems after the initial full backups.
I did have one very pertinent issue, the speed of the full backups and
incrementals never went over 10MB/s. On gigabit switches with all
gigabit-linked computers, this was very strange. The Windows machines could
transfer anywhere from 50-75MB/s to each other so I figured something else
might be wrong. I understand many small files are being transferred, but I
would expect the average to be higher when using Samba over Gigabit.
./02\.
I later added in a few always-on Linux machines configured with Rsyncd. This is
where I started noticing some things that were fishy. One of those machines was
hooked up to the network using a USB Wi-Fi card and the other an Intel Gigabit
Ethernet adapter. Unlike common sense would show, the computer on WLAN had a
backup speed in MB/s was faster than that of the computer on LAN. Both machines
had similar setups and both had their own public IP addresses hosting web pages
publicly using a different interface than the one for backup purposes.
The one main difference in the two setups was the operating system. Both were
on Ubuntu, but one had been consistently upgraded so it was on Karmic (WLAN)
while the other was on Hardy (LAN). Either way, this should not have affected
much of anything. The LAN computer also has twice as much RAM and a slightly
faster processor. Both had IDE drives at similar speeds. I can only think the
speed difference might have been caused by the LAN computer having 4 more GB of
data, possibly many small files, than the WLAN machine which had only 2.7GB in
use at the time. but I cannot be sure.
./03\.
I eventually ended up with 18 machines configured in BackupPC with more being
added as I configure them. In those 18 are a few websites of mine for which the
servers are remote. Those website I am backing up using the Rsync
configuration. The Rsync configuration with these websites works exactly as it
should. One of the sites had 11GB of data and while that took forever to
download on my connection, incremental updates are speedy.
The full updates are speedy too. Backup number 61 on this website took only 6.4
minutes for 2.5GB of files. This is because those 2.5GB were already backed up
in the last full and therefore no longer needed to be redownloaded. It is my
understanding that this is how BackupPC is supposed to work. While the very
first full took around 160 minutes, the second was quite a huge difference,
only taking a bit more than three times as long as the previous incrementals.
./04\.
After some time, I noticed my Samba backups were missing files. I had begun to
need older deleted files and noticed they were not backed up in the last full.
It appeared as though BackupPC would get to a point and just give up but make
sure to backup the root folder list first at least. There were times where
certain folders were gone but the rest adjacent were filled. This last sentence
may be wrong; I am judging based on my memory as those backups have already
been cleaned.
./05\.
It was at this point I decided it was time to setup Rsyncd for all my Windows
machines too and then the majority of problems all became readily apparent.
Whereas I could backup my laptops in under 15 minutes each day seemingly no
matter what I downloaded or changed, backups began taking in excess of 40
minutes even up to 2 hours. Sure, everything I wanted to be backed up did get
backed up, but it was becoming slower and slower as the days progressed. I
noticed each incremental seemed to take longer than the one before it and that
the two Linux boxes began to take longer and longer to backup as well. I even
had a small embedded version of FreeBSD running solely in RAM on a machine and
noticed even it taking almost up to 2 minutes to backup whereas it'd always
take 0.4 to 0.6 minutes.
The strain of Rsyncd started to really wain on the machine. It's now to the
point where backups take up to 4 hours for most machines, it seems as though
the CPU usage never changes from 100%, and laptops need to be left on overnight
to backup at all. It's really annoying to constantly get administrative e-mails
that certain machines weren't backed up because they are mobile machines and
Rsyncd just isn't speedy enough.
This never seemed to be a problem before until I converted all my Windows
machines by the sheer need to have all things backed up which should have been
backed up in the last full. I've been using DeltaCopy but I did update the DLLs
and executable files. I even once tried the BackupPC Cygwin files but that
ended up taking 2-3 times as long as the newer files I use.
./06\.
The logs seem to suck really bad. I can't seem to find out which new files are
being created and what's making the biggest impact on each backup operation
every night. I'd really like to know where the 700MB came from even if I wasn't
at the machine. It's one thing to backup the changes on my laptop, but there's
no way I downloaded or changed 300MB-worth of new files all which I had to
backup. My understanding is that these are probably a lot of text files because
the compression ratio averages around 66% or more, but I still have to transfer
those files over the network as slowly as 0.06MB/s can be.
What's worse, no matter how much more stuff I put in my excludes, more data
seems to be downloaded. The growth of the pool seems to have leveled with old
incrementals going away and new ones coming in, but it was at a point where
there was at least 1GB of growth a night from when it was a 160GB pool to a
210GB pool when it began jumping half-dozens of gigabytes during the period of
a week.
Even with only using the web browser on my laptop there was over a gigabyte of
information last backed up with 950MB of it new files. The laptop is only using
22GB of space and if there really were 400MB of new files a night, I'd surely
run out of room on this drive fast. I've done my best to pinpoint the issue,
but no where do I see more than 100MB of changes in total so I'm lost as to
what changes are actually being made and if they're necessary. Blocking all
*.edb files should really have lowered the amount of new information being
backed up. I have disabled indexing on all my machines and have no drives with
compressed data. I have a decent list of excludes and try to get as much as I
can as a just-in-case measure, but it looks as if there is a huge flaw here I
am completely missing out on.
I wish I could figure this out. Everything I'm seeing is attributed to Rsyncd.
Samba was so fast and Rsync seems to work just fine but Rsyncd is entirely
botched. All of my machines show a high amount of CPU usage whenever BackupPC
beings an Rsyncd backup anywhere from 33% on my most-powerful rig (a quad-core)
to 95-100% on my netbook.
Tonight, to do a final test, I've set 8 backups to go in tandem. Four seemed to
have dropped off very quickly leaving me with four final machines to watch.
Through writing this post, not one of those four has finished. Full backups
seem to average 1MB/s instead of at least 3MB/s like before. I recommend people
write in explaining this phenomenon so there may be some resolution. I've done
far too much testing to just let all of this slide. I'd really like for my
machines to actually backup, not sit there and waste battery and use up CPU
only for the backup to fail because I finished using the machine and 4 hours
hadn't passed yet.
+----------------------------------------------------------------------
|This was sent by Saturn2888 AT gmail DOT com via Backup Central.
|Forward SPAM to abuse AT backupcentral DOT com.
+----------------------------------------------------------------------
------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
