|
Re: [BackupPC-users] Really slow with rsync, even on incrementals
2011-02-23 15:03:35
John Goerzen <jgoerzen AT complete DOT org> wrote on 02/23/2011
11:39:57 AM:
> Michael Stowe <mstowe <at> chicago.us.mensa.org> writes:
>
> > Another way to put this is that you'd like BackupPC to overcome
> > performance problems caused by your amazingly slow I/O. Not
that I'm
> > arguing against rewriting BackupPC to accomodate slow media,
but I'm
> > certainly not going to bother, since it's certainly not slow
for me.
>
> Well, I don't think that's entirely accurate. BackupPC is on
the order of 10
> times slower than alternatives making full backups after the first.
It is on
> the order of 20-50 times slower than alternatives making incrementals.
This
> referring to the same disk setup.
I think the numbers you're seeing are *very* unusual.
I've got servers that do incrementals of servers that have hundreds
of thousands of files, but only change a few files a day (like 8-10), and
it takes just a few minutes to do an incremental on these systems.
I have other servers that change 50% of the files
on the system, which reflects about 30GB of incrementals per day. I
think the largest file that changes is 7GB (not 25), but we're talking
about large files here. That server only takes 45 minutes to back
up.
I have servers that are also more typical file-servers:
hundreds of thousands of files, several hundred GB of used storage,
and thousands of changed files per day equalling maybe 1-2GB of daily deltas.
These servers take a decently long time to do a full (as in 700 minutes,
or say 12 hours), and a relatively short time to do an incremental (under
2 hours easy).
All of these file servers are plenty powerful (multi-core,
multi-GB of RAM, SAS drives, etc.), but the backup servers are not: these
are 1.5GHz VIA processors (read: SLOW) with 512MB RAM and a single SATA
drive (1TB, IIRC.), and are connected via 100Mb (right: 100, *not*
1,000) Ethernet.
> > Might I suggest that you back up to something
quicker, and then copy it
> > over to your USB drive? (If that's too slow, than I fail
to see how
> > BackupPC can do any better.)
>
> I don't expect that to be worth the expense. In other words,
half of 55 hours
> is still too long.
Again, I think you've got something fundamentally
wrong with your hardware, software or configuration. The results
you are seeing just seem *wrong*.
Timothy J. Massey
------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev _______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
| <Prev in Thread] |
Current Thread |
[Next in Thread>
|
- [BackupPC-users] Really slow with rsync, even on incrementals, John Goerzen
- Re: [BackupPC-users] Really slow with rsync, even on incrementals, Michael Stowe
- Re: [BackupPC-users] Really slow with rsync, even on incrementals, Les Mikesell
- Re: [BackupPC-users] Really slow with rsync, even on incrementals, John Goerzen
- Re: [BackupPC-users] Really slow with rsync, even on incrementals, Les Mikesell
- Re: [BackupPC-users] Really slow with rsync, even on incrementals, John Goerzen
- Re: [BackupPC-users] Really slow with rsync, even on incrementals,
Timothy J Massey <=
Re: [BackupPC-users] Really slow with rsync, even on incrementals, Jeffrey J. Kosowsky
|
|
|
