I'm trying to use Bacula to do daily backups of data stored in iSCSI LUNs on a
NetApp filer, using NetApp snapshots to ensure consistency. The hosts to be
backed up have dual Gigabit Ethernet connections to the NetApp. The backup
host consists of:
- a desktop-class (32-bit, 2.4GHz) machine with a single local SATA drive
- an Overland Storage autochanger with room for 12 LTO-4 tapes
- a built-in Fast Ethernet adapter (3com 3c509) and an add-in Gigabit Ethernet
adapter (Linksys rev 10)
- running Ubuntu G server and kernel 2.6.22; Bacula is storing its catalog in
a local Postgres database
One issue we've struggled with is speed. With the GB adapter, reading files
from a snapshot via iSCSI, we were consistently getting less than 2MByte/sec,
sometimes as low as 300kbyte/sec. Yesterday we switched to the 100Mbit
adapter, and were sometimes able to almost max it out during a full backup
(network usage of 10 to 11 MByte/sec on the Fast Ethernet adapter), but it
also slowed down sometimes: it took 25 minutes to back up a 22GB LUN with 7GB
of files, and it took 25 minutes to back up a 6GB LUN with 1.1GB of files
(yes, almost exactly the same amount of total time).
I recently did dd to a raw tape and got a speed of at least 17MByte/sec. The
local drive seems to have a write speed of about 7Mbyte/sec, so pooling to
local disk is not an option. On our faster servers with dual server-class
Gigabit Ethernet adapters, I can get burst read speeds of 40 to 70
Mbyte/sec.
We'd also like our tape-rotation policy, for at least some of our tapes, to
mirror as closely as possible what we do for our existing servers with local
tape drives: daily tape rotation in a two-week cycle, with tapes written at
night and taken off-site for one week starting the day after they're written.
That gives us an 18-hour window in which to write the tapes, and we should be
able to fill an 800-GB tape in 17 hours 46 minutes ( 800e8 / 1.25e7 / 3600 =
17.77 ) at Fast Ethernet speed. We probably have less data than that to back
up; in fact, if we keep our other current tape drives and don't back
up /usr/portage or similar directories anywhere, we probably have less than
400GB. Therefore, I think we should do a full backup each day; perhaps even
a full backup of the first snapshot and incremental backups for later
snapshots that same day. Is that reasonable?
Is it possible to initiate an incremental backup that would store all changes
against the contents of a certain medium? (Say tape 5 is in the drive today
and has a 380GB full backup and 6 20-GB incremental backups going back 3
months. File /foo/bar/xxx changed monday and tuesday, so the newest copy is
on the tuesday tape; but write a copy to the friday tape as well.)
Has anyone seen similar speed problems with a NetApp filer, or another device
that serves up snapshots of iSCSI or FCP LUNs, and solved them?
Supposing that round-trip-time over the network or disk seek latency on the
NetApp is the problem, could we solve it by running multiple parallel backup
jobs to the same tape (without spooling)?
How can we initiate an external script from Bacula that would do all the
snapshots and mount them before any backup job runs; or would we have to do
that kind of thing from cron?
It took about 5 minutes to enter the "select files" phase when doing a restore
of a backup with 7 GB of data and 128000 files. Does that mean that if we
made one big backup job over all hosts with 700 GB of data, it would take 8
hours to enter the "select files" phase?
--
David Lee Lambert ... Software Developer
Cell phone: +1 586-873-8813 ; alt. email <dlambert AT bmtcarhaul DOT com> or
<lamber45 AT msu DOT edu>
GPG key at http://www.lmert.com/keyring.txt
pgpDw08A3HsiN.pgp
Description: PGP signature
------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you. Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ _______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
