|
Re: [Bacula-users] Job transfer rate
2014-10-30 11:19:41
|
On 14-10-30 07:50 AM, Jeff MacDonald
wrote:
Tips/Suggestions?
Jeff.
What is the
content of your backups? Some things (ie thousands of
tiny
files) will
cause a lot of seeks on the machine to be backed up. If
you
aren't using
attribute spooling then each backed up file also causes a
record to be
inserted in to the database, which may take time depending
on your DB
environment.
The
'suggestions' for tuning will be different if you are
backing up a
few dozen 10GB
files versus backing up a million 10kb files.
Its mostly a windows os, with all its sundry smaller files
and a few larger database dumps etc.
I guess what I have to accertain is the slow part getting
the data FROM the servers or the slow part putting the data TO
the storage.
I’m not sure which value the rate is in the job report, or
if rate is somehow encompassing both.
jeff
The job report rate will be the final average rate of the job, it
doesn't know/specify the difference between the 'input' rate and the
'output' rate.
Yep, you're going to need to do some investigation on the storage
side of the VM machine you are backing up, the director itself, the
storage daemon itself (though I'm guessing it is on the same system
as the director for you) and the final storage.
Also it's not quite clear from your description, is the final
storage on a different NAS all together from your VMs? (hoping so!)
What virtualization platform are you running?
Finally the question about attribute spooling is a big one - if you
are backing up a lot of small files and you do not have attribute
spooling turned on, you will have abysmal performance especially if
the director is running on the same disks that you are backing up.
Database writes are (almost) always synchronous writes, meaning the
system will stop and wait for the storage layer to say "yes the data
is ACTUALLY committed to disk" before proceeding. If you are
seeking all over backing up a bunch of small files, then trying to
do a whole ton of tiny DB writes at the same time to the same
spindles your hard drive heads are going to be flying around like
crazy. An array of 7200 RPM disks in any sort of parity RAID
configuration will not be able to handle more than 50-90 random IOPs
(Operations per Second) at best in real life, with a DB write or a
file read counting as an IOP. If you are backing up lots of small
files randomly distributed around the storage you are quite likely
hitting an IOP wall - an IOP to read the file and an IOP to write
the DB record means not more than 25-45 files per second. 4kb files
= 100-180kb/sec and a completely maxed out storage layer.
Even WITH attribute spooling enabled you are still going to be in a
less-than-ideal position since the spooled attributes still need to
be written to the same spindles with the hardware configuration
you've described.
Bryn
|
------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
|
|
