Re: [Bacula-users] how to debug a job

Hello,

2015-01-23 21:36 GMT+01:00 Josh Fisher <jfisher AT pvct DOT com>:

On 1/23/2015 2:33 PM, Heitor Faria wrote:
>> On 01/23/2015 12:22 PM, Radosław Korzeniewski wrote:
>>
>>> If you are using a vchanger then I guess it is a disk only backup, so why
>>> do you spool data? It is not required but making you backup slow down at
>>> least 2 times then standard job.
>> It's 2 times slower if your spool/despool is strictly sequential and the
>> same size pipes go in and out. If you have 10 clients spooling over a
>> 1Gb/s link, that's roughly 10MB/s/client. Iostat clocks ext4/basic sata
>> drives at around 110MB/s write speed, so despooling is 10 times faster.
>> If only the clients could keep the pipe full, it could be ~10 times
>> faster than the "standard" job. In a purely hypothetical perfectly
>> spherical world of uniform density, obviously, but still: no, 2x slow
>> down is not how it really works.
> Even though, IMHO, spooling disks backup is just "muda" (Japanese Term): http://en.wikipedia.org/wiki/Muda_(Japanese_term)

No. It is not wasteful if it serves a purpose. The goal is to get the
client's part in the job done faster.

You can do a client's part of the backup job faster only when a whole job will fit in spooling area. But it will not make a backup job to finish earlier.

If you have enough space you should consider implementing D2D2T/D strategy.

Fast storage is expensive. A
shared spool area on fast disk gets the clients part done faster,

Only when a whole job will fit into this area. If not then a particular job needs to be stopped for despooling. In the overall the job speed will be as fast as your slow storage. You can make a calculations to check if I'm right or not. I've made this calculations for Dimitri in my other email.

and
then the SD can de-spool to slower (and cheaper) volume file storage
without affecting the client.

Affecting a client is always a matter of read speed and time. If your backup job saturate a IO subsystem, especially disks and its interface or network interface then it is a problem for any service running on this server. For this purpose there is a network bandwidth limitation feature. It can limit utilization level during backup. On the other hand when you backup with snapshots (with limited snapshot space) or offline database backups then you want to finish your backup as soon as possible saturating all your available resources on the client side.

A fast spooling storage will not help you as during despooling your backup is still running holding a database in offline state or snapshot, etc.

Using spooling for disk volumes is exactly
the same reasoning as using it for tape volumes.

No. No, and again No.

Why anyone wants to use data spooling for tapes? Because a tape drive require minimal throughput to avoid shoe-shine of the tape during write. When your are unable to provide a minimal throughput to the tape drive directly from the client you need to buffer (spool) data on the local storage which should be fast enough to provide optimal write speed to the tape drive. So, you have: client throughput < tape drive minimal throughput. And this is a purpose of data spooling.

For disk based storage it is not required to provide a minimal throughput.

best regards

Radosław Korzeniewski
radoslaw AT korzeniewski DOT net

------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users