Re: [Bacula-users] slow performance copy/migrate disk to tape
2008-11-04 16:25:53
On Tue, 2008-11-04 at 22:38 +0200, Pasi Kärkkäinen wrote:
> On Tue, Nov 04, 2008 at 09:02:46PM +0100, Ulrich Leodolter wrote:
> > Hi,
> >
> > Problem: Migrate/Copy jobs from disk pool (DiskBackup)
> > to tape (DiskCopy) only get overall speed of 10-20MB/s.
> >
> > full backup job size varies from 10-50GB.
> >
> > Pool setup is simple, just one pool for full an incremental backups
> > to disk (automatic recycle works good)
> >
> > Pool {
> > Name = DiskBackup
> > Pool Type = Backup
> > Recycle = yes
> > RecyclePool = DiskBackup
> > AutoPrune = yes
> > Volume Retention = 15 days
> > Volume Use Duration = 6 days
> > Maximum Volume Bytes = 4G
> > Label Format = Backup-
> > Next Pool = DiskCopy
> > }
> >
> > DiskCopy pool goes to LTO4 tape device.
> > DiskBackup goes to SATA external raid (6T ext3).
> >
>
> Which raid-level your SATA external raid is?
>
Raid 5
>
> > concurrency for DiskBackup jobs is 15, jobs are spread
> > over DiskBackup Valumes (maybe thats the main problem)
> >
> > i can read write/continuous on both devices at about 70MB/s
> > (SATA is not fast)
> >
>
> So your SATA-raid can do max 70 MB/sec..
>
> How much your LTO4 can do?
70MB/sec
>
> > i tried spooling to local SAS raid, but overall speed is lower
> > than direct writing to tape.
> > despooling from SAS raid to Tape runs at Tape maximum spped.
> >
>
> So SAS-RAID to Tape is faster than SATA-RAID to Tape?
>
about 180MB/s continuous write
> >
> >
> > I need some Performance tuning tips, maybe:
> >
> > Limit jobs per Volume in DiskBackup pools?
> > Split DiskBackup into DiskFull and DiskIncr pools?
> >
>
> First determine the bottleneck and then work around it..
>
Thats why i am asking :-)
i tried to monitor disk using dstat
http://dag.wieers.com/home-made/dstat/
i reports 70MB/s read while bacula Copy Job is running.
looks like some overhead is in bacula itself:
avg iostat while bacula CopyDiskToTape
user 9.12
system 2.79
iowait 17.00
avg iostat while EMC Networker (Legato) Clone Disk to Tape
(on the same hardware)
user 2.14
system 2.02
iowait 12.84
as u can see user and iowait are higher for bacula.
total Copy Jobs size is about 800GB (one LTO4 Tape)
Networker backup jobs to disk run in parallel (max 10),
i dont think backup file sets are continous on disk.
> Btw. could you post your 'copy' job? I'm in the process of trying it out but
> I'm still stuck with trying to implement 'copy only uncopied jobs'-feature..
>
> -- Pasi
>
Job {
Name = "CopyDiskToTape"
Type = Copy
Client = dir-fd
Level = Full # must be defined, but is ignored
FileSet = "Full Set" # must be defined, but is ignored
Pool = DiskBackup
Storage = File
Messages = Standard
Selection Type = PoolUncopiedJobs
Maximum Concurrent Jobs = 10
# SpoolData = yes
}
PoolUncopiedJobs is based on an SQLQuery i posted in the devel list.
i am running 2.5.17 svn at the server side
Ulrich
--
Ulrich Leodolter <ulrich.leodolter AT obvsg DOT at>
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
|
|