On Tue, 2008-11-04 at 23:37 +0200, Pasi Kärkkäinen wrote:
> On Tue, Nov 04, 2008 at 10:21:54PM +0100, Ulrich Leodolter wrote:
> >
> > On Tue, 2008-11-04 at 22:38 +0200, Pasi Kärkkäinen wrote:
> > > On Tue, Nov 04, 2008 at 09:02:46PM +0100, Ulrich Leodolter wrote:
> > > > Hi,
> > > >
> > > > Problem: Migrate/Copy jobs from disk pool (DiskBackup)
> > > > to tape (DiskCopy) only get overall speed of 10-20MB/s.
> > > >
> > > > full backup job size varies from 10-50GB.
> > > >
> > > > Pool setup is simple, just one pool for full an incremental backups
> > > > to disk (automatic recycle works good)
> > > >
> > > > Pool {
> > > > Name = DiskBackup
> > > > Pool Type = Backup
> > > > Recycle = yes
> > > > RecyclePool = DiskBackup
> > > > AutoPrune = yes
> > > > Volume Retention = 15 days
> > > > Volume Use Duration = 6 days
> > > > Maximum Volume Bytes = 4G
> > > > Label Format = Backup-
> > > > Next Pool = DiskCopy
> > > > }
> > > >
> > > > DiskCopy pool goes to LTO4 tape device.
> > > > DiskBackup goes to SATA external raid (6T ext3).
> > > >
> > >
> > > Which raid-level your SATA external raid is?
> > >
> >
> > Raid 5
> > >
> > > > concurrency for DiskBackup jobs is 15, jobs are spread
> > > > over DiskBackup Valumes (maybe thats the main problem)
> > > >
> > > > i can read write/continuous on both devices at about 70MB/s
> > > > (SATA is not fast)
> > > >
> > >
> > > So your SATA-raid can do max 70 MB/sec..
> > >
> > > How much your LTO4 can do?
> >
> > 70MB/sec
> >
>
> Have you tried with dd if=/dev/null of=/dev/tape bs=<bigblocksize>
> to get the actual max tape drive performance.. ?
yes, afaik 70MB/sec is normal for LTO4
>
> > >
> > > > i tried spooling to local SAS raid, but overall speed is lower
> > > > than direct writing to tape.
> > > > despooling from SAS raid to Tape runs at Tape maximum spped.
> > > >
> > >
> > > So SAS-RAID to Tape is faster than SATA-RAID to Tape?
> > >
> >
> > about 180MB/s continuous write
> >
>
> Yep. Over 2x compared to SATA-Raid.
>
> So clearly it's the RAID that's limiting your performance.
>
> > > >
> > > >
> > > > I need some Performance tuning tips, maybe:
> > > >
> > > > Limit jobs per Volume in DiskBackup pools?
> > > > Split DiskBackup into DiskFull and DiskIncr pools?
> > > >
> > >
> > > First determine the bottleneck and then work around it..
> > >
> >
> > Thats why i am asking :-)
> >
> > i tried to monitor disk using dstat
> > http://dag.wieers.com/home-made/dstat/
> > i reports 70MB/s read while bacula Copy Job is running.
> >
> > looks like some overhead is in bacula itself:
> >
> > avg iostat while bacula CopyDiskToTape
> >
> > user 9.12
> > system 2.79
> > iowait 17.00
> >
> > avg iostat while EMC Networker (Legato) Clone Disk to Tape
> > (on the same hardware)
> >
> > user 2.14
> > system 2.02
> > iowait 12.84
> >
> > as u can see user and iowait are higher for bacula.
> > total Copy Jobs size is about 800GB (one LTO4 Tape)
> >
> > Networker backup jobs to disk run in parallel (max 10),
> > i dont think backup file sets are continous on disk.
> >
>
> Hmm.. have you adjusted block sizes bacula uses?
>
No, where ?
> "iostat" is also useful tool.. I guess it's part of "sysstat" package.
>
above user,system,iowait are average iostat values.
i have also nice nagios performance data graphs based on iostat.
bacula simply consumes much more time for the same job
"copy all full backups from disk to tape"
maybe this is a bacula architecture problem or my misconfiguration.
i am unsure if larger disk volumes and less jobs per disk volume will
give me performance boost ???
Thx
Ulrich
> >
> > > Btw. could you post your 'copy' job? I'm in the process of trying it out
> > > but
> > > I'm still stuck with trying to implement 'copy only uncopied
> > > jobs'-feature..
> > >
> > > -- Pasi
> > >
> >
> > Job {
> > Name = "CopyDiskToTape"
> > Type = Copy
> > Client = dir-fd
> > Level = Full # must be defined, but is ignored
> > FileSet = "Full Set" # must be defined, but is ignored
> > Pool = DiskBackup
> > Storage = File
> > Messages = Standard
> > Selection Type = PoolUncopiedJobs
> > Maximum Concurrent Jobs = 10
> > # SpoolData = yes
> > }
> >
> >
> > PoolUncopiedJobs is based on an SQLQuery i posted in the devel list.
> > i am running 2.5.17 svn at the server side
> >
>
> Oh nice, I missed that.
>
> I'll try it out:)
>
> -- Pasi
>
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
