|
Re: [BackupPC-users] high load and stuck processes
2010-03-05 12:13:57
If you are using EXT3 or XFS then I suggest you use an external journal.
get yourself a small SSD or a small 15RPM disk. You could use a regular disk if you like but the faster the better.
(EXT3)Destroy the journal and re-create it on the extra disk.
#unmount the backuppc disk
#on the journal device
mke2fs -O journal_dev -L name_label /dev/journal_disk
#drop old journal
tune2fs -O ^has_journal /dev/current_disk
#recreate the journal
tune2fs -o journal_data -j -J device=LABEL=name_label /dev/current_disk
-or-
tune2fs -o journal_data -j -J device=/dev/journal_disk /dev/current_disk
#remount the disk
#You can add a directory index to the filesystem for a small gain
tune2fs -O dir_index /dev/current_disk
#also, mount with noatime. (/etc/fstab)
/dev/sdb1 /share ext3 defaults,noatime,errors=remount-ro 0 1
The external journal will cut your I/O load on the disk/disk set in half because the filesystem no longer writes the journal on each transaction to that drive. It's a small amount of data but it still requires a disk seek which is what hits the most for many small files (backuppc)
On Fri, Mar 5, 2010 at 7:01 AM, Josh Malone <jmalone AT nrao DOT edu> wrote:
> It's hard to judge; but basically if there are a lot of processes
waiting
> for I/O (a 'D' state in 'top'); try cutting down the number of
concurrent
> backups. You'll have to judge for yourself what the best number for you
is.
> It may be that things work fastest when there's a certain amount of disk
> contention; but no more and no less.
Also - you need a good filesystem to handle lots (or even not so many) of
backups. I reently switched from EXT3 to EXT4 and saw on order of magnitude
(I kid you not, 10+ hours to 1) reduction in the backup time and system
load. Unfortunately, I think this introduced some problems in the RHEL5
ext4 code so I also switched from 32-bit RHEL5 to 64-bit -- that seems to
have cleared up the problems.
-Josh
--
--------------------------------------------------------
Joshua Malone Systems Administrator
(jmalone AT nrao DOT edu) NRAO Charlottesville
434-296-0263 www.cv.nrao.edu
434-249-5699 (mobile)
BOFH excuse #202:
kernel panic: write-only-memory (/dev/wom0) capacity
exceeded.
--------------------------------------------------------
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev _______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
|
|
