Networker

Re: [Networker] backup mail server very slow (linux)

2004-04-29 09:25:35
Subject: Re: [Networker] backup mail server very slow (linux)
From: Scott Russell <lnxgeek AT US.IBM DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Thu, 29 Apr 2004 09:25:15 -0400
> >> David De Maeyer wrote:
> >>
> >>> Hi all!
> >>>
> >>> We are facing a performance issue when taking a full backup of our
> >>> mail
> >>> server (>7000 users, 350 GB of data, running Debian linux).
> >>>
> >>> The backup server has no problem with the other linux clients. It is
> >>> writing at around 100 KB/s... which is far from the 2 to 8 MB/s we are
> >>> used to have on the other clients.
> >>>
> >>> It takes forever to backup all the user directories! And we simply
> >>> can't find where the bottleneck is... just that there are a lot of I/O
> >>> on the RAID on which these directories are stored.
> >>>
> >>> Would it help to increase/decrease the parallelism for the client
> >>> and/or other clients (smaller filesystems)?

As others have indicated you have two points working against you:

1) Lots of small files can slow down networker
2) Your mail spool partition is large and changing the client
   parallelism alone won't help

Not much can be done about the first problem unless you're willing to
start doing raw partition backups or other similar techniques. These
were just recently discussed on the list.

Chances are you can work around the problem by increasing client
parallelism AND splitting up the data into smaller save sets instead of
using the All saveset.

Assuming for this example your mailspool directory is hashed:

        /var/mail/spool/a/
        /var/mail/spool/b/
        /var/mail/spool/c/
        ...
        /var/mail/spool/z/

and your partitions are setup as:

        /boot
        /
        /var
        /home

You could define a client saveset like:

        /
        /boot
        /home
        /var
        /var/mail/spool/a
        /var/mail/spool/b
        ...
        /var/mail/spool/z

The key is to setup a .nsr file in /var/mail/spool/.nsr that
will skip all the hashed mailspool dirs you are defining as save sets.
The net result is you divide up your large mail spool partition into
multiple save streams.

--
Scott Russell <lnxgeek AT us.ibm DOT com>
Linux Technology Center System Admin

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=