ADSM-L

Re: Groupwise Backup

2006-09-12 18:13:17
Subject: Re: Groupwise Backup
From: Troy Frank <Troy.Frank AT UWMF.WISC DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Tue, 12 Sep 2006 17:10:14 -0500
TSM does not support the use of the /enableGroupwise switch, as it is
the replacement for tsagw, which tsm also didn't support.  It doesn't
stop backups from running, but it caused more errors in my experience.
The /nocachingmode switch is important to do though...had forgotten
about that one.  You can also set it permanently by editing the
sys:/etc/sms/tsa.cfg file, and changing the "Enable Caching" option from
"yes", to "no".


>>> Marco Malgarini <marco AT MALGARINI DOT ORG> 9/12/2006 4:49 PM >>>
One more suggestion:

Have you tried the following TSAFS settings?

Tsafs /enableGroupwise=true this is a TSAFS load switch
And have you tried tsafs /nocachingmode this is an online switch which
we
use as a pre-schedule command.

Our post offices have about 1,500,000 objects and inspection time
alone
takes between 30min to 1 1/2 hours depending of other backups running
on the
same box. i.e. file and print data.




Kind Regards

Marco Malgarini

Malga Consulting
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf
Of Sam
Sheppard
Sent: Wednesday, 13 September 2006 06:58 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] Groupwise Backup

---------------------------- Top of message
----------------------------
>>--> 09-12-06  13:43  S.SHEPPARD     (SHS)    Re: Groupwise Backup

I'll try scaling back the TXNG, but the data is static (snapshot
copy),
so nothing is changing. My understanding of RESOURCEUTILIZATION is
that
you need multiple volumes (filespaces) to use multiple producer
threads
and since we're only backing up the one Groupwise volume I don't think
RESOURCEUTILIZATION will buy us anything.  I'll also try upping the
TCPBUFFSIZE.  I was taking the recommendation from the Performance
Tuning Guide, but I have plenty of memory to play with.
Thanks
Sam Sheppard
San Diego Data Processing Corp.
(858)-581-9668
-----------------------------------------------------------------------`


---------------------------- Top of message
----------------------------
>>--> 09-12-06  13:38  ..NETMAIL     ()     Re: [ADSM-L] Groupwise Ba
Date: Tue, 12 Sep 2006 15:32:11 -0500
From: "Troy Frank" <Troy.Frank AT UWMF.WISC DOT EDU>
Subject: Re: [ADSM-L] Groupwise Backup
To: ADSM-L AT VM.MARIST DOT EDU
_________________________________Top_of_Message_____________________________
____

Couple suggestions...

Server side - groupwise tends to work better with a smaller
TXNGROUPMAX
(ours is 1024).  Since groupwise is a lot of small files, and they can
change rapidly, big TXNGROUPMAX settings can equal a lot of aggregate
rebuilding before going to the server.

Client Side - Set RESOURCEUTILIZATION to 4 or so.  Netware seems to
have issues with going higher than 5 or 6, but 4 seems to work well.
You'll get more filesystem reader/data sender threads.  I've included
some of our client-side dsm.opt settings below...

ResourceUtilization          4
TCPBUFFSIZE   127
TCPWINDOWSIZE 64
TXNBYTELIMIT  25600
LARGECOMmbuffers   Yes

I've read some things that suggest LARGECOMmbuffers on netware is a
completely useless command, but it doesn't seem to hurt anything, so I
haven't taken it out.  Also keep in mind that groupwise backups will
always be slower than most other types of systems.  It's a pretty
worst-case scenario for an incremental backup since it involves a huge
number of very small files.  An ftp transfer doesn't really accurately
tell you anything, since that's only testing speed of reading a
single/few very large files, and filling the network pipe.  Disk
performance will likely be the dominating factor in this kind of
scenario, so I'd look at what kind of raid set you have, how many
spindles, on how many controllers, with what block size, what rpm your
drives are, ect.

>>> SHS AT SDDPC.SANNET DOT GOV 9/12/2006 1:45 PM >>>
We have installed a new TSM server intended to backup about a dozen
Novell Groupwise post offices, totaling around 600GB. These are being
directed to IBM Ultrium-TD3 LTO tapes in a FC-attached Spectralogic
library.  The TSM server is Version 5.3.3.3 running under Solaris 10
(Sunfire V240, 2 1.5G CPUs, 8GB memory).  The clients machines are
dual-processor 3GHz, 3GB memory running the 5.3.4 version of the
Netware
client.  Client/Server connection is over a GigE VLan.  Server
options:

COMMmethod TCPIP
TCPWindowsize 128
BUFPOOLSIZE 128000
EXPINTERVAL 0
SELFTUNEBUFPOOLSIZE YES
TXNGROUPMAX 2048

Client Options:

COMMMETHOD         TCPip
TCPSERVERADDRESS   172.18.16.6
TCPBUFFSIZE        32
TCPWINDOWSIZE      64
TCPPORT            1500
TXNB               2097152
PASSWORDACCESS    GENERATE
PROCESSORUTILIZATION 100
MEMORYEFFICIENTBACKUP NO

Initial test backup of a small (9GB) PO showed a throughput of around
20GB/hour.  Subsequent tests have improved to 25-29GB/hour after
upping
the TXNG and PROCESSORUTILIZATION parms, but this still seems awfully
slow for what, to us, seems like a pretty beefy system.

The data resides on a NetApp FAS device and we actually backup a
snapshot of the PO to avoid having to take Groupwise down. For reasons
I
won't go into, NDMP was removed as an option when putting this
configuration together.

Stats show 60%+ Comm. Wait, so I'm assuming this is a client-side or
network issue, but I'm at a loss as to what to try next.  FTP tests
from
the client show excellent throughput (200+GB/hour), so I don't believe
it's a network issue.  We're going to up the client-side TCPW to 128,
but I'm not optomistic.  Can anyone else out there give me their
experiences, performance-wise, with large Groupwise backups and any
hints at how to increase this throughput?  I'm beginning to think it
may be limitations of the Netware OS/client.  We're hoping to get near
40GB/hour to make our window.

Thanks in advance,
Sam Sheppard
San Diego Data Processing Corp.
(858)-581-9668

Confidentiality Notice follows:

The information in this message (and the documents attached to it, if
any)
is confidential and may be legally privileged. It is intended solely
for
the addressee. Access to this message by anyone else is unauthorized.
If
you are not the intended recipient, any disclosure, copying,
distribution
or any action taken, or omitted to be taken in reliance on it is
prohibited and may be unlawful. If you have received this message in
error, please delete all electronic copies of this message (and the
documents attached to it, if any), destroy any hard copies you may
have
created and notify me immediately by replying to this email. Thank
you.

-----------------------------------------------------------------------`

Confidentiality Notice follows:

The information in this message (and the documents attached to it, if any)
is confidential and may be legally privileged. It is intended solely for
the addressee. Access to this message by anyone else is unauthorized. If
you are not the intended recipient, any disclosure, copying, distribution
or any action taken, or omitted to be taken in reliance on it is
prohibited and may be unlawful. If you have received this message in
error, please delete all electronic copies of this message (and the
documents attached to it, if any), destroy any hard copies you may have
created and notify me immediately by replying to this email. Thank you.

<Prev in Thread] Current Thread [Next in Thread>