Networker

Re: [Networker] slow san backup performance

2002-12-04 10:32:11
Subject: Re: [Networker] slow san backup performance
From: Robert Maiello <robert.maiello AT MEDEC DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Wed, 4 Dec 2002 10:32:11 -0500
Terry, Brian,

Thanks..  I'll definitely go about getting those settings set to their
max.   In regards to the large incrementals...if no one else here is
having that problem then I'll definitely look elsewhere on the system for
the cause.

Again, thanks for helping a unix guy with these W2k issues..

Robert Maiello
Thomson
Medical Economics

On Tue, 3 Dec 2002 12:55:05 -0500, Narkinsky, Brian
<Brian.Narkinsky AT DEP.STATE.FL DOT US> wrote:

>You need to clear the Networker uses change journal then reselect it.  You
>can then change the settings.
>
>Brian
>
>-----Original Message-----
>From: Terry Lemons [mailto:lemons_terry AT EMC DOT COM]
>Sent: Tuesday, December 03, 2002 12:48 PM
>To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
>Subject: Re: [Networker] slow san backup performance
>
>
>Hi Robert
>
>The incremental backup done after the full backup should definitely NOT look
>like a full backup.  You have something else happening here (maybe virus
>checking software is marking every file as modified, every time you access
>them during a backup)?
>
>The settings shouldn't be grayed out.  Make sure that you have selected a
>volume (from the list presented above the settings) that uses NTFS.
>
>tl
>
>-----Original Message-----
>From: Maiello, Robert [mailto:Robert.Maiello AT medec DOT com]
>Sent: Monday, December 02, 2002 12:02 PM
>To: 'lemons_terry AT emc DOT com'
>Cc: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
>Subject: RE: [Networker] slow san backup performance
>
>
>
>I understand the change journal is not used for a full backup.  That said,
>is it at least zero'd out?   That is, is the incremental backup I do after
>the
>full backup based on the full backup or the incremental from before the
>full backup?    Instead of seeing the journal emptied after a full backup
>I'm seeing all files placed in it...ie  my incremental after a full is just
>as big
>as a full with the change journal turned on.
>
>
>Also for the 2 setting you mentioned;  in my client 6.1.2 they appear ..I
>see
>the max is 2% and 25% but they are greyed out and cannot be set.
>Robert Maiello
>Thomson Medical Economics
>
>
> -----Original Message-----
>From:   lemons_terry AT emc DOT com [ mailto:lemons_terry AT emc DOT com
><mailto:lemons_terry AT emc DOT com> ]
>Sent:   Monday, December 02, 2002 11:09 AM
>To:     robert.maiello AT MEDEC DOT COM
>Cc:     NETWORKER AT LISTMAIL.TEMPLE DOT EDU
>Subject:        RE: [Networker] slow san backup performance
>
>Hi Robert
>
>The Change Journal is only used for non-full backups; a full backup does not
>
>use the Change Journal, and the Change Journal has no affect on how long
>full backups take to run.
>
>If you're not seeing the "the NTFS change journal will be used.." message,
>make sure that you've used the NetWorker Change Journal Manager to set the
>two values to their maximums.
>
>tl
>
>-----Original Message-----
>From: Robert Maiello [ mailto:robert.maiello AT MEDEC DOT COM
><mailto:robert.maiello AT MEDEC DOT COM> ]
>Sent: Monday, December 02, 2002 11:00 AM
>To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
>Subject: Re: [Networker] slow san backup performance
>
>
>Terry,
>
>Could you expand on using the NTFS change journal on W2K.   We enabled
>it on several boxes but have the following problems:
>
>The backup after a full backup (an incremental) appears to be as large
>as a full backup (a giant incremental)...its as if the full backup
>touched all the files and added them to the change journal.   This appears
>to be pretty broken??
>
>The other problem is the level 9 backups we do don't seem to have the
>customary "the NTFS change journal will be used.."  message.
>
>It looks promising but perhaps I have something set wrong??
>
>Robert Maiello
>Thomson Medical Economics
>
>
>On Wed, 27 Nov 2002 10:49:40 -0500, Terry Lemons <lemons_terry AT EMC DOT COM>
>wrote:
>
>>Hi Hanny
>>
>>To make sure that your infrastructure is solid, place several large files
>in
>>a single directory on your storage array; the files can be multiple copies
>>of the same file.  Then, back up just that directory.  Make sure that you
>>have enough data, so that the backup will run for ~5 minutes or longer,
>long
>>enough for you to watch it run.  Put the directory that you are backing up
>>on the same LUN with which you are experiencing poor performance during
>>backups.  If the backup of these large files is fast, using the same
>>environment as your regular backups, then you'll know that the small files
>>are the problem.
>>
>>That said, I have good news.  Since you are using Windows 2000 (and, I'm
>>assuming, the NTFS file system), you can take advantage of NetWorker's
>>support for Change Journal.  This is described beginning on p.592 of the
>>"NetWorker for Windows Administrator's Guide"
>>(
>http://web1.legato.com/infodev/publications/NetWorker/WINDOWS/6.2/winag.pd
><http://web1.legato.com/infodev/publications/NetWorker/WINDOWS/6.2/winag.pd>
>
>f
>>).
>>
>>Others in this mailing list have reported that NetWorker's support for
>>Change Journal doesn't work; I, too, had problems with it not appearing to
>>work.  But, Legato Technical Support advised me to use the maximum values
>>(using the NetWorker Change Journal Manager application) for '% of volume
>>for log file' and % of log for Allocation Delta'.  Once I did this,
>>NetWorker began using the Change Journal for all non-full backups.  The
>>results have been dramatic:  the nightly incremental backup completion
>times
>>of one disk with several hundred thousand small files has decreased from
>>12-21 minutes to 1-3 minutes.  Note that this only helps with non-full
>>backups; full backups do not use the Change Journal.
>>
>>Hope this helps!
>>tl
>>
>>-----Original Message-----
>>From: hanny [ mailto:k0s5 AT YAHOO DOT COM <mailto:k0s5 AT YAHOO DOT COM> ]
>>Sent: Wednesday, November 27, 2002 5:08 AM
>>To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
>>Subject: [Networker] slow san backup performance
>>
>>
>>hi guys..
>>
>>our client recently reported a slow backup performance.
>>i currently run out of idea, hope the list can shed some light...:)
>>
>>environtment
>>1 backup server w2k sp3
>>2 x xeon 700, 2 GB RAM
>>1 Emulex FC8000
>>1 Emulex FC7000
>>1 ATLP6000 w/ 12 DLT7K
>>1 ATLP3000 w/ 6 SDLT
>>
>>8 storage nodes
>>w2k sp3
>>2 x emulex 8000
>>san disk 3 x 130 GB
>>millions of small files, average size 40K/file.
>>storage box: 2 x fc4500 (total 2.4TB) and 1 x hp512 (2.6TB)
>>
>>emulex details:
>>firmware 3.02
>>driver version 2.11a2
>>
>>
>>When the backup start, the backup speed ranging from 300KB/s to 2MB/s
>>But the speed can easily drop to 50KB/s and stays there forever.
>>
>>I guess small files contributes to this problem,
>>but with the speed of only 50KB, there must be something is not correct
>>somewhere.
>>
>>So any of you guys using emulex hba with w2k, please share the firwmare and
>
>>driver version used
>>
>>any other idea deeply appreciate...
>>
>>
>>
>>Network diagram (simplified)
>>
>>                 backup svr
>>         ____________|  |_____________
>>         |                            |
>>       brocade                      brocade
>>    ___|  | |______               ___|  | |______
>>    |      |       |              |     |       |
>>2xFC4500 |      4 SN           1xXP512 |      4 SN
>>           |                            |
>>         ATL P3K                       ATLP6K
>>
>>
>>
>>
>>cheers - han
>>
>>--
>>Note: To sign off this list, send a "signoff" command via email
>>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>> http://listmail.temple.edu/archives/networker.html
><http://listmail.temple.edu/archives/networker.html>  where you can
>>also view and post messages to the list.
>>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>>
>>--
>>Note: To sign off this list, send a "signoff" command via email
>>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>> http://listmail.temple.edu/archives/networker.html
><http://listmail.temple.edu/archives/networker.html>  where you can
>>also view and post messages to the list.
>>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
>--
>Note: To sign off this list, send a "signoff" command via email
>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>http://listmail.temple.edu/archives/networker.html
><http://listmail.temple.edu/archives/networker.html>  where you can
>also view and post messages to the list.
>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
>________________________________________________________________________
>This email has been scanned for all viruses by the MessageLabs SkyScan
>service.
>________________________________________________________________________
>
>
>________________________________________________________________________
>The information contained in this communication may be confidential or
>legally privileged and is intended only for the recipient named above. If
>the reader of this message is not the intended recipient, you are hereby
>notified that any dissemination, distribution or copying of this
>communication or its contents is strictly prohibited. If you have received
>this communication in error, please immediately advise the sender and delete
>the original and any copies from your computer system.
>
>
>
>--
>Note: To sign off this list, send a "signoff" command via email
>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>http://listmail.temple.edu/archives/networker.html where you can
>also view and post messages to the list.
>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
>--
>Note: To sign off this list, send a "signoff" command via email
>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>http://listmail.temple.edu/archives/networker.html where you can
>also view and post messages to the list.
>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

<Prev in Thread] Current Thread [Next in Thread>