ADSM-L

Journaling

2002-12-09 14:06:31
Subject: Journaling
From: Pete Tanenhaus <tanenhau AT US.IBM DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 9 Dec 2002 14:02:41 -0500
Increasing the notification buffer size is a good suggestion.

This buffer is preallocated when the file system monitor for the particular
file system being journaled is started.

If the amount of change activity generated by the file system can't be
processed quickly enough to empty
the buffer it will overflow and the journal must be invalidated (forcing
full incremental backup incremental backup
to be performed before Journal Based Backup can continue).

Note that if the buffer overflows messages are logged to both the journal
errorlog and the NT application eventlog.

The size the buffer needs to be depends on the volume of change activity
generated by the file system
(it really has nothing to do with the size of the file system) and in many
cases the default size of 1 megabyte
(0x100000) may not be sufficient.

All of the testing we've done seems to indicate that a size of between 2
and 3 megabytes (0x200000 to 0x300000) is sufficient for
very large amounts of change activity, but there are certain situations
which will almost always cause overflow problems and force
the journal to be invalidated.

In particular, file system operations which flood the file system with a
large amounts of change activity in a short period of time
will always cause problems.

Deleting, copying, or modifying every object in very large (both in terms
of number of objects and director depth) directory trees
will most likely flood the journal file system monitor with more change
notifications than can be processed before overflowing
the buffer, and thus will force the journal to be invalidated.

One of the limitations of Journal Based Backup is that only a finite amount
of change activity can be tolerated in a short period of time
(it's not so much the volume, it's how quickly it is generated).

Hope this answers your question and that increasing the size of the
notification buffer helps.




Pete Tanenhaus
Tivoli Storage Solutions Software Development
email: tanenhau AT us.ibm DOT com
tieline: 320.8778, external: 607.754.4213

"Those who refuse to challenge authority are condemned to conform to it"

---------------------- Forwarded by Pete Tanenhaus/San Jose/IBM on
12/09/2002 01:44 PM ---------------------------

"Gill, Geoffrey L." <GEOFFREY.L.GILL AT SAIC DOT COM>@VM.MARIST.EDU> on 
12/09/2002
10:46:49 AM

Please respond to "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>

Sent by:    "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>


To:    ADSM-L AT VM.MARIST DOT EDU
cc:
Subject:    Journaling



We have a server with over 4 million files and growing. It's been a real
pain getting journaling to work and I was wondering if anyone else with a
similar size server has this running smoother than I do. I don't seem to
have any trouble with servers that have less than half a million files.

TSM support has recommended I bump this option. What might be a good number
to start with? Are there any other options in the tsmjbbd.ini that need
attention as well?

; Notification Buffer Size
;
NotifyBufferSize=0x00100000

Thanks,
Geoff Gill
TSM Administrator
NT Systems Support Engineer
SAIC
E-Mail:    <mailto:gillg AT saic DOT com> gillg AT saic DOT com
Phone:  (858) 826-4062
 Pager:   (877) 905-7154

<Prev in Thread] Current Thread [Next in Thread>
  • Journaling, Gill, Geoffrey L.
    • Journaling, Pete Tanenhaus <=