Well, I guess I really stepped in it... :-)
Journal based backup is primarily targeted for cases where there are lots
(hundreds of thousands or millions) of files with a small percent of
changed files. In these cases, the backup operation might take hours to
run, but only spend a handful of minutes actually backing up data. The
rest of the time is spent doing the following:
- Downloading inventory from the server
- Scanning the file system for new/changed/deleted files
Journal based backup is intended to eliminate both of the above items --
not just the inventory download part. As an added benefit, it can help
work around out-of-memory issues on 32-bit Windows systems (this is not an
issue for 64-bit), notwithstanding the times when a full backup is needed
again.
While maintaining a copy of the backup inventory metadata on the local
file system is interesting, it is not immune to many of the same issues
that face the journal. If the metadata becomes corrupt, or the disk runs
out of space, or someone inadvertently deletes it, then you would need to
fall back to the full incremental backup process to repopulate the local
inventory. Considering (a) out-of-memory issue, (b) download time, and (c)
file system scan time, is it only (b) that is an issue in your shop?
The MEMORYEFFICIENTBACKUP=DISKCACHEMETHOD is an alternative when faced
with the out-of-memory issue. This downloads the inventory to local disk
(rather than virtual memory). It helps with the memory issue, but does not
help the inventory download time.
Best regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: storman AT us.ibm DOT com
IBM Tivoli Storage Manager support web page:
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.html
The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.
"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 07/09/2008
10:03:21 AM:
> Forgive me here, but I think whoever is setting this up is going
> down the wrong path. It's getting very irritating to say the least
> and I share the frustration others are having with IBM's apparent
> inability to get this right.
>
> The "Journaling system" should not be "actively" monitoring the file
> system anyway. It should only be a means of tracking what was
> backed up so that TSM doesn't have to download the entire inventory
> from the server when it does a backup.
>
> If we take this approach, then it doesn't matter whether the service
> is left running or not. If the journal, or record, was saved when
> the backup finished including only what was backed up and all of the
> pertinent data needed to make sure files have not changed when the
> next backup starts.
>
> You see the problem here is that if the Journal db gets deleted
> every time you reboot the server then you have to do a complete
> incremental, and from what I have seen a selective doesn't cut it.
> That means I have to delete the file space from TSM to get an
> incremental to complete when I have millions of small files. This
> is just crappy, and I can't think of a better way to put it.
>
> I think the Devs should either fix the journaling system, or fix TSM
> so that it doesn't have to download 4 GB of inventory to do a backup
> if you have millions of small files.
>
> See Ya'
> Howard
>
>
> > Yes and no.
> >
> > For the case where you use PreserveDbOnExit=1 and the journal service
> > is
> > shut down for some period of time, the behavior will remain unchanged.
> > Use
> > at your own risk. If the journal service is down, it cannot
effectively
> > monitor file system changes so there is no known solution for this.
> >
> > For the journal corruption issue, we are currently targeting an
> > enhancement that allows the journal service to know whether the
> > database
> > was closed cleanly (by a clean journal service shutdown). Upon
restart,
> > if
> > the journal service does not detect the "I was closed cleanly" flag in
> > the
> > journal database, it will effectively ignore PreserveDbOnExit=1 and
> > reset
> > the journal database, as if PreserveDbOnExit=0 was in effect. This
> > would
> > be in a future release of the product. Note that this does not
> > represent a
> > formal announcement or commitment, so this is subject to change.
> >
> > A THEORETICALLY cleaner solution could include implementation of of a
> > journal database with transactional capabilities, including rollback,
> > more
> > thorough structural integrity checking, etc. Even in a rollback
> > scenario,
> > in-flight file system changes would be lost (like the case where you
> > use
> > PreserveDbOnExit=1 and journal service is down for some period of
> > time),
> > leaving a less than perfect solution. But this is all hypothetical,
and
> > there are no plans in place.
> >
> > Best regards,
> >
> > Andy
> >
> > Andy Raibeck
> > IBM Software Group
> > Tivoli Storage Manager Client Product Development
> > Level 3 Team Lead
> > Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
> > Internet e-mail: storman AT us.ibm DOT com
> >
> > IBM Tivoli Storage Manager support web page:
> >
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageMa
> > nager.html
> >
> > The only dumb question is the one that goes unasked.
> > The command line is your friend.
> > "Good enough" is the enemy of excellence.
> >
> > "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 07/09/2008
> > 09:02:05 AM:
> >
> > > The way the downtime for this particular machine goes, there should
> > be
> > no
> > > file system changes occurring.
> > >
> > > Sooo...with all of these issues with jbb, are they being worked
> > > on/fine-tuned?
> > >
> > > -----Original Message-----
> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On
Behalf
> > Of
> > > Andrew Raibeck
> > > Sent: Wednesday, July 09, 2008 10:40 AM
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > > Subject: Re: [ADSM-L] Journaling database retaining upon reboot?
> > >
> > > While the journal service is down, any file system changes are not
> > > tracked. If a file changes while the journal service is down, it
will
> > not
> > > be backed up until either another full (non-journaled) backup runs,
> > OR
> > the
> > > file changes again and the journal service is able to track it at
> > that
> > > time.
> > >
> > > If the journal service is not shut down cleanly (e.g., a cluster
> > failover
> > > in a cluster setup, or something else that causes the journal to not
> > shut
> > > down cleanly), the journal database might be in an inconsistent
> > > ("corrupt") state, after which behavior is unpredictable.
> > "Unpredictable"
> > > could run the gamut of files not getting backed up when they should
> > or a
> >
> > > journal service crash when the corruption is encountered.
> > >
> > > So PreserveDbOnExit=1 should only be used when you know you can stop
> > the
> >
> > > journal service cleanly and you expect it to start shortly
thereafter
> > > (like in a controlled reboot scenario).
> > >
> > > Best regards,
> > >
> > > Andy
> > >
> > > Andy Raibeck
> > > IBM Software Group
> > > Tivoli Storage Manager Client Product Development
> > > Level 3 Team Lead
> > > Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
> > > Internet e-mail: storman AT us.ibm DOT com
> > >
> > > IBM Tivoli Storage Manager support web page:
> > >
> >
http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageMa
> > nager.
> > > html
> > >
> > > The only dumb question is the one that goes unasked.
> > > The command line is your friend.
> > > "Good enough" is the enemy of excellence.
> > >
> > > "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on
> > > 07/09/2008
> > > 08:09:54 AM:
> > >
> > > > Yes, that's it. You set it to 1 if you want it to retain/preserve
> > after
> > > going
> > > > offline.
> > > >
> > > > I wonder why IBM recommended I keep it to zero in my situation?
> > > >
> > > > -----Original Message-----
> > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On
> > Behalf
> > Of
> > > > Erwann Simon
> > > > Sent: Wednesday, July 09, 2008 9:38 AM
> > > > To: ADSM-L AT VM.MARIST DOT EDU
> > > > Subject: Re: [ADSM-L] Journaling database retaining upon reboot?
> > > >
> > > > Hi Charles,
> > > >
> > > > Look at the PreserveDbOnExit parameter of the tsmjbbd.ini
> > configuration
> > > > file.
> > > >
> > > > As a general reference about JBB, read the FAQ :
> > > > http://www-1.ibm.com/support/docview.wss?uid=swg21155524
> > > >
> > > > --
> > > > Best regards / Cordialement / مع تحياتي
> > > > Erwann SIMON
> > > >
> > > > Bell, Charles (Chip) a écrit :
> > > > > I have a machine that has 36 NTFS Mount Points, each with
> > millions
> > of
> > > > files.
> > > > > Since they are all hosted on the same machines, TSM Support
> > highly
> > > > > recommended setting up a node for each, and to have TSM journal
> > each.
> > > Now I
> > > > > am running into a situation where every time that machine is
> > rebooted
> > > for
> > > > > patches/etc., the incrementals have to rebuild the journal
> > database.
> >
> > > Is
> > > > there
> > > > > a way to avoid that, as I am only 1/3 of the way through the
> > Mount
> > > Point
> > > > > Usage (They have only used 12 of the 36 thus far). Please help.
> > J
> > > > >
> > > > >
> > > > >
> > > > > God bless you!!!
> > > > >
> > > > > Chip Bell
> > > > > Network Engineer I
> > > > > IBM Tivoli Certified Deployment Professional (ITSM)
> > > > > Baptist Health System
> > > > > Birmingham, AL
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -----------------------------------------
> > > > > Confidentiality Notice:
> > > > > The information contained in this email message is privileged
and
> > > > > confidential information and intended only for the use of the
> > > > > individual or entity named in the address. If you are not the
> > > > > intended recipient, you are hereby notified that any
> > dissemination,
> > > > > distribution, or copying of this information is strictly
> > > > > prohibited. If you received this information in error, please
> > > > > notify the sender and delete this information from your computer
> > > > > and retain no copies of any of this information.
> > > >
> > > > --------------------------------------------------------------
> > > > Ce message ne contient aucun virus connu. http://www.astaro.fr
> > > >
> > > >
> > > > --------------------------------------------------------------
> > > > Ce message ne contient aucun virus connu. http://www.astaro.fr
|
