Journal based backup advice

mattl

Active Newcomer
Joined
Oct 16, 2019
Messages
9
Reaction score
0
Points
0
PREDATAR Control23

Hi,

I want to configure Journal based backup for a Windows client that has a lot of files to backup.

I read the following in the IBM FAQs:
Because of this and the stated limitation of journal backup, it is recommended that periodic full incremental backups be done along with more frequent journal backups.

So I planned to have a normal client schedule to run on weekdays which would use the journal service and a second incremental client schedule to run on a weekend with the -nojournal option.

A TSM consultant in our service partner told me that I would need to use two different TSM client nodes for this single Windows server - one for the journal backups and a second one for the weekend incremental backups.

Is this correct? Has anyone else configured Journal based backups with two different client nodes for a single Windows server?

I was planning to use just one TSM client node associated with both schedules for backing up this single Windows server.

Thanks in advance
 
PREDATAR Control23

A TSM consultant in our service partner told me that I would need to use two different TSM client nodes for this single Windows server - one for the journal backups and a second one for the weekend incremental backups.

Is this correct?
No. You were on the right track to have a schedule that runs Sunday to Friday, and a 2nd one for the same node with the option -nojournal on Saturday.

The weekly regular incremental will resync the journal database and handle anything that may have been missed by the journal during the week.
 
PREDATAR Control23

Hi marclant,

I'm also "playing" with the journal....we have round about 19 million files to backup. I have only one schedule for this node that runs every day.
What would be -in your opinion- the best way to backup this huge filesystem ?

Here is my dsm.opt:
NODE ZDHPROFILES
CLUSTERNODE YES
PASSWORDACCESS GENERATE
DOMAIN P:
HTTPPORT 1588
TCPSERVERADDRESS tsm05-backup
TCPPORT 1500
RESOURCEUTILIZATION 2
TXNBYTELIMIT 5G
TCPWINDOWSIZE 63
TCPNODELAY yes
LANG AMENG
SCHEDMODE POLLING
QUERYSCHEDPERIOD 1

JournalPipe \\.\pipe\jnlServiceClusterDisk_P

ERRORLOGNAME P:\Shares\profile\TSM\dsmerror.log
SCHEDLOGNAME P:\Shares\profile\TSM\dsmsched.log
ERRORLOGRETENTION 5
SCHEDLOGRETENTION 5


This is the tsmjbbd.ini:
[JournalSettings]
Errorlog=P:\Shares\profile\TSM\journal-error.log
JournalPipe=\\.\pipe\jnlServiceClusterDisk_P
;
[JournalExcludeList]
%SystemRoot%\System32\Config\*
%SystemDrive%\Adsm.Sys\*
%TEMP%\*
%TMP%\*
%USERPROFILE%\*\index.dat
*\System Volume Information\*
*\Recycle*\*
*\*.jbbdb
*\*\obi.tsm
*\*\obh.tsm
*\*\obd.tsm
*\tsmlvsacache
*\tsmlvsacache\*
*\pagefile.sys
*\*.crmlog
*\hiberfil.sys
%SystemRoot%\csc\*
%SystemRoot%\System32\DTCLog\MSDTC.LOG
%SystemRoot%\netlogon.chg
%SystemRoot%\schedlgu.txt
%SystemRoot%\registration\*.clb
%WINDIR%\debug\*
%WINDIR%\system32\config\*
%TEMP%\*
%TMP%\*
%SystemDrive%\Documents and Settings\NetShowServices\ntuser.dat
%SystemDrive%\Documents and Settings\NetShowServices\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
%USERPROFILE%\ntuser.dat
%USERPROFILE%\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
*\microsoft uam volume\*
*\microsoft uam volume\*
*\IBMBIO.COM*
*\IBMDOS.COM*
*\MSDOS.SYS
*\IO.SYS
%WINDIR%\system32\Perflib*.dat
%WINDIR%\system32\dhcp\*
%WINDIR%\system32\dhcp\backup*\*
%WINDIR%\system32\dns*\*
%WINDIR%\system32\dns\backup\*
*\System Volume Information*
*\*Temporary Internet Files*
*\Recycled*
*\Recycler*
*\OLDLOGS\*
*\Journal_ZDHPROFILES\*
;
;
[JournaledFileSystemSettings]
;
JournaledFileSystems=P:\
JournalDir=P:\Shares\profile\TSM\Journal_ZDHPROFILES
JournalDBSize=0x00a00000
NotifyFilter=0x00000117
NotifyBufferSize=0x00008000
DirNotifyBufferSize=0x00004000
PreserveDBOnExit=1


My system is a 2-node-cluster. so I created a cluster-ressource that starts the journal-service. This service runs only all the time for one dedicated filesystem.
When I now create another schedule (i.e. for saturday) with the option "-nojournal" does it really disable the journal-service and runs as a "normal incremental backup" ?
 
PREDATAR Control23

When I now create another schedule (i.e. for saturday) with the option "-nojournal" does it really disable the journal-service and runs as a "normal incremental backup" ?
It doesn't disable the journal service, the journal service stays running, but the client does a normal incremental backup and the journal database is reset at the end of the backup and ready for the next journal backup.

As long as the journal backups work well, there's really no need to do a weekly backup without the journal, you could go monthly. There's also no harm in doing it weekly if your backup window permits it because it does take a long time to inspect 19 million files. You could also increase the RESOURCEUTIL to have more threads processing the large filesystem as long as the backups are not going to tape. That might reduce the backup time a little when not using the journal.
 
PREDATAR Control23

I tried the journaled backup over the weekend and that's what I got

dsmerror.log:
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Pipe connection was severed, attempting re-establish session on pipe '\\.\pipe\jnlServiceClusterDisk_P__jnlSession_1' ...
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Reconnect failed.
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
.....
.....
02/27/2021 10:35:48 ANS2250W A IBM Spectrum Protect core file or crash report was found: c:\Program Files\tivoli\tsm\baclient\dsmcrash.dmp
02/27/2021 10:35:56 ANS2250W A IBM Spectrum Protect core file or crash report was found: c:\program files\tivoli\tsm\baclient\dsmcrash.dmp
02/27/2021 11:08:48 ANS2250W A IBM Spectrum Protect core file or crash report was found: C:\Program Files\Tivoli\TSM\baclient\dsmcrash.dmp

And in the journal's errorlog:
02/26/2021 18:51:48 ANS0361I DIAG: handleExceedUserLimit(): Journal database for filespace P: exceeded user-defined maximum of 0x00a00000.
handleExceedUserLimit(): Restarting the journal db in the invalid state.
02/27/2021 10:35:03 ANS0361I DIAG: psNpRead: Error reading data, Win32 rc=233
02/27/2021 10:35:03 ANS0361I DIAG: JnlReadVerb(): error reading verb, psNpRead(): rc=233 .
02/27/2021 10:35:03 ANS0361I DIAG: JnlClientSession(Session 1): Error reading data from client, rc=233 .
02/27/2021 10:35:03 ANS0361I DIAG: psNpDisconnect(): Failure: DisconnectPipe: Win32 rc=6 .
02/27/2021 10:35:05 ANS0361I DIAG: psNpRead: Error reading data, Win32 rc=6
02/27/2021 10:35:05 ANS0361I DIAG: JnlReadVerb(): error reading verb, psNpRead(): rc=6 .
02/27/2021 10:35:05 ANS0361I DIAG: JnlClientSession(Session 2): Error reading data from client, rc=6 .
02/27/2021 10:35:05 ANS0361I DIAG: psNpDisconnect(): Failure: DisconnectPipe: Win32 rc=6 .
02/27/2021 22:22:15 ANS0361I DIAG: handleExceedUserLimit(): Journal database for filespace P: exceeded user-defined maximum of 0x00a00000.
handleExceedUserLimit(): Restarting the journal db in the invalid state.

Seems that some config parameters are still wrong....
 
PREDATAR Control23

I changed the settings now to (in tsmjbbd.ini):

[JournaledFileSystemSettings]
;
JournaledFileSystems=P:\
JournalDir=P:\Shares\profile\TSM\Journal_ZDHPROFILES
JournalDBSize=0x00000000
NotifyFilter=0x00000117
NotifyBufferSize=0x50000000
DirNotifyBufferSize=0x10000000
PreserveDBOnExit=1
DeferFsMonStart=1
DeferRetryInterval=30
SessionTimeout=2220
 
PREDATAR Control23

I'm not convinced it's a settings issue. Two errors stick out:

Code:
ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
ANS0361I DIAG: psNpRead: Error reading data, Win32 rc=233

The 6 and 233 are Windows return codes, here's what they tranlate to.
1614608941205.png

So it does appear that possibly the journal service went down, or the client. Either way, looks like they were not able to connect on that pipe. Might not hurt to restart the journal service and the scheduler if you are doing a a scheduled backup, or restart the client if you are doing a manual backup.
 
PREDATAR Control23

Just wondering, you have 19m files, how much of that data changes? The Journal based backup has issues with high change rates. See https://www.ibm.com/support/pages/tivoli-storage-manager-journal-based-backup-faq.

**EDIT: I thought there was a note about journal based backups and filesystems with over 1 million files that its not advisable to do it, but I can't find that one right now.
Good morning,

daily chages in this filesystem about 125.000 files with a total volume of approx. 25 GB

That's not much in size....but 125.000 (very) small files...
 
PREDATAR Control23

I played a little with the parameters and was still having problems.... the JournalDB was not built...even with a successful incremental backup.

I changed the RESOURCEUTIL parameter to 8 and now approx. 19 million files will be checked and backed up in about 2 hrs. That's fine for us and we are happy with it !

Thanks to all for your help !!

Oh...still one thing....
I don't want to backup $RECYCLE.BIN and it's contents...... but these commands in inclexcl.txt seem not wo work:

EXCLUDE "*:\...\$RECYCLE.BIN"
EXCLUDE "*:\...\$RECYCLE.BIN\*.*"
EXCLUDE "*:\...\$RECYCLE.BIN\...\*.*"

What's wrong with this ? INCLEXCL is still a little secret to me.
 
Top