• Please help support our sponsors by considering their products and services.
    Our sponsors enable us to serve you with this high-speed Internet connection and fast webservers you are currently using at ADSM.ORG.
    They support this free flow of information and knowledge exchange service at no cost to you.

    Please welcome our latest sponsor Tectrade . We can show our appreciation by learning more about Tectrade Solutions
  • Community Tip: Please Give Thanks to Those Sharing Their Knowledge.

    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.

  • Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)

    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

Journal based backup advice

mattl

Newcomer
Joined
Oct 16, 2019
Messages
4
Reaction score
0
Points
0
Hi,

I want to configure Journal based backup for a Windows client that has a lot of files to backup.

I read the following in the IBM FAQs:
Because of this and the stated limitation of journal backup, it is recommended that periodic full incremental backups be done along with more frequent journal backups.

So I planned to have a normal client schedule to run on weekdays which would use the journal service and a second incremental client schedule to run on a weekend with the -nojournal option.

A TSM consultant in our service partner told me that I would need to use two different TSM client nodes for this single Windows server - one for the journal backups and a second one for the weekend incremental backups.

Is this correct? Has anyone else configured Journal based backups with two different client nodes for a single Windows server?

I was planning to use just one TSM client node associated with both schedules for backing up this single Windows server.

Thanks in advance
 

marclant

ADSM.ORG Moderator
Joined
Jun 16, 2006
Messages
3,703
Reaction score
611
Points
0
Location
Canada
Website
www.ibm.com
A TSM consultant in our service partner told me that I would need to use two different TSM client nodes for this single Windows server - one for the journal backups and a second one for the weekend incremental backups.

Is this correct?
No. You were on the right track to have a schedule that runs Sunday to Friday, and a 2nd one for the same node with the option -nojournal on Saturday.

The weekly regular incremental will resync the journal database and handle anything that may have been missed by the journal during the week.
 

obiwan2

ADSM.ORG Member
Joined
Dec 8, 2005
Messages
184
Reaction score
0
Points
0
Location
Cologne / Germany
Website
Visit site
Hi marclant,

I'm also "playing" with the journal....we have round about 19 million files to backup. I have only one schedule for this node that runs every day.
What would be -in your opinion- the best way to backup this huge filesystem ?

Here is my dsm.opt:
NODE ZDHPROFILES
CLUSTERNODE YES
PASSWORDACCESS GENERATE
DOMAIN P:
HTTPPORT 1588
TCPSERVERADDRESS tsm05-backup
TCPPORT 1500
RESOURCEUTILIZATION 2
TXNBYTELIMIT 5G
TCPWINDOWSIZE 63
TCPNODELAY yes
LANG AMENG
SCHEDMODE POLLING
QUERYSCHEDPERIOD 1

JournalPipe \\.\pipe\jnlServiceClusterDisk_P

ERRORLOGNAME P:\Shares\profile\TSM\dsmerror.log
SCHEDLOGNAME P:\Shares\profile\TSM\dsmsched.log
ERRORLOGRETENTION 5
SCHEDLOGRETENTION 5


This is the tsmjbbd.ini:
[JournalSettings]
Errorlog=P:\Shares\profile\TSM\journal-error.log
JournalPipe=\\.\pipe\jnlServiceClusterDisk_P
;
[JournalExcludeList]
%SystemRoot%\System32\Config\*
%SystemDrive%\Adsm.Sys\*
%TEMP%\*
%TMP%\*
%USERPROFILE%\*\index.dat
*\System Volume Information\*
*\Recycle*\*
*\*.jbbdb
*\*\obi.tsm
*\*\obh.tsm
*\*\obd.tsm
*\tsmlvsacache
*\tsmlvsacache\*
*\pagefile.sys
*\*.crmlog
*\hiberfil.sys
%SystemRoot%\csc\*
%SystemRoot%\System32\DTCLog\MSDTC.LOG
%SystemRoot%\netlogon.chg
%SystemRoot%\schedlgu.txt
%SystemRoot%\registration\*.clb
%WINDIR%\debug\*
%WINDIR%\system32\config\*
%TEMP%\*
%TMP%\*
%SystemDrive%\Documents and Settings\NetShowServices\ntuser.dat
%SystemDrive%\Documents and Settings\NetShowServices\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
%USERPROFILE%\ntuser.dat
%USERPROFILE%\Local Settings\Application Data\Microsoft\Windows\UsrClass.dat
*\microsoft uam volume\*
*\microsoft uam volume\*
*\IBMBIO.COM*
*\IBMDOS.COM*
*\MSDOS.SYS
*\IO.SYS
%WINDIR%\system32\Perflib*.dat
%WINDIR%\system32\dhcp\*
%WINDIR%\system32\dhcp\backup*\*
%WINDIR%\system32\dns*\*
%WINDIR%\system32\dns\backup\*
*\System Volume Information*
*\*Temporary Internet Files*
*\Recycled*
*\Recycler*
*\OLDLOGS\*
*\Journal_ZDHPROFILES\*
;
;
[JournaledFileSystemSettings]
;
JournaledFileSystems=P:\
JournalDir=P:\Shares\profile\TSM\Journal_ZDHPROFILES
JournalDBSize=0x00a00000
NotifyFilter=0x00000117
NotifyBufferSize=0x00008000
DirNotifyBufferSize=0x00004000
PreserveDBOnExit=1


My system is a 2-node-cluster. so I created a cluster-ressource that starts the journal-service. This service runs only all the time for one dedicated filesystem.
When I now create another schedule (i.e. for saturday) with the option "-nojournal" does it really disable the journal-service and runs as a "normal incremental backup" ?
 

marclant

ADSM.ORG Moderator
Joined
Jun 16, 2006
Messages
3,703
Reaction score
611
Points
0
Location
Canada
Website
www.ibm.com
When I now create another schedule (i.e. for saturday) with the option "-nojournal" does it really disable the journal-service and runs as a "normal incremental backup" ?
It doesn't disable the journal service, the journal service stays running, but the client does a normal incremental backup and the journal database is reset at the end of the backup and ready for the next journal backup.

As long as the journal backups work well, there's really no need to do a weekly backup without the journal, you could go monthly. There's also no harm in doing it weekly if your backup window permits it because it does take a long time to inspect 19 million files. You could also increase the RESOURCEUTIL to have more threads processing the large filesystem as long as the backups are not going to tape. That might reduce the backup time a little when not using the journal.
 

obiwan2

ADSM.ORG Member
Joined
Dec 8, 2005
Messages
184
Reaction score
0
Points
0
Location
Cologne / Germany
Website
Visit site
I tried the journaled backup over the weekend and that's what I got

dsmerror.log:
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Pipe connection was severed, attempting re-establish session on pipe '\\.\pipe\jnlServiceClusterDisk_P__jnlSession_1' ...
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Reconnect failed.
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
02/27/2021 10:35:10 ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
02/27/2021 10:35:10 ANS0361I DIAG: jnlWrite(): Write failed, rc -190 from commWrite().
.....
.....
02/27/2021 10:35:48 ANS2250W A IBM Spectrum Protect core file or crash report was found: c:\Program Files\tivoli\tsm\baclient\dsmcrash.dmp
02/27/2021 10:35:56 ANS2250W A IBM Spectrum Protect core file or crash report was found: c:\program files\tivoli\tsm\baclient\dsmcrash.dmp
02/27/2021 11:08:48 ANS2250W A IBM Spectrum Protect core file or crash report was found: C:\Program Files\Tivoli\TSM\baclient\dsmcrash.dmp

And in the journal's errorlog:
02/26/2021 18:51:48 ANS0361I DIAG: handleExceedUserLimit(): Journal database for filespace P: exceeded user-defined maximum of 0x00a00000.
handleExceedUserLimit(): Restarting the journal db in the invalid state.
02/27/2021 10:35:03 ANS0361I DIAG: psNpRead: Error reading data, Win32 rc=233
02/27/2021 10:35:03 ANS0361I DIAG: JnlReadVerb(): error reading verb, psNpRead(): rc=233 .
02/27/2021 10:35:03 ANS0361I DIAG: JnlClientSession(Session 1): Error reading data from client, rc=233 .
02/27/2021 10:35:03 ANS0361I DIAG: psNpDisconnect(): Failure: DisconnectPipe: Win32 rc=6 .
02/27/2021 10:35:05 ANS0361I DIAG: psNpRead: Error reading data, Win32 rc=6
02/27/2021 10:35:05 ANS0361I DIAG: JnlReadVerb(): error reading verb, psNpRead(): rc=6 .
02/27/2021 10:35:05 ANS0361I DIAG: JnlClientSession(Session 2): Error reading data from client, rc=6 .
02/27/2021 10:35:05 ANS0361I DIAG: psNpDisconnect(): Failure: DisconnectPipe: Win32 rc=6 .
02/27/2021 22:22:15 ANS0361I DIAG: handleExceedUserLimit(): Journal database for filespace P: exceeded user-defined maximum of 0x00a00000.
handleExceedUserLimit(): Restarting the journal db in the invalid state.

Seems that some config parameters are still wrong....
 

obiwan2

ADSM.ORG Member
Joined
Dec 8, 2005
Messages
184
Reaction score
0
Points
0
Location
Cologne / Germany
Website
Visit site
I changed the settings now to (in tsmjbbd.ini):

[JournaledFileSystemSettings]
;
JournaledFileSystems=P:\
JournalDir=P:\Shares\profile\TSM\Journal_ZDHPROFILES
JournalDBSize=0x00000000
NotifyFilter=0x00000117
NotifyBufferSize=0x50000000
DirNotifyBufferSize=0x10000000
PreserveDBOnExit=1
DeferFsMonStart=1
DeferRetryInterval=30
SessionTimeout=2220
 

marclant

ADSM.ORG Moderator
Joined
Jun 16, 2006
Messages
3,703
Reaction score
611
Points
0
Location
Canada
Website
www.ibm.com
I'm not convinced it's a settings issue. Two errors stick out:

Code:
ANS0361I DIAG: win32NpWrite(): Error 6 writing to named pipe
ANS0361I DIAG: psNpRead: Error reading data, Win32 rc=233
The 6 and 233 are Windows return codes, here's what they tranlate to.
1614608941205.png

So it does appear that possibly the journal service went down, or the client. Either way, looks like they were not able to connect on that pipe. Might not hurt to restart the journal service and the scheduler if you are doing a a scheduled backup, or restart the client if you are doing a manual backup.
 

RecoveryOne

ADSM.ORG Senior Member
Joined
Mar 15, 2017
Messages
307
Reaction score
68
Points
0

obiwan2

ADSM.ORG Member
Joined
Dec 8, 2005
Messages
184
Reaction score
0
Points
0
Location
Cologne / Germany
Website
Visit site
Just wondering, you have 19m files, how much of that data changes? The Journal based backup has issues with high change rates. See https://www.ibm.com/support/pages/tivoli-storage-manager-journal-based-backup-faq.

**EDIT: I thought there was a note about journal based backups and filesystems with over 1 million files that its not advisable to do it, but I can't find that one right now.
Good morning,

daily chages in this filesystem about 125.000 files with a total volume of approx. 25 GB

That's not much in size....but 125.000 (very) small files...
 

obiwan2

ADSM.ORG Member
Joined
Dec 8, 2005
Messages
184
Reaction score
0
Points
0
Location
Cologne / Germany
Website
Visit site
I played a little with the parameters and was still having problems.... the JournalDB was not built...even with a successful incremental backup.

I changed the RESOURCEUTIL parameter to 8 and now approx. 19 million files will be checked and backed up in about 2 hrs. That's fine for us and we are happy with it !

Thanks to all for your help !!

Oh...still one thing....
I don't want to backup $RECYCLE.BIN and it's contents...... but these commands in inclexcl.txt seem not wo work:

EXCLUDE "*:\...\$RECYCLE.BIN"
EXCLUDE "*:\...\$RECYCLE.BIN\*.*"
EXCLUDE "*:\...\$RECYCLE.BIN\...\*.*"

What's wrong with this ? INCLEXCL is still a little secret to me.
 

Advertise at ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

UpCloud high performance VPS at $5/month

Get started with $25 in credits on Cloud Servers. You must use link below to receive the credit. Use the promo to get upto 5 month of FREE Linux VPS.

The Spectrum Protect TLA (Three-Letter Acronym): ISP or something else?

  • Every product needs a TLA, Let's call it ISP (IBM Spectrum Protect).

    Votes: 20 18.9%
  • Keep using TSM for Spectrum Protect.

    Votes: 64 60.4%
  • Let's be formal and just say Spectrum Protect

    Votes: 13 12.3%
  • Other (please comement)

    Votes: 9 8.5%

Forum statistics

Threads
31,837
Messages
135,770
Members
21,788
Latest member
london
Top