Andy,
No proxies in use for any backups.
Here is their whole dsm.sys (Linux/Centos). No, I do not know what their
pre/post commands do...
# more /opt/tivoli/tsm/client/ba/bin/dsm.sys
SErvername PROCESSOR
COMMmethod TCPip
TCPPort 1500
TCPServeraddress tsmlinux9.ucc.vcu.edu
TCPWindowsize 127
TCPBuffsize 32
TXNBytelimit 25600
TCPNODELAY Yes
Inclexcl /opt/tivoli/tsm/client/ba/bin/inclexcl.sys
SCHEDMODe Prompted
SCHEDLOGname /opt/tivoli/tsm/client/ba/bin/dsmsched.vcu-gs1.log
ErrorLogName /opt/tivoli/tsm/client/ba/bin/dsmerror.vcu-gs1.log
SCHEDLOGRETENTION 14 D
ERRORLOGRETENTION 14 D
PasswordAccess Generate
Compression Yes
NodeName vcu-gs1.chpc.vcu.edu
PreScheduleCmd "/opt/tivoli/tsm/client/ba/bin/tsm_checker.sh --pre"
PostScheduleCmd "/opt/tivoli/tsm/client/ba/bin/tsm_checker.sh --post"
resourceutilization 5
And here is a q node f=d
9:01:39 AM PROCESSOR : q node vcu-gs1.chpc.vcu.edu f=d
Node Name: VCU-GS1.CHPC.VCU.EDU
Platform: Linux x86-64
Client OS Level: 2.6.32-431.17.1.el6
Client Version: Version 7, release 1, level 6.2
Policy Domain Name: CHPC
Last Access Date/Time: 09/04/2016 13:06:21
Days Since Last Access: 3
Password Set Date/Time: 09/04/2016 13:06:17
Days Since Password Set: 3
Invalid Sign-on Count: 0
Locked?: No
Contact: Carlisle Childress
Compression: Yes
Archive Delete Allowed?: Yes
Backup Delete Allowed?: No
Registration Date/Time: 06/09/2014 15:17:09
Registering Administrator: ZFORRAY
Last Communication Method Used: Tcp/Ip
Bytes Received Last Session: 9,796,989.78 M
Bytes Sent Last Session: 712.69 M
Duration of Last Session: 444,134.42
Pct. Idle Wait Last Session: 0.74
Pct. Comm. Wait Last Session: 41.48
Pct. Media Wait Last Session: 0.00
Optionset:
URL:
Node Type: Client
Password Expiration Period:
Keep Mount Point?: No
Maximum Mount Points Allowed: 1
Auto Filespace Rename : No
Validate Protocol: No
TCP/IP Name: godel21
TCP/IP Address: 192.168.160.21
Globally Unique ID:
6f.1c.6b.7e.f3.16.11.e3.a7.3b.00.23.8b.17.2-
0.17
Transaction Group Max: 0
Data Write Path: ANY
Data Read Path: ANY
Session Initiation: ClientOrServer
High-level Address:
Low-level Address:
Collocation Group Name:
Proxynode Target:
Proxynode Agent:
Node Groups:
Email Address:
Deduplication: ServerOnly
Users allowed to back up: All
Role: Server
Role Override: UseReported
Processor Vendor:
Processor Brand:
Processor Type:
Processor Model:
Processor Count: 0
Hypervisor:
API Application: No
Scan Error: No
MAC Address:
Replication State: None
Replication Mode: None
Backup Replication Rule: DEFAULT
Archive Replication Rule: DEFAULT
Space Management Replication Rule: DEFAULT
Client OS Name: LNX:CentOS release 6.5 (Final)
Client Processor Architecture: x64
Client Products Installed: BA
Client Target Version: (?)
Authentication: Local
SSL Required: No
On Wed, Sep 7, 2016 at 8:56 AM, Andrew Raibeck <storman AT us.ibm DOT com>
wrote:
> Hi Zoltan,
>
> Is your user doing proxied sessions? If yes, where is the MAXNUMMP
> configured: on the agent node (NODENAME aaaaa)? Or the target node
> (ASNODENAME bbbbb)? You need to ensure that the target node name has the
> limited MAXNUMMP when proxied backups are performed.
>
> Best regards,
>
> Andy
>
> ____________________________________________________________
> ________________
>
> Andrew Raibeck | IBM Spectrum Protect Level 3 | storman AT us.ibm DOT com
>
> IBM Tivoli Storage Manager links:
> Product support:
> https://www.ibm.com/support/entry/portal/product/tivoli/
> tivoli_storage_manager
>
> Online documentation:
> http://www.ibm.com/support/knowledgecenter/SSGSG7/
> landing/welcome_ssgsg7.html
>
> Product Wiki:
> https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%
> 20Storage%20Manager
>
> "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 2016-09-07
> 08:19:42:
>
> > From: Zoltan Forray <zforray AT VCU DOT EDU>
> > To: ADSM-L AT VM.MARIST DOT EDU
> > Date: 2016-09-07 08:29
> > Subject: Re: Extra client sessions
> > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> >
> > Hi Andy,
> >
> > But my point is that the user is getting whatever they set for
> > RESOURCEUTILIZATION vs what I set for MAXNUMPOINT, without the noise.
> This
> > user is running 4-multi terabyte backup streams since they set
> > RESOURCEUTILIZATION 5 while I still have MAXNUMPOINT set to 1/default.
> >
> > The users aren't supposed to be able to override the server
> > settings/maximums. It would be anarchy!
> >
> > On Tue, Sep 6, 2016 at 7:10 PM, Andrew Raibeck <storman AT us.ibm DOT com>
> wrote:
> >
> > > Hi Zoltan,
> > >
> > > Yes, if MAXNUMMP is too low to accommodate a non-default
> > > RESOURCEUTILIZATION value, then while the backup should work okay, you
> may
> > > see some warning messages about there being insufficient mount points.
> The
> > > client will continue, albeit with fewer sessions. This is why I suggest
> > > making sure that if you have a higher RESOURCEUTILIZATION setting, you
> have
> > > the MAXNUMMP to avoid the "noise", and make sure that mutual
> expectations
> > > are met (the server is configured to deliver what the client requests).
> > > Regardless, though, you are right, in the end it's not fatal to the
> > > operation if the settings are mismatched.
> > >
> > > Best regards,
> > >
> > > Andy
> > >
> > > ____________________________________________________________
> > > ________________
> > >
> > > Andrew Raibeck | IBM Spectrum Protect Level 3 | storman AT us.ibm DOT com
> > >
> > > IBM Tivoli Storage Manager links:
> > > Product support:
> > > https://www.ibm.com/support/entry/portal/product/tivoli/
> > > tivoli_storage_manager
> > >
> > > Online documentation:
> > > http://www.ibm.com/support/knowledgecenter/SSGSG7/
> > > landing/welcome_ssgsg7.html
> > >
> > > Product Wiki:
> > > https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%
> > > 20Storage%20Manager
> > >
> > > "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on
> > > 2016-09-01
> > > 08:40:33:
> > >
> > > > From: Zoltan Forray <zforray AT VCU DOT EDU>
> > > > To: ADSM-L AT VM.MARIST DOT EDU
> > > > Date: 2016-09-01 08:43
> > > > Subject: Re: Extra client sessions
> > > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> > > >
> > > > Thanks for the info. Yes the user does(did) have RESOURCEUTILIZATION
> 4
> > > > configured.
> > > >
> > > > I note the APAR you refer to is still open. It refers to v7.1 but how
> far
> > > > back does it go? The client recently upgrade all of his nodes to
> > > 7.1.6.2,
> > > > the latest available for Linux - not sure what level he was at when I
> > > first
> > > > saw this issue.
> > > >
> > > > As I said, I always though if MAXNUMPOINTS was set to 1 (the
> default),
> > > then
> > > > what you specified for RESOURCEUTILZATION was ignored and you were
> only
> > > > supposed to get 2-sessions? Am I wrong in this assumption?
> > > >
> > > > On Wed, Aug 31, 2016 at 5:36 PM, Andrew Raibeck <storman AT us.ibm DOT
> > > > com>
> > > wrote:
> > > >
> > > > > Yes, do not use a RESOURCEUTILIZATION higher than the MAXNUMMP
> setting.
> > > > >
> > > > > Having said that, there is an APAR that might ("might" is the
> operative
> > > > > word!) be a match for this issue, IT16004:
> > > > >
> > > > > https://www.ibm.com/support/docview.wss?uid=swg1IT16004
> > > > >
> > > > > In this case, the symptom is seeing more consumer sessions than you
> > > would
> > > > > expect given the RESOURCEUTILIZATION setting. Even if the specific
> > > symptoms
> > > > > described in the APAR do not match your scenario, if no other
> logical
> > > > > explanation fits, it might stil be a match. You can contact support
> for
> > > > > further problem determination assistance.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Andy
> > > > >
> > > > > ____________________________________________________________
> > > > > ________________
> > > > >
> > > > > Andrew Raibeck | IBM Spectrum Protect Level 3 | storman AT us.ibm DOT
> > > > > com
> > > > >
> > > > > IBM Tivoli Storage Manager links:
> > > > > Product support:
> > > > > https://www.ibm.com/support/entry/portal/product/tivoli/
> > > > > tivoli_storage_manager
> > > > >
> > > > > Online documentation:
> > > > > http://www.ibm.com/support/knowledgecenter/SSGSG7/
> > > > > landing/welcome_ssgsg7.html
> > > > >
> > > > > Product Wiki:
> > > > >
> https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%
> > > > > 20Storage%20Manager
> > > > >
> > > > > "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on
> 2016-08-31
> > > > > 17:22:19:
> > > > >
> > > > > > From: Karel Bos <tsm.wad AT GMAIL DOT COM>
> > > > > > To: ADSM-L AT VM.MARIST DOT EDU
> > > > > > Date: 2016-08-31 17:23
> > > > > > Subject: Re: Extra client sessions
> > > > > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> > > > > >
> > > > > > Might want to check resourceutil settings as that limits the
> number
> > > of
> > > > > > sessions clients try to setup. It should match maxnummp or be
> lower.
> > > > > >
> > > > > > Op 31 aug. 2016 22:21 schreef "Zoltan Forray" <zforray AT vcu DOT
> > > > > > edu>:
> > > > > >
> > > > > > > AHA - so I am not loosing my mind (at least in this situation).
> I
> > > too
> > > > > have
> > > > > > > been seeing clients getting >3-sessions eventhough the NODE
> > > > > maxnumpoints is
> > > > > > > 1! I was always under the impression that maxnumpoints trumps
> > > > > > > resourceutilization.
> > > > > > >
> > > > > > > On Wed, Aug 31, 2016 at 3:40 PM, Thomas Denier <
> > > > > > > Thomas.Denier AT jefferson DOT edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > We are occasionally seeing some odd behavior in our TSM
> > > environment.
> > > > > > > >
> > > > > > > > We write incoming client files to sequential disk storage
> pools.
> > > > > Almost
> > > > > > > > all of our client nodes use the default maxnummp value of 1.
> > > > > > > >
> > > > > > > > When the odd behavior occurs, a number of clients will go
> through
> > > the
> > > > > > > > following sequence of events:
> > > > > > > > 1.The TSM server will send a request to start a backup.
> > > > > > > > 2.The client will almost immediately open a TCP connection to
> be
> > > used
> > > > > as
> > > > > > > a
> > > > > > > > producer session (a session used to obtain information from
> the
> > > TSM
> > > > > > > > database).
> > > > > > > > 3.Somewhere between tens of seconds and a few minutes later
> the
> > > > > client
> > > > > > > > will open a TCP connection to be used as a consumer session
> (a
> > > > > session
> > > > > > > used
> > > > > > > > to send copies of new and changed files).
> > > > > > > > 4.Sometime later the client will open a third TCP connection
> and
> > > > > start
> > > > > > > > using it as a consumer session.
> > > > > > > > 5.The TSM server will report large numbers of transaction
> > > failures
> > > > > > > because
> > > > > > > > it considers the original consumer session to be tying up the
> one
> > > > > mount
> > > > > > > > point allowed for the node and hence has no way of storing
> files
> > > > > arriving
> > > > > > > > on the new consumer session.
> > > > > > > >
> > > > > > > > In most cases, all of the affected clients will hit step four
> > > within
> > > > > an
> > > > > > > > interval of a couple of minutes.
> > > > > > > >
> > > > > > > > My current theory is that step four occurs when the client
> system
> > > > > detects
> > > > > > > > a condition that is viewed as a fatal error in the original
> > > consumer
> > > > > > > > session, triggering the opening of a replacement consumer
> > > session. In
> > > > > > > most
> > > > > > > > cases the TSM server never detects a problem with the
> original
> > > > > consumer
> > > > > > > > session, and eventually terminates the session after five
> hours
> > > of
> > > > > > > > inactivity (we have database backups that can legitimately go
> > > through
> > > > > > > long
> > > > > > > > periods with no data transfer). More rarely the TSM server
> > > eventually
> > > > > > > > reports that the original consumer session was severed.
> > > > > > > >
> > > > > > > > We occasionally see cases where the replacement consumer
> session
> > > is
> > > > > in
> > > > > > > > turn replaced by another new session, and even cases where
> the
> > > latter
> > > > > > > > session is replaced by yet another session.
> > > > > > > >
> > > > > > > > Our client population is a bit over half Windows, but almost
> all
> > > > > > > instances
> > > > > > > > of the odd behavior involve only Windows client systems.
> > > > > > > >
> > > > > > > > The affected systems are frequently split between two data
> > > centers,
> > > > > each
> > > > > > > > with its own TSM server.
> > > > > > > >
> > > > > > > > We have usually not found any correlation between the odd TSM
> > > > > behavior
> > > > > > > and
> > > > > > > > issues with other applications. The most recent case was an
> > > > > exception.
> > > > > > > > There were some e-mail delivery failures at about the same
> time
> > > as
> > > > > step
> > > > > > > > four of the odd TSM behavior. The failures occurred when
> e-mail
> > > > > servers
> > > > > > > > were unable to perform LDAP queries.
> > > > > > > >
> > > > > > > > When we have asked our Network Operations group to check on
> > > previous
> > > > > > > > occurrences of the odd behavior they have consistently
> reported
> > > that
> > > > > they
> > > > > > > > found no evidence of a network problem.
> > > > > > > >
> > > > > > > > Each of our TSM servers runs under zSeries Linux on a z10 BC.
> > > Each
> > > > > server
> > > > > > > > has a VIPA address with two associated network interfaces on
> > > > > different
> > > > > > > > subnets.
> > > > > > > >
> > > > > > > > I would welcome any suggestions for finding the underlying
> cause
> > > of
> > > > > the
> > > > > > > > odd behavior.
> > > > > > > >
> > > > > > > > Thomas Denier,
> > > > > > > > Thomas Jefferson University
> > > > > > > > The information contained in this transmission contains
> > > privileged
> > > > > and
> > > > > > > > confidential information. It is intended only for the use of
> the
> > > > > person
> > > > > > > > named above. If you are not the intended recipient, you are
> > > hereby
> > > > > > > notified
> > > > > > > > that any review, dissemination, distribution or duplication
> of
> > > this
> > > > > > > > communication is strictly prohibited. If you are not the
> intended
> > > > > > > > recipient, please contact the sender by reply email and
> destroy
> > > all
> > > > > > > copies
> > > > > > > > of the original message.
> > > > > > > >
> > > > > > > > CAUTION: Intended recipients should NOT use email
> communication
> > > for
> > > > > > > > emergent or urgent health care matters.
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Zoltan Forray*
> > > > > > > TSM Software & Hardware Administrator
> > > > > > > Xymon Monitor Administrator
> > > > > > > VMware Administrator (in training)
> > > > > > > Virginia Commonwealth University
> > > > > > > UCC/Office of Technology Services
> > > > > > > www.ucc.vcu.edu
> > > > > > > zforray AT vcu DOT edu - 804-828-4807
> > > > > > > Don't be a phishing victim - VCU and other reputable
> organizations
> > > will
> > > > > > > never use email to request that you reply with your password,
> > > social
> > > > > > > security number or confidential personal information. For more
> > > details
> > > > > > > visit http://infosecurity.vcu.edu/phishing.html
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *Zoltan Forray*
> > > > TSM Software & Hardware Administrator
> > > > Xymon Monitor Administrator
> > > > VMware Administrator (in training)
> > > > Virginia Commonwealth University
> > > > UCC/Office of Technology Services
> > > > www.ucc.vcu.edu
> > > > zforray AT vcu DOT edu - 804-828-4807
> > > > Don't be a phishing victim - VCU and other reputable organizations
> will
> > > > never use email to request that you reply with your password, social
> > > > security number or confidential personal information. For more
> details
> > > > visit http://infosecurity.vcu.edu/phishing.html
> > > >
> > >
> >
> >
> >
> > --
> > *Zoltan Forray*
> > TSM Software & Hardware Administrator
> > Xymon Monitor Administrator
> > VMware Administrator (in training)
> > Virginia Commonwealth University
> > UCC/Office of Technology Services
> > www.ucc.vcu.edu
> > zforray AT vcu DOT edu - 804-828-4807
> > Don't be a phishing victim - VCU and other reputable organizations will
> > never use email to request that you reply with your password, social
> > security number or confidential personal information. For more details
> > visit http://infosecurity.vcu.edu/phishing.html
> >
>
--
*Zoltan Forray*
TSM Software & Hardware Administrator
Xymon Monitor Administrator
VMware Administrator (in training)
Virginia Commonwealth University
UCC/Office of Technology Services
www.ucc.vcu.edu
zforray AT vcu DOT edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html
|
