Re: [ADSM-L] Extra client sessions

Hi Zoltan,

Is your user doing proxied sessions? If yes, where is the MAXNUMMP
configured: on the agent node (NODENAME aaaaa)? Or the target node
(ASNODENAME bbbbb)? You need to ensure that the target node name has the
limited MAXNUMMP when proxied backups are performed.

Best regards,

Andy

____________________________________________________________________________

Andrew Raibeck | IBM Spectrum Protect Level 3 | storman AT us.ibm DOT com

IBM Tivoli Storage Manager links:
Product support:
https://www.ibm.com/support/entry/portal/product/tivoli/tivoli_storage_manager

Online documentation:
http://www.ibm.com/support/knowledgecenter/SSGSG7/landing/welcome_ssgsg7.html

Product Wiki:
https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%20Storage%20Manager

"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 2016-09-07
08:19:42:

> From: Zoltan Forray <zforray AT VCU DOT EDU>
> To: ADSM-L AT VM.MARIST DOT EDU
> Date: 2016-09-07 08:29
> Subject: Re: Extra client sessions
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
>
> Hi Andy,
>
> But my point is that the user is getting whatever they set for
> RESOURCEUTILIZATION vs what I set for MAXNUMPOINT, without the noise.
This
> user is running 4-multi terabyte backup streams since they set
> RESOURCEUTILIZATION 5 while I still have MAXNUMPOINT set to 1/default.
>
> The users aren't supposed to be able to override the server
> settings/maximums. It would be anarchy!
>
> On Tue, Sep 6, 2016 at 7:10 PM, Andrew Raibeck <storman AT us.ibm DOT com>
wrote:
>
> > Hi Zoltan,
> >
> > Yes, if MAXNUMMP is too low to accommodate a non-default
> > RESOURCEUTILIZATION value, then while the backup should work okay, you
may
> > see some warning messages about there being insufficient mount points.
The
> > client will continue, albeit with fewer sessions. This is why I suggest
> > making sure that if you have a higher RESOURCEUTILIZATION setting, you
have
> > the MAXNUMMP to avoid the "noise", and make sure that mutual
expectations
> > are met (the server is configured to deliver what the client requests).
> > Regardless, though, you are right, in the end it's not fatal to the
> > operation if the settings are mismatched.
> >
> > Best regards,
> >
> > Andy
> >
> > ____________________________________________________________
> > ________________
> >
> > Andrew Raibeck | IBM Spectrum Protect Level 3 | storman AT us.ibm DOT com
> >
> > IBM Tivoli Storage Manager links:
> > Product support:
> > https://www.ibm.com/support/entry/portal/product/tivoli/
> > tivoli_storage_manager
> >
> > Online documentation:
> > http://www.ibm.com/support/knowledgecenter/SSGSG7/
> > landing/welcome_ssgsg7.html
> >
> > Product Wiki:
> > https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%
> > 20Storage%20Manager
> >
> > "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 2016-09-01
> > 08:40:33:
> >
> > > From: Zoltan Forray <zforray AT VCU DOT EDU>
> > > To: ADSM-L AT VM.MARIST DOT EDU
> > > Date: 2016-09-01 08:43
> > > Subject: Re: Extra client sessions
> > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> > >
> > > Thanks for the info.  Yes the user does(did) have RESOURCEUTILIZATION
4
> > > configured.
> > >
> > > I note the APAR you refer to is still open. It refers to v7.1 but how
far
> > > back does it go?  The client recently upgrade all of his nodes to
> > 7.1.6.2,
> > > the latest available for Linux - not sure what level he was at when I
> > first
> > > saw this issue.
> > >
> > > As I said, I always though if MAXNUMPOINTS was set to 1 (the
default),
> > then
> > > what you specified for RESOURCEUTILZATION was ignored and you were
only
> > > supposed to get 2-sessions?  Am I wrong in this assumption?
> > >
> > > On Wed, Aug 31, 2016 at 5:36 PM, Andrew Raibeck <storman AT us.ibm DOT 
> > > com>
> > wrote:
> > >
> > > > Yes, do not use a RESOURCEUTILIZATION higher than the MAXNUMMP
setting.
> > > >
> > > > Having said that, there is an APAR that might ("might" is the
operative
> > > > word!) be a match for this issue, IT16004:
> > > >
> > > > https://www.ibm.com/support/docview.wss?uid=swg1IT16004
> > > >
> > > > In this case, the symptom is seeing more consumer sessions than you
> > would
> > > > expect given the RESOURCEUTILIZATION setting. Even if the specific
> > symptoms
> > > > described in the APAR do not match your scenario, if no other
logical
> > > > explanation fits, it might stil be a match. You can contact support
for
> > > > further problem determination assistance.
> > > >
> > > > Best regards,
> > > >
> > > > Andy
> > > >
> > > > ____________________________________________________________
> > > > ________________
> > > >
> > > > Andrew Raibeck | IBM Spectrum Protect Level 3 | storman AT us.ibm DOT 
> > > > com
> > > >
> > > > IBM Tivoli Storage Manager links:
> > > > Product support:
> > > > https://www.ibm.com/support/entry/portal/product/tivoli/
> > > > tivoli_storage_manager
> > > >
> > > > Online documentation:
> > > > http://www.ibm.com/support/knowledgecenter/SSGSG7/
> > > > landing/welcome_ssgsg7.html
> > > >
> > > > Product Wiki:
> > > >
https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%
> > > > 20Storage%20Manager
> > > >
> > > > "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on
2016-08-31
> > > > 17:22:19:
> > > >
> > > > > From: Karel Bos <tsm.wad AT GMAIL DOT COM>
> > > > > To: ADSM-L AT VM.MARIST DOT EDU
> > > > > Date: 2016-08-31 17:23
> > > > > Subject: Re: Extra client sessions
> > > > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
> > > > >
> > > > > Might want to check resourceutil settings as that limits the
number
> > of
> > > > > sessions clients try to setup. It should match maxnummp or be
lower.
> > > > >
> > > > > Op 31 aug. 2016 22:21 schreef "Zoltan Forray" <zforray AT vcu DOT 
> > > > > edu>:
> > > > >
> > > > > > AHA - so I am not loosing my mind (at least in this situation).
I
> > too
> > > > have
> > > > > > been seeing clients getting >3-sessions eventhough the NODE
> > > > maxnumpoints is
> > > > > > 1!  I was always under the impression that maxnumpoints trumps
> > > > > > resourceutilization.
> > > > > >
> > > > > > On Wed, Aug 31, 2016 at 3:40 PM, Thomas Denier <
> > > > > > Thomas.Denier AT jefferson DOT edu>
> > > > > > wrote:
> > > > > >
> > > > > > > We are occasionally seeing some odd behavior in our TSM
> > environment.
> > > > > > >
> > > > > > > We write incoming client files to sequential disk storage
pools.
> > > > Almost
> > > > > > > all of our client nodes use the default maxnummp value of 1.
> > > > > > >
> > > > > > > When the odd behavior occurs, a number of clients will go
through
> > the
> > > > > > > following sequence of events:
> > > > > > > 1.The TSM server will send a request to start a backup.
> > > > > > > 2.The client will almost immediately open a TCP connection to
be
> > used
> > > > as
> > > > > > a
> > > > > > > producer session (a session used to obtain information from
the
> > TSM
> > > > > > > database).
> > > > > > > 3.Somewhere between tens of seconds and a few minutes later
the
> > > > client
> > > > > > > will open a TCP connection to be used as a consumer session
(a
> > > > session
> > > > > > used
> > > > > > > to send copies of new and changed files).
> > > > > > > 4.Sometime later the client will open a third TCP connection
and
> > > > start
> > > > > > > using it as a consumer session.
> > > > > > > 5.The TSM server will report large numbers of transaction
> > failures
> > > > > > because
> > > > > > > it considers the original consumer session to be tying up the
one
> > > > mount
> > > > > > > point allowed for the node and hence has no way of storing
files
> > > > arriving
> > > > > > > on the new consumer session.
> > > > > > >
> > > > > > > In most cases, all of the affected clients will hit step four
> > within
> > > > an
> > > > > > > interval of a couple of minutes.
> > > > > > >
> > > > > > > My current theory is that step four occurs when the client
system
> > > > detects
> > > > > > > a condition that is viewed as a fatal error in the original
> > consumer
> > > > > > > session, triggering the opening of a replacement consumer
> > session. In
> > > > > > most
> > > > > > > cases the TSM server never detects a problem with the
original
> > > > consumer
> > > > > > > session, and eventually terminates the session after five
hours
> > of
> > > > > > > inactivity (we have database backups that can legitimately go
> > through
> > > > > > long
> > > > > > > periods with no data transfer). More rarely the TSM server
> > eventually
> > > > > > > reports that the original consumer session was severed.
> > > > > > >
> > > > > > > We occasionally see cases where the replacement consumer
session
> > is
> > > > in
> > > > > > > turn replaced by another new session, and even cases where
the
> > latter
> > > > > > > session is replaced by yet another session.
> > > > > > >
> > > > > > > Our client population is a bit over half Windows, but almost
all
> > > > > > instances
> > > > > > > of the odd behavior involve only Windows client systems.
> > > > > > >
> > > > > > > The affected systems are frequently split between two data
> > centers,
> > > > each
> > > > > > > with its own TSM server.
> > > > > > >
> > > > > > > We have usually not found any correlation between the odd TSM
> > > > behavior
> > > > > > and
> > > > > > > issues with other applications. The most recent case was an
> > > > exception.
> > > > > > > There were some e-mail delivery failures at about the same
time
> > as
> > > > step
> > > > > > > four of the odd TSM behavior. The failures occurred when
e-mail
> > > > servers
> > > > > > > were unable to perform LDAP queries.
> > > > > > >
> > > > > > > When we have asked our Network Operations group to check on
> > previous
> > > > > > > occurrences of the odd behavior they have consistently
reported
> > that
> > > > they
> > > > > > > found no evidence of a network problem.
> > > > > > >
> > > > > > > Each of our TSM servers runs under zSeries Linux on a z10 BC.
> > Each
> > > > server
> > > > > > > has a VIPA address with two associated network interfaces on
> > > > different
> > > > > > > subnets.
> > > > > > >
> > > > > > > I would welcome any suggestions for finding the underlying
cause
> > of
> > > > the
> > > > > > > odd behavior.
> > > > > > >
> > > > > > > Thomas Denier,
> > > > > > > Thomas Jefferson University
> > > > > > > The information contained in this transmission contains
> > privileged
> > > > and
> > > > > > > confidential information. It is intended only for the use of
the
> > > > person
> > > > > > > named above. If you are not the intended recipient, you are
> > hereby
> > > > > > notified
> > > > > > > that any review, dissemination, distribution or duplication
of
> > this
> > > > > > > communication is strictly prohibited. If you are not the
intended
> > > > > > > recipient, please contact the sender by reply email and
destroy
> > all
> > > > > > copies
> > > > > > > of the original message.
> > > > > > >
> > > > > > > CAUTION: Intended recipients should NOT use email
communication
> > for
> > > > > > > emergent or urgent health care matters.
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Zoltan Forray*
> > > > > > TSM Software & Hardware Administrator
> > > > > > Xymon Monitor Administrator
> > > > > > VMware Administrator (in training)
> > > > > > Virginia Commonwealth University
> > > > > > UCC/Office of Technology Services
> > > > > > www.ucc.vcu.edu
> > > > > > zforray AT vcu DOT edu - 804-828-4807
> > > > > > Don't be a phishing victim - VCU and other reputable
organizations
> > will
> > > > > > never use email to request that you reply with your password,
> > social
> > > > > > security number or confidential personal information. For more
> > details
> > > > > > visit http://infosecurity.vcu.edu/phishing.html
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > *Zoltan Forray*
> > > TSM Software & Hardware Administrator
> > > Xymon Monitor Administrator
> > > VMware Administrator (in training)
> > > Virginia Commonwealth University
> > > UCC/Office of Technology Services
> > > www.ucc.vcu.edu
> > > zforray AT vcu DOT edu - 804-828-4807
> > > Don't be a phishing victim - VCU and other reputable organizations
will
> > > never use email to request that you reply with your password, social
> > > security number or confidential personal information. For more
details
> > > visit http://infosecurity.vcu.edu/phishing.html
> > >
> >
>
>
>
> --
> *Zoltan Forray*
> TSM Software & Hardware Administrator
> Xymon Monitor Administrator
> VMware Administrator (in training)
> Virginia Commonwealth University
> UCC/Office of Technology Services
> www.ucc.vcu.edu
> zforray AT vcu DOT edu - 804-828-4807
> Don't be a phishing victim - VCU and other reputable organizations will
> never use email to request that you reply with your password, social
> security number or confidential personal information. For more details
> visit http://infosecurity.vcu.edu/phishing.html
>