ADSM-L

Re: [ADSM-L] Reg: datamover sessions hung for tsm/ve backups

2016-07-05 10:47:47
Subject: Re: [ADSM-L] Reg: datamover sessions hung for tsm/ve backups
From: Tom Alverson <tom.alverson AT GMAIL DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Tue, 5 Jul 2016 10:45:36 -0400
This is an ongoing issue that I suspect began when the VMware environment
was upgraded from 5.0 to 5.5 (the TSM for VE backups have been in place for
several years and this issue was never seen until the last 6 months or
so).  I suspect our storage servers have been rebooted in the last 6 months
but I will double check on that.

On Tue, Jul 5, 2016 at 10:04 AM, Billaudeau, Pierre <P.Billaudeau AT saq.qc DOT 
ca>
wrote:

> We had a similar situation and we resolved it by halting and restarting
> TSM server. TSM/VE were taking longer and longer to complete and some
> Datamovers were hanging.
>
> Pierre Billaudeau
> SAQ
>
> -----Message d'origine-----
> De : ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] De la part 
> de
> Tom Alverson
> Envoyé : 5 juillet 2016 09:40
> À : ADSM-L AT VM.MARIST DOT EDU
> Objet : Re: [ADSM-L] Reg: datamover sessions hung for tsm/ve backups
>
> Does anyone have a resolution for this yet?  I have a PMR open with a level
> 2 tech and so far all he has said was to update VMware tools to 10.0.9.   I
> did that on the datamovers which did not help at all.  I am trying to get
> our VM team to update the VM's but that is a very slow process since they
> would all need to be rebooted.   Last night half of our datamovers crashed
> with this error.
>
> On Wed, Apr 6, 2016 at 10:33 AM, Tom Alverson <tom.alverson AT gmail DOT com>
> wrote:
>
> > I am getting this error on three different TSM for VE VBS servers today.
> > Waiting 60 minutes for each VM is causing the schedule to run into the
> > next days schedule (the job runs for over 24 hours).  I have to stop
> > and restart the service to snap it out of this mode and hope the next
> backup runs OK.
> >
> > Tom
> >
> >
> > On Sun, Mar 27, 2016 at 4:45 AM, Srikanth Kola23 <srkola23 AT in.ibm DOT 
> > com>
> > wrote:
> >
> >> Hi Team,
> >>
> >> I am facing tsm/ve backups hung  over some couple of weeks which is
> >> requiring manual intervention to reboot the datamover server then
> >> only I am able to get the backup sessions running fine.
> >>
> >> share your experience and fixes if any you come across for this issue
> >>
> >> tsm/ve version 7140
> >> platform windows 2012
> >> tsm server version  7.1.4.100
> >>
> >> I do not find any errors in error log for the hungs when we had
> >> traces enabled so disabled them then the error log saying as below
> >> error.VE_POK11DM3.log ========================
> >> 03/26/2016 22:29:37 ANS1717E A VMware vStorage web service task failed.
> >> 03/26/2016 23:07:17 ANS0361I DIAG: The VDDK Utility Thread may be
> >> hung processing another request.
> >> 03/26/2016 23:07:17 ANS0361I DIAG: Waiting up to another 3600000
> >> milliseconds to process a request to VddkFullVMCloseVMDKs for vm
> >> 'pokcsmeta01'.
> >> 03/26/2016 23:07:24 ANS0361I DIAG: The VDDK Utility Thread may be
> >> hung processing another request.
> >> 03/26/2016 23:07:24 ANS0361I DIAG: Waiting up to another 3600000
> >> milliseconds to process a request to VddkFullVMOpenCloseSingleVMDK
> >> for vm 'POKCBSAPAP62'.
> >> 03/26/2016 23:27:58 ANS0361I DIAG: The VDDK Utility Thread may be
> >> hung processing another request.
> >> 03/26/2016 23:27:58 ANS0361I DIAG: Waiting up to another 3600000
> >> milliseconds to process a request to VddkFullVMCloseVMDKs for vm
> >> 'pokcsmetvm01'.
> >> 03/26/2016 23:28:20 ANS0361I DIAG: The VDDK Utility Thread may be
> >> hung processing another request.
> >> 03/26/2016 23:28:20 ANS0361I DIAG: Waiting up to another 3600000
> >> milliseconds to process a request to VddkFullVMOpenCloseSingleVMDK
> >> for vm 'pokcdmetvm01'.
> >> 03/26/2016 23:28:25 ANS0361I DIAG: The VDDK Utility Thread may be
> >> hung processing another request.
> >> 03/26/2016 23:28:25 ANS0361I DIAG: Waiting up to another 3600000
> >> milliseconds to process a request to VddkFullVMOpenCloseSingleVMDK
> >> for vm 'POKCDSAPAP31'.
> >> 03/26/2016 23:29:37 ANS0361I DIAG: The VDDK Utility Thread may be
> >> hung processing another request.
> >> 03/26/2016 23:29:37 ANS0361I DIAG: Waiting up to another 3600000
> >> milliseconds to process a request to VddkFullVMCloseVMDKs for vm
> >> 'pokctcvs01'.
> >> 03/27/2016 00:07:17 ANS0361I DIAG: The VDDK Utility Thread may be
> >> hung processing another request.
> >>
> >> Thanks & Regards,
> >>
> >> *Srikanth kola*
> >> Backup & Recovery
> >> IBMIndiaPvt Ltd, Bangalore
> >> Mobile: +91 9885473450
> >>
> >
> >
>
> ------------------
>
>
> Information confidentielle : Le présent message, ainsi que tout fichier
> qui y est joint, est envoyé à l'intention exclusive de son ou de ses
> destinataires; il est de nature confidentielle et peut constituer une
> information privilégiée. Nous avertissons toute personne autre que le
> destinataire prévu que tout examen, réacheminement, impression, copie,
> distribution ou autre utilisation de ce message et de tout fichier qui y
> est joint est strictement interdit, à moins d'avoir obtenu le consentement
> du destinataire. Si vous n'êtes pas le destinataire prévu, veuillez en
> aviser immédiatement l'expéditeur par retour de courriel et supprimer ce
> message et tout document joint de votre système. Merci.
>

<Prev in Thread] Current Thread [Next in Thread>