Bacula-users

[Bacula-users] bacula hang waiting for storage

2008-11-26 15:25:48
Subject: [Bacula-users] bacula hang waiting for storage
From: Bob Hetzel <beh AT case DOT edu>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 26 Nov 2008 15:22:55 -0500
I've got bacula currently in a hung state with the following interesting 
info.  When I run a status storage produces the following...


Automatically selected Storage: Dell-PV136T
Connecting to Storage daemon Dell-PV136T at gyrus:9103

gyrus-sd Version: 2.4.3 (10 October 2008) i686-pc-linux-gnu suse 10.2
Daemon started 25-Nov-08 19:20, 59 Jobs run since started.
  Heap: heap=3,756,032 smbytes=3,519,564 max_bytes=3,684,397 bufs=555 
max_bufs=557
Sizes: boffset_t=8 size_t=4 int32_t=4 int64_t=8

Running Jobs:
Writing: Incremental Backup job axh93-gx270 JobId=45634 Volume="LTO261L2"
     pool="Default" device="IBMLTO2-3" (/dev/nst2)
     spooling=0 despooling=0 despool_wait=1
     Files=78 Bytes=21,123,239 Bytes/sec=2,337
     FDReadSeqNo=970 in_msg=750 out_msg=9 fd=20
Writing: Incremental Backup job bxn4-gx280 JobId=45641 Volume="LTO261L2"
     pool="Default" device="IBMLTO2-3" (/dev/nst2)
     spooling=0 despooling=0 despool_wait=1
     Files=155 Bytes=2,925,138,595 Bytes/sec=323,648
     FDReadSeqNo=45,916 in_msg=45480 out_msg=9 fd=35
Writing: Incremental Backup job cdking JobId=45646 Volume="LTO261L2"
     pool="Default" device="IBMLTO2-3" (/dev/nst2)
     spooling=0 despooling=0 despool_wait=1
     Files=88 Bytes=11,846,912 Bytes/sec=1,310
     FDReadSeqNo=920 in_msg=672 out_msg=9 fd=23
Writing: Incremental Backup job ceg3-d810 JobId=45648 Volume="LTO253L2"
     pool="Default" device="IBMLTO2-2" (/dev/nst1)
     spooling=0 despooling=1 despool_wait=0
     Files=35 Bytes=1,391,695,993 Bytes/sec=176,588
     FDReadSeqNo=21,542 in_msg=21439 out_msg=9 fd=36
Writing: Incremental Backup job clifford3 JobId=45651 Volume="LTO261L2"
     pool="Default" device="IBMLTO2-3" (/dev/nst2)
     spooling=0 despooling=0 despool_wait=0
     Files=0 Bytes=0 Bytes/sec=0
     FDReadSeqNo=6 in_msg=6 out_msg=4 fd=32
Writing: Incremental Backup job cxj57-gx270 JobId=45657 Volume="LTO261L2"
     pool="Default" device="IBMLTO2-3" (/dev/nst2)
     spooling=0 despooling=0 despool_wait=0
     Files=0 Bytes=0 Bytes/sec=0
     FDReadSeqNo=6 in_msg=6 out_msg=4 fd=33
Writing: Incremental Backup job dxa2-d630 JobId=45665 Volume="LTO261L2"
     pool="Default" device="IBMLTO2-3" (/dev/nst2)
     spooling=0 despooling=0 despool_wait=0
     Files=0 Bytes=0 Bytes/sec=0
     FDReadSeqNo=6 in_msg=6 out_msg=4 fd=17
Writing: Incremental Backup job educationdean JobId=45667 Volume=""
     pool="Default" device="IBMLTO2-1" (/dev/nst0)
     spooling=0 despooling=0 despool_wait=0
     Files=0 Bytes=0 Bytes/sec=0
     FDSocket closed
====

Jobs waiting to reserve a drive:
    3605 JobId=45667 wants free drive but device "IBMLTO2-1" (/dev/nst0) 
is busy.
====
[terminated jobs info snipped out]
Device status:
Autochanger "Dell-PV136T" with devices:
    "IBMLTO2-1" (/dev/nst0)
    "IBMLTO2-2" (/dev/nst1)
    "IBMLTO2-3" (/dev/nst2)
Device "IBMLTO2-1" (/dev/nst0) is mounted with:
     Volume:      LTO342L2
     Pool:        Default
     Media type:  LTO-2
     Slot 32 is loaded in drive 0.
     Total Bytes=11,991,168,000 Blocks=185,874 Bytes/block=64,512
     Positioned at File=14 Block=0
Device "IBMLTO2-2" (/dev/nst1) is mounted with:
     Volume:      LTO253L2
     Pool:        Default
     Media type:  LTO-2
     Slot 48 is loaded in drive 1.
     Total Bytes=2,193,408 Blocks=33 Bytes/block=66,466
     Positioned at File=1 Block=0
Device "IBMLTO2-3" (/dev/nst2) is not open.
     Device is being initialized.
     Drive 2 status unknown.
====

Used Volume status:
[nothing further and the bconsole program hangs here]

Note that the last Writing line has no volume listed.  The odd thing is 
that there actually is a tape in IBMLTO2-1.  There's no tape in drive 
IBMLTO2-3.  The pool apparently needs another appendable volume and 
there are several available in the Scratch pool but bacula is stuck.

I tried to mount a volume into the empty drive and got back the following...
*mount slot=61 drive=2
Automatically selected Storage: Dell-PV136T
3001 Device "IBMLTO2-3" (/dev/nst2) is doing acquire.

Does anybody have any idea what to do to further troubleshoot this?  I 
have had some other instances of bacula getting hung up and so I have 
already previously applied the 2.4.3-orphaned-jobs.patch

    Bob

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>