Amanda-Users

Re: no space left on device during amflush

2005-10-05 23:49:17
Subject: Re: no space left on device during amflush
From: Shang-Lin Chen <schen AT gps.caltech DOT edu>
To: Paddy Sreenivasan <paddy.sreenivasan AT gmail DOT com>
Date: Wed, 05 Oct 2005 20:34:23 -0700
Paddy Sreenivasan wrote:

Use ipcrm -m <id> to remove them and try amflush again.

It worked! Thanks for your help. :-)

Shang-Lin

It appears taper is not cleaning up properly. tape_writer_side should call
detach_buffers() in case 'Q' to explicitly detach the shared memory (might
be required for Solaris)

Paddy


On 10/5/05, *Shang-Lin Chen* <schen AT gps.caltech DOT edu <mailto:schen AT gps.caltech DOT edu>> wrote:

    On Wed, 2005-10-05 at 16:39 -0700, Paddy Sreenivasan wrote:
    > Maybe some other processes is using more shared memory or shared
    > memory segments.
    > Check output of ipcs (ipcs equivalent in Solaris)
    >
    This is the output I'm getting from ipcs -m:
    IPC status from <running system> as of Wed Oct  5 17:25:34 PDT 2005
    T         ID      KEY        MODE        OWNER    GROUP
    Shared Memory:
    m          0   0x50000674 --rw-r--r--     root     root
    m          1   0          --rw-------   amanda   amanda
    m       2702   0          --rw-rw-rw-   webmgr    staff
    m       1203   0          --rw-------   amanda   amanda
    m        104   0          --rw-------   amanda   amanda
    m          5   0          --rw-------   amanda   amanda
    m        106   0          --rw-------   amanda   amanda
    m        207   0          --rw-------   amanda   amanda
    m          8   0          --rw-------   amanda   amanda
    m        109   0          --rw-------   amanda   amanda
    m        210   0          --rw-------   amanda   amanda
    m        511   0          --rw-------   amanda   amanda
    m        312   0          --rw-------   amanda   amanda
    m       1213   0          --rw-------   amanda   amanda
    m       1014   0          --rw-------   amanda   amanda
    m        115   0          --rw-------   amanda   amanda
    m        716   0          --rw-------   amanda   amanda
    m         17   0          --rw-------   amanda   amanda
    m        118   0          --rw-------   amanda   amanda
    m        319   0          --rw-------   amanda   amanda
    m        620   0          --rw-------   amanda   amanda
    m        221   0          --rw-------   amanda   amanda
    m        622   0          --rw-------   amanda   amanda
    m        523   0          --rw-------   amanda   amanda
    m        224   0          --rw-------   amanda   amanda
    m        125   0          --rw-------   amanda   amanda
    m        126   0          --rw-------   amanda   amanda
    m        227   0          --rw-------   amanda   amanda
    m        128   0          --rw-------   amanda   amanda
    m       1029   0          --rw-------   amanda   amanda
    m       2330   0          --rw-------   amanda   amanda
    m        231   0          --rw-------   amanda   amanda
    m       1532   0          --rw-------   amanda   amanda
    m         33   0          --rw-------   amanda   amanda
    m       2034   0          --rw-------   amanda   amanda
    m        235   0          --rw-------   amanda   amanda
    m        436   0          --rw-------   amanda   amanda
    m       1037   0          --rw-------   amanda   amanda
    m        638   0          --rw-------   amanda   amanda
    m        539   0          --rw-------   amanda   amanda
    m        140   0          --rw-------   amanda   amanda
    m        141   0          --rw-------   amanda   amanda
    m       1542   0          --rw-------   amanda   amanda
    m        643   0          --rw-------   amanda   amanda
    m        144   0          --rw-------   amanda   amanda
    m        345   0          --rw-------   amanda   amanda
    m       1346   0          --rw-------   amanda   amanda
    m       1447   0          --rw-------   amanda   amanda
    m        848   0          --rw-------   amanda   amanda
    m       1849   0          --rw-------   amanda   amanda
    m         50   0          --rw-------   amanda   amanda
    m        151   0          --rw-------   amanda   amanda
    m        152   0          --rw-------   amanda   amanda
    m        153   0          --rw-------   amanda   amanda
    m        154   0          --rw-------   amanda   amanda
    m        155   0          --rw-------   amanda   amanda
    m        556   0          --rw-------   amanda   amanda
    m        257   0          --rw-------   amanda   amanda
    m        258   0          --rw-------   amanda   amanda
    m        659   0          --rw-------   amanda   amanda
    m       1160   0          --rw-------   amanda   amanda
    m        261   0          --rw-------   amanda   amanda
    m        262   0          --rw-------   amanda   amanda
    m        363   0          --rw-------   amanda   amanda
    m         64   0          --rw-------   amanda   amanda
    m        265   0          --rw-------   amanda   amanda
    m         66   0          --rw-------   amanda   amanda
    m        267   0          --rw-------   amanda   amanda
    m        468   0          --rw-------   amanda   amanda
    m        769   0          --rw-------   amanda   amanda
    m        170   0          --rw-------   amanda   amanda
    m        171   0          --rw-------   amanda   amanda
    m        172   0          --rw-------   amanda   amanda
    m        973   0          --rw-------   amanda   amanda
    m       1074   0          --rw-------   amanda   amanda
    m       1275   0          --rw-------   amanda   amanda
    m       4176   0          --rw-------   amanda   amanda
    m        577   0          --rw-------   amanda   amanda
    m        878   0          --rw-------   amanda   amanda
    m        479   0          --rw-------   amanda   amanda
    m         80   0          --rw-------   amanda   amanda
    m        381   0          --rw-------   amanda   amanda
    m        382   0          --rw-------   amanda   amanda
    m        183   0          --rw-------   amanda   amanda
    m         84   0          --rw-------   amanda   amanda
    m        185   0          --rw-------   amanda   amanda
    m       1186   0          --rw-------   amanda   amanda
    m       1887   0          --rw-------   amanda   amanda
    m       4688   0          --rw-------   amanda   amanda
    m        689   0          --rw-------   amanda   amanda
    m        190   0          --rw-------   amanda   amanda
    m        191   0          --rw-------   amanda   amanda
    m        692   0          --rw-------   amanda   amanda
    m        693   0          --rw-------   amanda   amanda
    m       1094   0          --rw-------   amanda   amanda
    m        195   0          --rw-------   amanda   amanda
    m       1496   0          --rw-------   amanda   amanda
    m       1397   0          --rw-------   amanda   amanda
    m        298   0          --rw-------   amanda   amanda
    m        499   0          --rw-------   amanda   amanda

    However, when I did
    ps -ef | grep amanda
    I couldn't find any processes running as amanda.  I also tried running
    amflush using the next tape in the cycle (I couldn't find any spare
    tapes that worked with the drive) and got the same error about no
    space
    left on the device.

    Shang-Lin

    > Paddy
    >
    > On 10/5/05, Shang-Lin Chen <schen AT gps.caltech DOT edu
    <mailto:schen AT gps.caltech DOT edu>> wrote:
    >         The Amanda server is running Solaris, so it doesn't
    >         have /proc. But I
    >         checked /etc/system and found this:
    >         set shmsys:shminfo_shmmax=268435456
    >         set shmsys:shminfo_shmmin=1
    >         set shmsys:shminfo_shmmni=100
    >         set shmsys:shminfo_shmseg=10
    >         set semsys:seminfo_semmns=70
    >         set semsys:seminfo_semmni=100
    >         set semsys:seminfo_semmsl=70
    >         set semsys:seminfo_semopm=100
    >         set semsys:seminfo_semvmx=32767
    >         forceload: sys/shmsys
    >
    >         If these settings worked before, how come amflush is failing
    >         now?
    >
    >         Thanks,
    >         Shang-Lin
    >
    >         On Wed, 2005-10-05 at 14:54 -0700, Paddy Sreenivasan wrote:
    >         > Can you check the number of shared memory segments and
    >         shared memory
    >         > allocated in the system? You may have to adjust the
    values
    >         of
    >         > shmall and shmmni (cat /proc/sys/kernel/shmmni)
    >         >
    >         > Paddy
    >         >
    >         > On 10/5/05, Shang-Lin Chen < schen AT gps.caltech DOT edu
    <mailto:schen AT gps.caltech DOT edu>> wrote:
    >         >         I added
    >         >         tapebufs 50
    >         >         to amanda.conf after the labelstr line, and
    this is
    >         the output
    >         >         I'm
    >         >         getting now:
    >         >
    >         >
    >         >         FAILURE AND STRANGE DUMP SUMMARY:
    >         >           taper: FATAL shmget: (1638600 tapebufs) No space
    >         left on
    >         >         device
    >         >           amflush: ERROR Cannot flush without tape.  Try
    >         again.
    >         >
    >         >         Shang-Lin
    >         >
    >         >         On Wed, 2005-10-05 at 17:02 -0400, Scott R. Burns
    >         wrote:
    >         >         > Try
    >         >         >
    >         >         > tapebufs 50
    >         >         >
    >         >         > In your amanda.conf file
    >         >         >
    >         >         > Scott...
    >         >         >
    >         >         > Scott R. Burns
    >         >         > NETCON Technologies Incorporated
    >         >         > Suite 135 - 4474 Blakie Road
    >         >         > London, Ontario, Canada
    >         >         > N6L 1G7
    >         >         > Voice: +1.519.652.0401
    >         >         > Fax: +1.519.652.9275
    >         >         > Web: www.netcontech.com
    <http://www.netcontech.com>
    >         >         >
    >         >         > -----Original Message-----
    >         >         > From: Shang-Lin Chen [mailto:
    >         schen AT gps.caltech DOT edu <mailto:schen AT gps.caltech DOT 
edu>]
    >         >         > Sent: Wednesday, October 05, 2005 4:21 PM
    >         >         > To: Paul Bijnens
    >         >         > Subject: Re: no space left on device during
    >         amflush
    >         >         >
    >         >         >
    >         >         > > Shang-Lin Chen wrote:
    >         >         > > >   taper: FATAL shmget: (655440 tapebufs) No
    >         space left
    >         >         on device
    >         >         > >
    >         >         > > The default value for tapebufs is 20
    (buffers of
    >         32K bytes
    >         >         each).
    >         >         > > You probably got some weird value in
    amanda.conf
    >         for that
    >         >         parameter.
    >         >         > >
    >         >         > > The above value results if over 20 GByte
    of RAM.
    >         >         > >
    >         >         > My amanda.conf file doesn't set anything for
    >         tapebufs, and
    >         >         I've never
    >         >         > had this problem before. How do I change the
    >         tapebufs
    >         >         setting?
    >         >         >
    >         >         > Thanks,
    >         >         > Shang-Lin
    >         >         > >
    >         >         >
    >         >
    >         >
    >
    >