Amanda-Users

Re: no space left on device during amflush

2005-10-05 21:05:14
Subject: Re: no space left on device during amflush
From: Paddy Sreenivasan <paddy.sreenivasan AT gmail DOT com>
To: Shang-Lin Chen <schen AT gps.caltech DOT edu>
Date: Wed, 5 Oct 2005 17:47:47 -0700
Use ipcrm -m <id> to remove them and try amflush again.

It appears taper is not cleaning up properly. tape_writer_side should call
detach_buffers() in case 'Q' to explicitly detach the shared memory (might
be required for Solaris)

Paddy


On 10/5/05, Shang-Lin Chen <schen AT gps.caltech DOT edu> wrote:
On Wed, 2005-10-05 at 16:39 -0700, Paddy Sreenivasan wrote:
> Maybe some other processes is using more shared memory or shared
> memory segments.
> Check output of ipcs (ipcs equivalent in Solaris)
>
This is the output I'm getting from ipcs -m:
IPC status from <running system> as of Wed Oct  5 17:25:34 PDT 2005
T         ID      KEY        MODE        OWNER    GROUP
Shared Memory:
m          0   0x50000674 --rw-r--r--     root     root
m          1   0          --rw-------   amanda   amanda
m       2702   0          --rw-rw-rw-   webmgr    staff
m       1203   0          --rw-------   amanda   amanda
m        104   0          --rw-------   amanda   amanda
m          5   0          --rw-------   amanda   amanda
m        106   0          --rw-------   amanda   amanda
m        207   0          --rw-------   amanda   amanda
m          8   0          --rw-------   amanda   amanda
m        109   0          --rw-------   amanda   amanda
m        210   0          --rw-------   amanda   amanda
m        511   0          --rw-------   amanda   amanda
m        312   0          --rw-------   amanda   amanda
m       1213   0          --rw-------   amanda   amanda
m       1014   0          --rw-------   amanda   amanda
m        115   0          --rw-------   amanda   amanda
m        716   0          --rw-------   amanda   amanda
m         17   0          --rw-------   amanda   amanda
m        118   0          --rw-------   amanda   amanda
m        319   0          --rw-------   amanda   amanda
m        620   0          --rw-------   amanda   amanda
m        221   0          --rw-------   amanda   amanda
m        622   0          --rw-------   amanda   amanda
m        523   0          --rw-------   amanda   amanda
m        224   0          --rw-------   amanda   amanda
m        125   0          --rw-------   amanda   amanda
m        126   0          --rw-------   amanda   amanda
m        227   0          --rw-------   amanda   amanda
m        128   0          --rw-------   amanda   amanda
m       1029   0          --rw-------   amanda   amanda
m       2330   0          --rw-------   amanda   amanda
m        231   0          --rw-------   amanda   amanda
m       1532   0          --rw-------   amanda   amanda
m         33   0          --rw-------   amanda   amanda
m       2034   0          --rw-------   amanda   amanda
m        235   0          --rw-------   amanda   amanda
m        436   0          --rw-------   amanda   amanda
m       1037   0          --rw-------   amanda   amanda
m        638   0          --rw-------   amanda   amanda
m        539   0          --rw-------   amanda   amanda
m        140   0          --rw-------   amanda   amanda
m        141   0          --rw-------   amanda   amanda
m       1542   0          --rw-------   amanda   amanda
m        643   0          --rw-------   amanda   amanda
m        144   0          --rw-------   amanda   amanda
m        345   0          --rw-------   amanda   amanda
m       1346   0          --rw-------   amanda   amanda
m       1447   0          --rw-------   amanda   amanda
m        848   0          --rw-------   amanda   amanda
m       1849   0          --rw-------   amanda   amanda
m         50   0          --rw-------   amanda   amanda
m        151   0          --rw-------   amanda   amanda
m        152   0          --rw-------   amanda   amanda
m        153   0          --rw-------   amanda   amanda
m        154   0          --rw-------   amanda   amanda
m        155   0          --rw-------   amanda   amanda
m        556   0          --rw-------   amanda   amanda
m        257   0          --rw-------   amanda   amanda
m        258   0          --rw-------   amanda   amanda
m        659   0          --rw-------   amanda   amanda
m       1160   0          --rw-------   amanda   amanda
m        261   0          --rw-------   amanda   amanda
m        262   0          --rw-------   amanda   amanda
m        363   0          --rw-------   amanda   amanda
m         64   0          --rw-------   amanda   amanda
m        265   0          --rw-------   amanda   amanda
m         66   0          --rw-------   amanda   amanda
m        267   0          --rw-------   amanda   amanda
m        468   0          --rw-------   amanda   amanda
m        769   0          --rw-------   amanda   amanda
m        170   0          --rw-------   amanda   amanda
m        171   0          --rw-------   amanda   amanda
m        172   0          --rw-------   amanda   amanda
m        973   0          --rw-------   amanda   amanda
m       1074   0          --rw-------   amanda   amanda
m       1275   0          --rw-------   amanda   amanda
m       4176   0          --rw-------   amanda   amanda
m        577   0          --rw-------   amanda   amanda
m        878   0          --rw-------   amanda   amanda
m        479   0          --rw-------   amanda   amanda
m         80   0          --rw-------   amanda   amanda
m        381   0          --rw-------   amanda   amanda
m        382   0          --rw-------   amanda   amanda
m        183   0          --rw-------   amanda   amanda
m         84   0          --rw-------   amanda   amanda
m        185   0          --rw-------   amanda   amanda
m       1186   0          --rw-------   amanda   amanda
m       1887   0          --rw-------   amanda   amanda
m       4688   0          --rw-------   amanda   amanda
m        689   0          --rw-------   amanda   amanda
m        190   0          --rw-------   amanda   amanda
m        191   0          --rw-------   amanda   amanda
m        692   0          --rw-------   amanda   amanda
m        693   0          --rw-------   amanda   amanda
m       1094   0          --rw-------   amanda   amanda
m        195   0          --rw-------   amanda   amanda
m       1496   0          --rw-------   amanda   amanda
m       1397   0          --rw-------   amanda   amanda
m        298   0          --rw-------   amanda   amanda
m        499   0          --rw-------   amanda   amanda

However, when I did
ps -ef | grep amanda
I couldn't find any processes running as amanda.  I also tried running
amflush using the next tape in the cycle (I couldn't find any spare
tapes that worked with the drive) and got the same error about no space
left on the device.

Shang-Lin

> Paddy
>
> On 10/5/05, Shang-Lin Chen <schen AT gps.caltech DOT edu> wrote:
>         The Amanda server is running Solaris, so it doesn't
>         have /proc. But I
>         checked /etc/system and found this:
>         set shmsys:shminfo_shmmax=268435456
>         set shmsys:shminfo_shmmin=1
>         set shmsys:shminfo_shmmni=100
>         set shmsys:shminfo_shmseg=10
>         set semsys:seminfo_semmns=70
>         set semsys:seminfo_semmni=100
>         set semsys:seminfo_semmsl=70
>         set semsys:seminfo_semopm=100
>         set semsys:seminfo_semvmx=32767
>         forceload: sys/shmsys
>
>         If these settings worked before, how come amflush is failing
>         now?
>
>         Thanks,
>         Shang-Lin
>
>         On Wed, 2005-10-05 at 14:54 -0700, Paddy Sreenivasan wrote:
>         > Can you check the number of shared memory segments and
>         shared memory
>         > allocated in the system? You may have to adjust the values
>         of
>         > shmall and shmmni (cat /proc/sys/kernel/shmmni)
>         >
>         > Paddy
>         >
>         > On 10/5/05, Shang-Lin Chen < schen AT gps.caltech DOT edu> wrote:
>         >         I added
>         >         tapebufs 50
>         >         to amanda.conf after the labelstr line, and this is
>         the output
>         >         I'm
>         >         getting now:
>         >
>         >
>         >         FAILURE AND STRANGE DUMP SUMMARY:
>         >           taper: FATAL shmget: (1638600 tapebufs) No space
>         left on
>         >         device
>         >           amflush: ERROR Cannot flush without tape.  Try
>         again.
>         >
>         >         Shang-Lin
>         >
>         >         On Wed, 2005-10-05 at 17:02 -0400, Scott R. Burns
>         wrote:
>         >         > Try
>         >         >
>         >         > tapebufs 50
>         >         >
>         >         > In your amanda.conf file
>         >         >
>         >         > Scott...
>         >         >
>         >         > Scott R. Burns
>         >         > NETCON Technologies Incorporated
>         >         > Suite 135 - 4474 Blakie Road
>         >         > London, Ontario, Canada
>         >         > N6L 1G7
>         >         > Voice: +1.519.652.0401
>         >         > Fax: +1.519.652.9275
>         >         > Web: www.netcontech.com
>         >         >
>         >         > -----Original Message-----
>         >         > From: Shang-Lin Chen [mailto:
>         schen AT gps.caltech DOT edu]
>         >         > Sent: Wednesday, October 05, 2005 4:21 PM
>         >         > To: Paul Bijnens
>         >         > Subject: Re: no space left on device during
>         amflush
>         >         >
>         >         >
>         >         > > Shang-Lin Chen wrote:
>         >         > > >   taper: FATAL shmget: (655440 tapebufs) No
>         space left
>         >         on device
>         >         > >
>         >         > > The default value for tapebufs is 20 (buffers of
>         32K bytes
>         >         each).
>         >         > > You probably got some weird value in amanda.conf
>         for that
>         >         parameter.
>         >         > >
>         >         > > The above value results if over 20 GByte of RAM.
>         >         > >
>         >         > My amanda.conf file doesn't set anything for
>         tapebufs, and
>         >         I've never
>         >         > had this problem before. How do I change the
>         tapebufs
>         >         setting?
>         >         >
>         >         > Thanks,
>         >         > Shang-Lin
>         >         > >
>         >         >
>         >
>         >
>
>