Amanda-Users

Re: no space left on device during amflush

2005-10-06 00:22:30
Subject: Re: no space left on device during amflush
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: amanda-users AT amanda DOT org
Date: Thu, 06 Oct 2005 00:07:51 -0400
On Wednesday 05 October 2005 23:34, Shang-Lin Chen wrote:
>Paddy Sreenivasan wrote:
>> Use ipcrm -m <id> to remove them and try amflush again.
>
>It worked! Thanks for your help. :-)
>
>Shang-Lin

Yes, but I think this one bears watching.  Perhaps a miss-match in the
memory allocations methods amanda uses vs the Solaris methods is
causeing them to be left lying around.  Not a Good Thing(TM).

Have cron run this "ipcs -m" sometime after amanda should be done, and
mail you the results every morning.

>> It appears taper is not cleaning up properly. tape_writer_side should
>> call detach_buffers() in case 'Q' to explicitly detach the shared
>> memory (might be required for Solaris)
>>
>> Paddy
>>
>>
>> On 10/5/05, *Shang-Lin Chen* <schen AT gps.caltech DOT edu
>> <mailto:schen AT gps.caltech DOT edu>> wrote:
>>
>>     On Wed, 2005-10-05 at 16:39 -0700, Paddy Sreenivasan wrote:
>>     > Maybe some other processes is using more shared memory or shared
>>     > memory segments.
>>     > Check output of ipcs (ipcs equivalent in Solaris)
>>
>>     This is the output I'm getting from ipcs -m:
>>     IPC status from <running system> as of Wed Oct  5 17:25:34 PDT
>> 2005 T         ID      KEY        MODE        OWNER    GROUP
>>     Shared Memory:
>>     m          0   0x50000674 --rw-r--r--     root     root
>>     m          1   0          --rw-------   amanda   amanda
>>     m       2702   0          --rw-rw-rw-   webmgr    staff
>>     m       1203   0          --rw-------   amanda   amanda
>>     m        104   0          --rw-------   amanda   amanda
>>     m          5   0          --rw-------   amanda   amanda
>>     m        106   0          --rw-------   amanda   amanda
>>     m        207   0          --rw-------   amanda   amanda
>>     m          8   0          --rw-------   amanda   amanda
>>     m        109   0          --rw-------   amanda   amanda
>>     m        210   0          --rw-------   amanda   amanda
>>     m        511   0          --rw-------   amanda   amanda
>>     m        312   0          --rw-------   amanda   amanda
>>     m       1213   0          --rw-------   amanda   amanda
>>     m       1014   0          --rw-------   amanda   amanda
>>     m        115   0          --rw-------   amanda   amanda
>>     m        716   0          --rw-------   amanda   amanda
>>     m         17   0          --rw-------   amanda   amanda
>>     m        118   0          --rw-------   amanda   amanda
>>     m        319   0          --rw-------   amanda   amanda
>>     m        620   0          --rw-------   amanda   amanda
>>     m        221   0          --rw-------   amanda   amanda
>>     m        622   0          --rw-------   amanda   amanda
>>     m        523   0          --rw-------   amanda   amanda
>>     m        224   0          --rw-------   amanda   amanda
>>     m        125   0          --rw-------   amanda   amanda
>>     m        126   0          --rw-------   amanda   amanda
>>     m        227   0          --rw-------   amanda   amanda
>>     m        128   0          --rw-------   amanda   amanda
>>     m       1029   0          --rw-------   amanda   amanda
>>     m       2330   0          --rw-------   amanda   amanda
>>     m        231   0          --rw-------   amanda   amanda
>>     m       1532   0          --rw-------   amanda   amanda
>>     m         33   0          --rw-------   amanda   amanda
>>     m       2034   0          --rw-------   amanda   amanda
>>     m        235   0          --rw-------   amanda   amanda
>>     m        436   0          --rw-------   amanda   amanda
>>     m       1037   0          --rw-------   amanda   amanda
>>     m        638   0          --rw-------   amanda   amanda
>>     m        539   0          --rw-------   amanda   amanda
>>     m        140   0          --rw-------   amanda   amanda
>>     m        141   0          --rw-------   amanda   amanda
>>     m       1542   0          --rw-------   amanda   amanda
>>     m        643   0          --rw-------   amanda   amanda
>>     m        144   0          --rw-------   amanda   amanda
>>     m        345   0          --rw-------   amanda   amanda
>>     m       1346   0          --rw-------   amanda   amanda
>>     m       1447   0          --rw-------   amanda   amanda
>>     m        848   0          --rw-------   amanda   amanda
>>     m       1849   0          --rw-------   amanda   amanda
>>     m         50   0          --rw-------   amanda   amanda
>>     m        151   0          --rw-------   amanda   amanda
>>     m        152   0          --rw-------   amanda   amanda
>>     m        153   0          --rw-------   amanda   amanda
>>     m        154   0          --rw-------   amanda   amanda
>>     m        155   0          --rw-------   amanda   amanda
>>     m        556   0          --rw-------   amanda   amanda
>>     m        257   0          --rw-------   amanda   amanda
>>     m        258   0          --rw-------   amanda   amanda
>>     m        659   0          --rw-------   amanda   amanda
>>     m       1160   0          --rw-------   amanda   amanda
>>     m        261   0          --rw-------   amanda   amanda
>>     m        262   0          --rw-------   amanda   amanda
>>     m        363   0          --rw-------   amanda   amanda
>>     m         64   0          --rw-------   amanda   amanda
>>     m        265   0          --rw-------   amanda   amanda
>>     m         66   0          --rw-------   amanda   amanda
>>     m        267   0          --rw-------   amanda   amanda
>>     m        468   0          --rw-------   amanda   amanda
>>     m        769   0          --rw-------   amanda   amanda
>>     m        170   0          --rw-------   amanda   amanda
>>     m        171   0          --rw-------   amanda   amanda
>>     m        172   0          --rw-------   amanda   amanda
>>     m        973   0          --rw-------   amanda   amanda
>>     m       1074   0          --rw-------   amanda   amanda
>>     m       1275   0          --rw-------   amanda   amanda
>>     m       4176   0          --rw-------   amanda   amanda
>>     m        577   0          --rw-------   amanda   amanda
>>     m        878   0          --rw-------   amanda   amanda
>>     m        479   0          --rw-------   amanda   amanda
>>     m         80   0          --rw-------   amanda   amanda
>>     m        381   0          --rw-------   amanda   amanda
>>     m        382   0          --rw-------   amanda   amanda
>>     m        183   0          --rw-------   amanda   amanda
>>     m         84   0          --rw-------   amanda   amanda
>>     m        185   0          --rw-------   amanda   amanda
>>     m       1186   0          --rw-------   amanda   amanda
>>     m       1887   0          --rw-------   amanda   amanda
>>     m       4688   0          --rw-------   amanda   amanda
>>     m        689   0          --rw-------   amanda   amanda
>>     m        190   0          --rw-------   amanda   amanda
>>     m        191   0          --rw-------   amanda   amanda
>>     m        692   0          --rw-------   amanda   amanda
>>     m        693   0          --rw-------   amanda   amanda
>>     m       1094   0          --rw-------   amanda   amanda
>>     m        195   0          --rw-------   amanda   amanda
>>     m       1496   0          --rw-------   amanda   amanda
>>     m       1397   0          --rw-------   amanda   amanda
>>     m        298   0          --rw-------   amanda   amanda
>>     m        499   0          --rw-------   amanda   amanda
>>
>>     However, when I did
>>     ps -ef | grep amanda
>>     I couldn't find any processes running as amanda.  I also tried
>> running amflush using the next tape in the cycle (I couldn't find any
>> spare tapes that worked with the drive) and got the same error about
>> no space
>>     left on the device.
>>
>>     Shang-Lin
>>
>>     > Paddy
>>     >
>>     > On 10/5/05, Shang-Lin Chen <schen AT gps.caltech DOT edu
>>
>>     <mailto:schen AT gps.caltech DOT edu>> wrote:
>>     >         The Amanda server is running Solaris, so it doesn't
>>     >         have /proc. But I
>>     >         checked /etc/system and found this:
>>     >         set shmsys:shminfo_shmmax=268435456
>>     >         set shmsys:shminfo_shmmin=1
>>     >         set shmsys:shminfo_shmmni=100
>>     >         set shmsys:shminfo_shmseg=10
>>     >         set semsys:seminfo_semmns=70
>>     >         set semsys:seminfo_semmni=100
>>     >         set semsys:seminfo_semmsl=70
>>     >         set semsys:seminfo_semopm=100
>>     >         set semsys:seminfo_semvmx=32767
>>     >         forceload: sys/shmsys
>>     >
>>     >         If these settings worked before, how come amflush is
>>     > failing now?
>>     >
>>     >         Thanks,
>>     >         Shang-Lin
>>     >
>>     >         On Wed, 2005-10-05 at 14:54 -0700, Paddy Sreenivasan 
wrote:
>>     >         > Can you check the number of shared memory segments and
>>     >
>>     >         shared memory
>>     >
>>     >         > allocated in the system? You may have to adjust the
>>
>>     values
>>
>>     >         of
>>     >
>>     >         > shmall and shmmni (cat /proc/sys/kernel/shmmni)
>>     >         >
>>     >         > Paddy
>>     >         >
>>     >         > On 10/5/05, Shang-Lin Chen < schen AT gps.caltech DOT edu
>>
>>     <mailto:schen AT gps.caltech DOT edu>> wrote:
>>     >         >         I added
>>     >         >         tapebufs 50
>>     >         >         to amanda.conf after the labelstr line, and
>>
>>     this is
>>
>>     >         the output
>>     >
>>     >         >         I'm
>>     >         >         getting now:
>>     >         >
>>     >         >
>>     >         >         FAILURE AND STRANGE DUMP SUMMARY:
>>     >         >           taper: FATAL shmget: (1638600 tapebufs) No
>>     >         > space
>>     >
>>     >         left on
>>     >
>>     >         >         device
>>     >         >           amflush: ERROR Cannot flush without tape. 
>>     >         > Try
>>     >
>>     >         again.
>>     >
>>     >         >         Shang-Lin
>>     >         >
>>     >         >         On Wed, 2005-10-05 at 17:02 -0400, Scott R.
>>     >         > Burns
>>     >
>>     >         wrote:
>>     >         >         > Try
>>     >         >         >
>>     >         >         > tapebufs 50
>>     >         >         >
>>     >         >         > In your amanda.conf file
>>     >         >         >
>>     >         >         > Scott...
>>     >         >         >
>>     >         >         > Scott R. Burns
>>     >         >         > NETCON Technologies Incorporated
>>     >         >         > Suite 135 - 4474 Blakie Road
>>     >         >         > London, Ontario, Canada
>>     >         >         > N6L 1G7
>>     >         >         > Voice: +1.519.652.0401
>>     >         >         > Fax: +1.519.652.9275
>>     >         >         > Web: www.netcontech.com
>>
>>     <http://www.netcontech.com>
>>
>>     >         >         > -----Original Message-----
>>     >         >         > From: Shang-Lin Chen [mailto:
>>     >
>>     >         schen AT gps.caltech DOT edu <mailto:schen AT gps.caltech DOT 
>> edu>]
>>     >
>>     >         >         > Sent: Wednesday, October 05, 2005 4:21 PM
>>     >         >         > To: Paul Bijnens
>>     >         >         > Subject: Re: no space left on device during
>>     >
>>     >         amflush
>>     >
>>     >         >         > > Shang-Lin Chen wrote:
>>     >         >         > > >   taper: FATAL shmget: (655440 tapebufs)
>>     >         >         > > > No
>>     >
>>     >         space left
>>     >
>>     >         >         on device
>>     >         >
>>     >         >         > > The default value for tapebufs is 20
>>
>>     (buffers of
>>
>>     >         32K bytes
>>     >
>>     >         >         each).
>>     >         >
>>     >         >         > > You probably got some weird value in
>>
>>     amanda.conf
>>
>>     >         for that
>>     >
>>     >         >         parameter.
>>     >         >
>>     >         >         > > The above value results if over 20 GByte
>>
>>     of RAM.
>>
>>     >         >         > My amanda.conf file doesn't set anything for
>>     >
>>     >         tapebufs, and
>>     >
>>     >         >         I've never
>>     >         >
>>     >         >         > had this problem before. How do I change the
>>     >
>>     >         tapebufs
>>     >
>>     >         >         setting?
>>     >         >
>>     >         >         > Thanks,
>>     >         >         > Shang-Lin

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.35% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.


<Prev in Thread] Current Thread [Next in Thread>