Bacula-users

Re: [Bacula-users] bacula-dir virtual memory limit during restore

2008-06-03 16:25:22
Subject: Re: [Bacula-users] bacula-dir virtual memory limit during restore
From: Arno Lehmann <al AT its-lehmann DOT de>
To: baculausers <bacula-users AT lists.sourceforge DOT net>
Date: Tue, 03 Jun 2008 22:25:02 +0200
Hi,

03.06.2008 16:52, John Kloss wrote:
> Hello,
> 
> I am currently running bacula-2.2.8 compiled as a 32bit binary.
> I am using postgresql-8.3.1 as the catalog, also compiled as a 32bit  
> binary.
> I am running bacula on solaris 9 running in 64bit mode which of  
> course means I can run both 64bit and 32bit binaries.
> 
> ulimit -v shows unlimited.  I know that's a lie and that soft limit  
> is 2GB.  I know that I can change that to 3.5GB or so for a 32bit  
> process.  I have done so and then started bacula-dir.
> ulimit -d shows unlimited.  I know that's also a lie and that the  
> default limit is 2GB.  I know that I change that to 3.5GB or so for a  
> 32bit process.  I have done so (along with ulimit -v) and then  
> started bacula-dir.

Good background information :-)

> I am trying to restore 2.5 terabytes of data composed of 6.5 million  
> files.
> 
> My process is
> 
> Run bconsole
> Choose restore
> Chose the most recent restore for a client
> Wait for the directory structure to be generated in memory (10  
> minutes tops-- postgres temp files are on a ram disk which makes life  
> fast)
>   Chose 'mark *'
> Watch bacula churn away for a couple of minutes and then report 6.5  
> million files marked.
> Type done.
> See that the prompt never returns.  The restore never happens.   
> Actually, I don't have time to wait for forever so I waited for 36  
> hours instead and saw that nothing had changed.  No prompt.  No restore.

Hmm... bad.

<snip observations>

...
> Previous versions of bacula (1.36) were able to restore 5.5 terabytes  
> of data composed of 9 million files via the above method.  Same  
> machine, less memory, 32bit binaries, old version of postgres (8.0).   
> The new version as I have compiled and configured cannot.
> 
> How does one recover 2.5 terabytes and 6.5 million files using the  
> latest version of bacula?

Well, I never had to restore such an number of files. I'd recommend to 
  try a command like 'restore all jobid=<id1,id2,id3,...,idn> done' 
(or, if I'm wrong and that doesn't do what I expect it to, use 
'restore all' and choose selection 3) first. If that does not work, 
you could try to generate the necessary bootstrap file manually and 
run that. Generating the bootstrap file might be a bit of work, though.

In any case I'd recommend to report your problem to the -devel mailing 
list or at bugs.bacula.org.

Especially if you can help debugging this sort of problem, I suppose 
Kern will be very interested in looking at the problem. I don't know 
if it's easily fixed, though - I suspect it will be not.

>  What am I doing wrong?  Is there anyway to  
> change smartalloc so that it will use 3.5GBs of memory (nothing  
> popped out at me when I scanned the include files)?

I have no idea if smartalloc is responsible for this issue, but Kern 
might know more.

> I should note that a couple of weeks ago I had a complete system  
> failure of my SAN and lost 25 terabytes of data.  Bacula 1.36  
> restored all of it.  I got to keep my job.  Thank you bacula.

Great... another success story :-) (losing 25 TB... I guess that made 
you sweat.)

Arno

>       John Kloss. <John.Kloss AT jhmi DOT edu>
>       IT Manager, Systems Manager
>       Institute of Genetic Medicine
>       Johns Hopkins Medical Institution
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

-- 
Arno Lehmann
IT-Service Lehmann
www.its-lehmann.de

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users