Amanda-Users

Re: Backing up filesystems too big for one tape

2003-09-05 10:08:04
Subject: Re: Backing up filesystems too big for one tape
From: Jon LaBadie <jon AT jgcomp DOT com>
To: AMANDA Users <amanda-users AT amanda DOT org>
Date: Fri, 5 Sep 2003 10:06:34 -0400
On Fri, Sep 05, 2003 at 09:47:26AM +0200, Paul Bijnens wrote:
> rwk AT americom DOT com wrote:
> 
> >>The basic way is to use gnutar as the backup program and NOT back up
> >>the entire FS, but create a series of disklist entries which back up
> >>a sub-tree of the total system.  For example if your big FS mounts as
> >>/big_mama, and has subdirs of /big_mama/proj1, /big_mama/proj2, ...
> >>you could specifiy a separate DLE for each "projx" directory tree.
> >
> >
> >It's videos and they are 1G to 5G each all in one directory.  The
> >program which creates them (mythtv) gives them files names which are
> >numbers, so they are not as easy to divide into categories like you
> >suggest, but I'm sure there is a way.
> 

In fact it would be possible to do your 'includes' like a hash table.
It becomes a lot of DLE's, and incrementals would be silly since they
are individual files.

Using Paul's examples below it would be some thing like:


fatboy  /bigmama_00  /bigmama {
        nocomp-user-tar
        include "./00*"
        }
fatboy  /bigmama_01 /bigmama {
        nocomp-user-tar
        include "./01*"
        }

For 100 total DLE's to 'include "./99*"'.

Or if 50 DLE's would be more desireable:


fatboy  /bigmama_1 /bigmama {
        nocomp-user-tar
        include "./0[01]*"
        }
fatboy  /bigmama_2 /bigmama {
        nocomp-user-tar
        include "./0[23]*"
        }

until you get to 'include "./9[89]"'.

And maybe a catchall extra one with 'include "./[!0-9]*"'
or 'exclude "./[0-9]"'.

Those are cumbersome, but with a good editor (vi comes to mind :)
it should be 3 minutes work.



> There is a possible approach using GNUTAR and the "include" directive
> together with explicit diskname + diskdevice in your disklist.
> 
> Your disklist would look like:
> 
> fatboy  /bigmama_BIGDIR  /bigmama {   # a big subdirectory
>       comp-user-tar
>       include "./bigdir"
>       }
> fatboy  /bigmama_FILES01 /bigmama {   # all files beginning with...
>       nocomp-user-tar
>       include "./file[01]*"
>       }
> fatboy  /bigmama_FILES23 /bigmama {
>       nocomp-user-tar
>       include "./file[23]*"
>       }
> ...
> fatboy  /bigmama_REST /bigmama {      # Catch-all
>       nocomp-user-tar
>       exclude "./file[0-9]*"
>       exclude append "./bigdir"
>       }
> 
> The disknames e.g. /bigmama_FILES23 are just labels and need to be uniq.
> The trick here lies in finding the patterns that divide the filesystem
> in chunks that fit on one tape.  (I wonder what happens if the "include"
> pattern matches no files. Hopefully it just backup up a zero size 
> directory.)  Also notice the last catchall to backup everything you
> did not specify in the previous includes.
> 
> You would probably also increase the "maxdumps" parameter, assuming
> your machine is fast enough.
> 
> Not very elegant, but while waiting when someone finally manages to
> adapt amanda to span tapes...
> 
> 
> -- 
> Paul Bijnens, Xplanation                            Tel  +32 16 397.511
> Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
> http://www.xplanation.com/          email:  Paul.Bijnens AT xplanation DOT com
> ***********************************************************************
> * I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
> * quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
> * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
> * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
> * kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
> * ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
> ***********************************************************************
> 
> 
> 
>>> End of included message <<<

-- 
Jon H. LaBadie                  jon AT jgcomp DOT com
 JG Computing
 4455 Province Line Road        (609) 252-0159
 Princeton, NJ  08540-4322      (609) 683-7220 (fax)

<Prev in Thread] Current Thread [Next in Thread>