Amanda-Users

Re: One DLE too many!

2006-03-10 14:40:01
Subject: Re: One DLE too many!
From: Paul Bijnens <paul.bijnens AT xplanation DOT com>
To: Vytas Janusauskas <vytas AT dendron DOT com>
Date: Fri, 10 Mar 2006 20:37:02 +0100
Vytas Janusauskas schreef:
Hi All,

I have been using Amanda (current version 2.4.4p3) for the last couple of years without any major issues. However recently due to an influx of new projects the volume of data has grown. What is happening is that users are adding new directories to the data servers and not notified me. So all of a sudden some of my DLEs which have been merrily archiving have started to exceed my available tape capacity. I have toyed with the idea of using tar excludes to address this issue, unless I am missing something would I have to create a separate DumpType for each DLE?

You can use "inline" dumptypes (with brace-newline-dumptypedefinition-newline-endbrace),
and explicitly make a the DLE-name different from the directory to
differentiate the different includes:

client1.example.com  /data06/Deforest3_PART1 /data06/Deforest3 {
      comp-user-tar
      include "./Air*"
      include append "./Nova?Scotia?Photos"
      }  1



For better illustrate the issue for the following DLE they added the 40G Nova Scotia Photos directory. I was planning on breaking up the DLE listed below into 8 individual entries but that may lead to problems of handling the directories with whitespaces in their names. If I was just dealing with a few DLEs separating them into smaller chunks would not be an issue, however I have already fragmented my data into 240 + DLEs.

There is currently a limitation that you cannot have white space
in the strings specified for include/exclude.
But you can get around it by using wildcards as I did above.



hal /mnt/data06/Deforest3 comp-user-tar 87

hal:/mnt/data06# du -h --max-depth=1 Deforest3
248K    Deforest3/AirPhoto Load Tool
40G     Deforest3/Nova Scotia Photos
2.7G    Deforest3/delivery
1.8G    Deforest3/NS_Oblique_Aerials
6.4G    Deforest3/Imagery
17M     Deforest3/Received
1.6M    Deforest3/Quality Control
28G     Deforest3/NAPL
78G     Deforest3

Any suggestion of how best to deal with this situation is greatly appreciated.

When using the "include" trick above to subdivide a directory,
you better also add one "REST" DLE, having only exclude statements
of all the includes string. This catches all the filenames and
directories that would be created after you specified the list.

BTW, 240 DLE's is in danger to bump into the max UDP packet size for
estimates, which currently is 64K / 2 or about 32Kbytes...


--
Paul Bijnens, Xplanation                            Tel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
http://www.xplanation.com/          email:  Paul.Bijnens AT xplanation DOT com
***********************************************************************
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
***********************************************************************

<Prev in Thread] Current Thread [Next in Thread>