Amanda-Users

Proposal v2: quota limit on backup clients and groupware (Hope to be added on the wishlist)

2003-04-22 13:35:39
Subject: Proposal v2: quota limit on backup clients and groupware (Hope to be added on the wishlist)
From: Stefano Coletta <creator AT mindcreations DOT com>
To: amanda-users AT amanda DOT org
Date: Tue, 22 Apr 2003 17:16:49 +0200
Hello everybody,

this is a release 2 of a previous message I've posted to the list on the same argument. I've taken my time to do proper evaluations and so I decided to update the text of the proposal
because I really think that these features are useful for the many.

I'm setting up Amanda to work as the global ISP solution for customers having to backup hosting/housing/spaces/servers. I've conviced management that this software can be a great solution avoiding to purchase expensive softwares like Arcserve, Legato, Tivoli ecc.
Now, while implementing it, I had to deal with a limit of Amanda.
Customers have to pay to be backed up and thus I have to control the size of every backup client (aka disk).

In my case the customer is supposed to pay for a "tape quota"... for example, 1Gb space for N servers (no matter how many servers the customer owns). I need a mechanism to control if the overall storage of customers' servers are hitting/exceeding the quota they have paid.

The 2 goals to achieve are:
1) to be informed about over quota events in relation to every globally purchased customer quota and even to notify the customer 2) to effectively deny the backup if the client has exceeded his quota after he has been warned several times
3) to inform technical staff about imminent tape space shortage

I propose two different approaches to partially/definitely solve the problem:


1st solution: more easy, the partial one
----------------------------------------

Add new parameters in the dumptypes declaration section of the amanda.conf file.

Example:

define dumptype customername {
   global
   program "GNUTAR"
   comment "root partitions dumped with tar"
   compress none
   index
   exclude list "/usr/local/lib/amanda/exclude.gtar"
   priority low

# -- BEGINNING OF NEW PARAMETERS

   quota 1Gb
   quotaexceed 7
   quotadeny 1

# -- END OF NEW PARAMETERS
}


In this way you can limit Amanda from doing backups larger than 1Gb for that dumptype that corresponds for example to "customername".

NOTE: with the word "limit" I mean not to truncate the backup but effectively avoiding to do it.

"quotaexceed" may be the maximum number of times that Amanda does the backup before refusing to do it (ie. 7 days). Obviously, during the amcheck you have to be notified of this so you can contact your customer to extend the quota or just making him to reduce the amount of data to backup. If the parameter "quotadeny" is set to 1 amanda has to deny the backup if you are over quota after the "quotaexceed" times.
I think that these new parameters can even be useful to do other things.

There may be even other ways to give a name to these parameters. For example like the hard & soft quota naming convention used in many OSs.

This solution does not solve in an elegant way the original problem to check for ALL customer server(s) because it controls only a dumptype that is assigned to a particular server (aka disk) and not customer.


2nd solution: the more complex but definitive
---------------------------------------------

Add support to Amanda to handle a cluster of disks to make them part of a group. The parameters detailed in the 1st solution could be applied to the entire group affecting all the disks.
In my scenario that group could be considered as the customer.

Grouping can be implemented like:

define group groupname {
diskname1
diskname2
...
disknameN

quota 1Gb
quotaexceed 7
quotadeny 1
}

inserting this text in the disklist file or in other places.


Additional handy feature
------------------------

To achieve an easy management of this new functionality it may be useful to add a new parameter in the group definition to alert who as to be. For example:

define group security_department {
gatekeeper
snortsrv
syslogger

quota 5Gb
quotaexceed 7
quotadeny 1

# -- BEGINNING OF NEW PARAMETER

email securteam AT company DOT com

# -- END OF NEW PARAMETER
}

With this new parameter additional folks can be alerted of this situation before the backup is denied. You will achieve a minimum of delegation in the entire backup process to avoid wasting space while backupping unneeded data that grows in the wild. Alternatively the responsible for that group can contact the backup administrator claiming more space.

----

These additions can also contribute to enhance the priority/security of the backup strategy even if they are not used for commercial scenarios.

Examples:

1) You can put more quota for some servers to avoid wasting tape space for others that may prevent regular backup of critical services. 2) You can be warned if a server (or a group) requires too much space of what you have planned on your "space strategy". 3) You can effectively control how much tape space your servers are consuming without continuosly checking at log files and reports. Grouping can help to identify departments for example.

--------

I'm suggesting an addition to the software core of amanda and not a wrapper script. With a wrapper you can easily implement the email notice but you can't do well everything I've described.

Waiting for your precious replies. :)

Bye all

Stefano Coletta
creator AT mindcreations DOT com

http://www.mindcreations.com