ADSM-L

Maximum database size

1995-09-19 11:53:19
Subject: Maximum database size
From: Andy Raibeck <raibeck AT TIAC DOT NET>
Date: Tue, 19 Sep 1995 11:53:19 -0400
I know this is of no consolation to you, but perhaps others can benefit. There
is a PTF to the V1 server that provides for a synchronous backup of the ADSM
database, eliminating the need to perform an audit after the reload. The
command is:

     DUMP DB DEV=<deviceclass>

where <deviceclass> is a device class you created for dumping the ADSM database.
I have one called "DUMPDB" that retains tapes for 14 days (I have an MVS
server).

This command can be issued from the command line Admin.



Attached note follows:



On Mon, 28 Aug 1995, Greg Tevis wrote:

> The ADSM database scales fairly well...theoretically there are almost no
> limitations to # clients, # files, and size of database.  However, very
> large databases will impact performance.  Degradation seen will largely
> be a function of server CPU/IO capacity and number of database entries.
> We are working on some enhancements to ADSM V2 that will improve
> the upward scalability of the ADSM database.  These enhancements (most
> of them are internal) will probably come out through service and
> should improve performance for large databases such as you might
> see with 10s of thousands of clients.
> With 35,000 servers...you're probably going to want to have
> multiple servers.

Another reason for multiple servers is the time to repair a currupted
database: we are just now doing an auditdb run after a database reload.
The run is underway for two full weeks now and it will probably go on for
a while (if we simply extrapolate the progress we have to reckon with
several years). Needless to say that our customers are uttering curses
because they have no access to the archived data and that we feel somewhat
unhappy, to say the least, that we cannot take new backups. - If we were a
commercial site, we would quite probably be bankrupt now.

The time taken by the audit seems to be dependent not so much on the
number of clients or the amount of data but on the number of database
entries.  Currently we have 40 mio of them (why so many for only 5 mio
files?) and this although we are not yet in full production due to the
extreme instability of the system.

The database is on RAID disks, so this should not be the problem. RAID
does not help against problems introduced by software crashes.

Increasing the number of servers as a remedy, however, is loaded with
problems, too; this is why we didn't consider it in the first place:

For the automated tape libraries, each drive can be used by only one
  server at any given time. Three drives per server is probably a minimum,
  so that you get a fair number of drives if you split servers so that
  each of them contains only as few database entries as ADSM can reasonably
  handle, even in case a database audit needs to be done. This will soon get
  expensive.

The files backed up form a uniform file space. We do not know any
  reasonable way to specify a large number of default servers so that
  each user will automatically get his or her default server. Without
  the server name being a default, we cannot use the password generation
  mechanism.

I wonder what the experiences of other sites with many files are. Have
they seen faster database recoveries or do they rely on the impossibility
that they might ever become necessary?

I am aware that version 2 of ADSM has another method for ensuring
database consistency. This is hardly a consolation for our customers
because

they are now losing gigabytes of data they had confidently handed over
  to ADSM version 1, and I cannot tell them "you should have waited for
  version 2 before you store data in ADSM" and

they have lost any confidence in the product, and a new version with
  new algorithms is not a situation that would allow confidence to grow.

If you read some bitterness out of this letter, you read correctly. I was,
and despite the depressing experience of the last nine months I still am,
committed to bring ADSM to successful production at our site, but I see
this goal disappear at the horizon faster than we proceed.

An important criterion for preferring ADSM over competing products was
that we expected a professional product from a company that has experience
in professional data mangement and that provides professional support. The
present impression is not as if these expectations could be met with ADSM.

Best regards,

Helmut Richter

 ============================================================================
Dr. Helmut Richter
Leibniz-Rechenzentrum     X.400:  S=Richter;OU=lrz;P=lrz-muenchen;A=d400;C=de
Barer Str. 21            RFC822:  Helmut.Richter AT lrz-muenchen DOT de
D-80333 Muenchen           Tel.:  ++49-89-2105-8785
Germany                     Fax:  ++49-89-2809460
 ============================================================================
<Prev in Thread] Current Thread [Next in Thread>
  • Maximum database size, Andy Raibeck <=