ADSM-L

Re: Database size in Version 3

2015-10-04 18:03:48
Subject: Re: Database size in Version 3
From: Jay P. Weidner [SMTP:jpw AT VETMED.WSU DOT EDU]
To: ADSM-L AT VM.MARIST DOT EDU
I don't know your application that needs these 182 million files but it
would seem unlikely that it would need them all concurrently.  Would it
work to use tar to combine a 1000 or 100000 of these small files into one
file that could be accessed and untarred as necessary in the future?  We
have done this followed by a gzip of the tarred file when we have wanted
to keep the data semi-accessable.  We used scripts to tar and zip the
files at time of archive and have other scripts to unzip and extract
required files on demand either manually or preferrably by a menu driven
scripts within the application.

With one application I'm currently using this method and converting
850-900 files in a directory into one file.  The single file is 520 Mbytes
and compresses to 73.4 Mbytes.  The tar takes under 5 minutes but the gzip
can take over 30 minutes.  Since it occurs locally it doesn't affect my
net traffic and the subsequent backup of the single 73 Mb file decreases
net traffic substantially.  With your scenario if you could use a similar
strategy you could not only derease net traffic but also decrease your
ADSM database by a factor of 1000 or greater.

Jay P. Weidner
College Of Veterinary Medicine
Washington State University

On Tue, 11 Nov 1997, Julie Phinney wrote:

> We have a need to keep 100,000 new files from one client every day, 5
> days a week, for 7 years.  This is 182 million files.  By my calculation
> (the utilization I'm currently getting on the database) this would take
> 118 GB of ADSM database space. (This would include keeping one offsite
> copy).
> Does anyone know if the way ADSM stores small files with V3 is any
> different?  I know it can move them differently, with small file
> aggregation, but is the database structure any different?
> IBM... will I use just as much database with V3 as V2?
> THANKS!
> Julie Phinney
> JPHINNEY AT HUMANA DOT COM
<Prev in Thread] Current Thread [Next in Thread>