ADSM-L

very slow retrievals, change from v2.1

1998-05-12 19:12:06
Subject: very slow retrievals, change from v2.1
From: Lester Buck <buck AT CLEARLAKE.IBM DOT COM>
Date: Tue, 12 May 1998 18:12:06 -0500
We are developing an image archive product that has been built around
ADSM for file archive and retrieval.  Images can be stored and
retrieved in all sizes of groups from one file to hundreds.  In ADSM
v2.1, we noticed nothing particularly anomalous.  We just installed
v3.1.1.1 and a retrieve from an IBM 3995 optical library gives the
following behaviour:

========================
Node Name: TEST23
Session established with server ADSM: AIX-RS/6000
Server Version 3, Release 1, Level 0.0
Server date/time: 05/07/1998 10:17:50  Last access: 05/07/1998
10:16:48

ANS1899I ***** Examined     1,000 files *****
ANS1899I ***** Examined     2,000 files *****
ANS1899I ***** Examined     3,000 files *****
ANS1899I ***** Examined     4,000 files *****
ANS1899I ***** Examined     5,000 files *****
ANS1899I ***** Examined     6,000 files *****
ANS1899I ***** Examined     7,000 files *****
ANS1899I ***** Examined     8,000 files *****
ANS1899I ***** Examined     9,000 files *****
ANS1899I ***** Examined    10,000 files *****
ANS1899I ***** Examined    11,000 files *****
ANS1899I ***** Examined    12,000 files *****
ANS1899I ***** Examined    13,000 files *****
ANS1899I ***** Examined    14,000 files *****
** Interrupted **
ANS1114I Waiting for mount of offline media.
Retrieving         232,236
/dicom/dinbrit/data/574_67_3625/CT.1.2.840.12355._835277545.894550013.670898.3.1
3.dcm
[Done]

Retrieve processing finished.

Total number of objects retrieved:        1
Total number of objects failed:           0
Total number of bytes transferred:   226.89 KB
Data transfer time:                    0.10 sec
Network data transfer rate:        2,169.60 KB/sec
Aggregate data transfer rate:          0.71 KB/sec
Elapsed processing time:           00:05:16
============================


The "Examined" lines take several seconds each, resulting in retrieve
times of several minutes, and it will only get worse.  This is
different than the old behaviour (I could provide an example).

Five minutes to retrieve a single file is ridiculous, will become even
more ludicrous with hundreds of thousands of images.

The messages manual, GC35-0271-01, does not list ANS1899I, and neither
does the HTML documentation.


Here is what we have figured out:

To archive an image:

ADSM 2.1 just archived the datasets and maintains its directory
structure internally.
ADSM 3.1 archives the datasets and all parent directories in that
filesystem.

To retrieve an image:

ADSM 2.1 just goes through it's internal directory structure to find the
dataset.
ADSM 3.1 must look through all archived directories to find the one
        containing the dataset that you are asking for (this takes
        several minutes (about 1 minute per 5000 files archived)
        and gets worse as more datasets are archived).


This is a very serious issue for us.  Can anyone shed some light on
this behaviour in v3.1?  Is there any way to turn it off?


Thanks alot!


Lester
--
A. Lester Buck
A. Lester Buck
IBM Government Systems, Houston
buck AT clearlake.ibm DOT com
<Prev in Thread] Current Thread [Next in Thread>
  • very slow retrievals, change from v2.1, Lester Buck <=