ADSM-L

Using the DBI::TSM module, memory problem

2006-12-04 09:21:32
Subject: Using the DBI::TSM module, memory problem
From: Hans Christian Riksheim <HCR AT STERIA DOT NO>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 4 Dec 2006 15:21:07 +0100
Hi,

anybody else using the perl module DBI::TSM?

I've recently been using it for reporting purposes and find it quite useful. 
However I have some problems with the perl memory consumption. This may very 
well be caused by my lack of knowledge of DBI, but I haven't found a remedy yet.

The problem is that I want to process the output of sql statements line by line 
as it is returned by the TSM-server, but it seems that DBI requires the output 
of the whole sql to be stored in memory before I can process it. For large 
queries the memory consumption will be huge.

Example(I know this is a simple one where it could be all handled in the sql 
itself, this is just for explaining my problem):


# Set up the database handle:
$dbh=DBI->connect("DBI::TSM:myserver", 'admin', 'password¨');

# SQL that will return a zillion rows
$sql='select description as "DESC" from archives where node_name='MYNODE'';

$sth=$dbh->prepare($sql);

$sth->execute;          #This is the one that slurps my memory

while($row_ref)=$sth->fetchrow_hashref){
  #this processes each row that is already stored in memory
  $archive_names{$row_ref->{'DESC'}}='';
}


I guess I could avoid the problem by using "dsmadmc" directly and write the 
output to a file that i could later have perl parse, but that would not be a 
very elegant solution. I want it to work with DBI.


Any ideas?


Best regards,

Hans Chr. Riksheim






<html><body><br /><br /><br /><font face="Arial" size="1"><hr>This email 
originates from Steria AS, Biskop Gunnerus' gate 14a, N-0051 OSLO, 
http://www.steria.no. This email and any attachments may contain 
confidential/intellectual property/copyright information and is only for the 
use of the addressee(s). You are prohibited from copying, forwarding, 
disclosing, saving or otherwise using it in any way if you are not the 
addressee(s) or responsible for delivery. If you receive this email by mistake, 
please advise the sender and cancel it immediately. Steria may monitor the 
content of emails within its network to ensure compliance with its policies and 
procedures. Any email is susceptible to alteration and its integrity cannot be 
assured. Steria shall not be liable if the message is altered, modified, 
falsified, or even edited.</font></body></html>

<Prev in Thread] Current Thread [Next in Thread>