Using the DBI::TSM module, memory problem
2006-12-04 09:21:32
Hi,
anybody else using the perl module DBI::TSM?
I've recently been using it for reporting purposes and find it quite useful.
However I have some problems with the perl memory consumption. This may very
well be caused by my lack of knowledge of DBI, but I haven't found a remedy yet.
The problem is that I want to process the output of sql statements line by line
as it is returned by the TSM-server, but it seems that DBI requires the output
of the whole sql to be stored in memory before I can process it. For large
queries the memory consumption will be huge.
Example(I know this is a simple one where it could be all handled in the sql
itself, this is just for explaining my problem):
# Set up the database handle:
$dbh=DBI->connect("DBI::TSM:myserver", 'admin', 'password¨');
# SQL that will return a zillion rows
$sql='select description as "DESC" from archives where node_name='MYNODE'';
$sth=$dbh->prepare($sql);
$sth->execute; #This is the one that slurps my memory
while($row_ref)=$sth->fetchrow_hashref){
#this processes each row that is already stored in memory
$archive_names{$row_ref->{'DESC'}}='';
}
I guess I could avoid the problem by using "dsmadmc" directly and write the
output to a file that i could later have perl parse, but that would not be a
very elegant solution. I want it to work with DBI.
Any ideas?
Best regards,
Hans Chr. Riksheim
<html><body><br /><br /><br /><font face="Arial" size="1"><hr>This email
originates from Steria AS, Biskop Gunnerus' gate 14a, N-0051 OSLO,
http://www.steria.no. This email and any attachments may contain
confidential/intellectual property/copyright information and is only for the
use of the addressee(s). You are prohibited from copying, forwarding,
disclosing, saving or otherwise using it in any way if you are not the
addressee(s) or responsible for delivery. If you receive this email by mistake,
please advise the sender and cancel it immediately. Steria may monitor the
content of emails within its network to ensure compliance with its policies and
procedures. Any email is susceptible to alteration and its integrity cannot be
assured. Steria shall not be liable if the message is altered, modified,
falsified, or even edited.</font></body></html>
|
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
- Using the DBI::TSM module, memory problem,
Hans Christian Riksheim <=
|
|
|