ADSM-L

Re: [ADSM-L] Sources of discrepancy between FILE volume size on disk and reported space?

2012-01-04 08:08:41
Subject: Re: [ADSM-L] Sources of discrepancy between FILE volume size on disk and reported space?
From: "Underdown,John William" <JUNDERDOWN AT SYNOVUS DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Wed, 4 Jan 2012 13:01:41 +0000
i have the same issue with our document-imaging backups. this is due to 
zillions of itty-bitty files being backed up, see below. btw, the 
TXNGROUPMAX option only applies to files being backed up, not moved or 
reclaimed.

Volume 100% utilized, Estimated Capacity does not meet MAXCAPacity
Technote (FAQ)

Problem
The output from "q vol" on a FILE devclass volume in a TSM 5.3 Server is 
reporting 100% utilized, but the "Estimated Capacity" has not reached 
the MAXCAPacity.

Cause
For example, a FILE devclass named FILEDIR has been set to have a MAX 
Capacity of 64MB but each volume has only been filled to 34MB before 
becoming full:

Output from "q devclass":
Device Device Storage Device Format Est/Max Mount
Class Access Pool Type Capacity Limit
Name Strategy Count (MB)
--------- ---------- ------- --------- ------ -------- ------
FILEDIR Sequential 1 FILE DRIVE 64.0 20

After a backup of small objects, the output from "q vol" will show 
something like:
Volume Name Storage Device Estimated Pct Volume
Pool Name Class Name Capacity Util Status
------------------------ ----------- ---------- --------- ----- --------
C:\TSMDATA\SERVER1\DIRM- FILEDIR FILEDIR 34.0 100.0 Full
C\000001E6.BFS
C:\TSMDATA\SERVER1\DIRM- FILEDIR FILEDIR 34.0 100.0 Full
C\000001E7.BFS

There was a design change to the file devclass in 5.3 where each I/O to 
the a FILE devclass volume will be at least 256K, regardless of how much 
data is being written. Under these conditions, if there is only one 
object backed up that is 500 bytes in size, it will take 256k to store 
it on the FILE devclass volume. This will have the biggest impact when 
backing up small objects (for instance, small files or directories).

Solution
Increasing the aggregate size may help reduce this overhead. This can be 
done by modifying the value for the TXNGROUPMAX on either the TSM Server 
or for this particular node (register/update node). This will make the 
aggregate size larger and less space is wasted with the overhead. Also 
the client option TXNBYTELIMIT may need to be adjusted as well.

NOTE: Increasing the TXNGROUPMAX value will result in increased recovery 
log utilization. Higher recovery log utilization may increase the risk 
of running out of log space. Evaluate each node's performance before 
changing this parameter. For more information on managing the recovery 
log, see the Administrator's Guide.

The results from the query volume may vary by environment. Also check 
the OS to see if the filesize of the volume is equal to the MAXCAPacity. 
If it is not, then APAR IC44597 is also affecting the environment.


On 01/03/2012 02:58 PM, Allen S. Rout wrote:
> So, I've got a passel of document-imaging data whose primary storage is
> on FILE. As you might imagine, it's amazingly static, and as a
> consequence gets little attention. But it just got some, and I'm
> confused by it.
>
> I recently set about moving my TSM infrastructure from the Old Storage
> (emc cx320) to the New Storage (emc VNX). I built new volume groups
> and filesystems, updated my management class to only write to the new
> filesystems, and started a long series of MOVE DATA macros.
>
> As I watched these go by, I was interested by the fact that the move
> processes were reporting less than 5G moved, sometimes substantially. I
> figured I'd neglected to set reclaim thresholds on this data, d'oh. So
> I was getting an opportunity to reclaim the whole dang infrastructure.
> how nice! I started counting space savings before it was hatched.
>
>
> So now the move is done, and I've got my 4.mumble TB of data all moved
> to brand new 5G volumes, none of which existed a month ago, all of which
> were populated by
>
> MOVE DATA [foo] reconst=yes
>
> and none of which have any reclaimable space
>
> tsm: VI>select pct_utilized,pct_reclaim,count(*) from volumes where
> devclass_name='MED FILE' group by pct_utilized,pct_reclaim
>
> PCT_UTILIZED PCT_RECLAIM Unnamed[3]
> ------------ ----------- -----------
> 6.1 0.0 1
> 100.0 0.0 848
>
>
> but the ESTCAP of these volumes is pretty far off 5G. Here's a
> histogram-like list of them, counted by GB of ESTCAP.
>
>
> 1 1.3
> 1 1.5
> 2 1.6
> 4 1.7
> 8 1.8
> 16 1.9
> 20 2.0
> 39 2.1
> 37 2.2
> 44 2.3
> 35 2.4
> 53 2.5
> 51 2.6
> 52 2.7
> 40 2.8
> 60 2.9
> 56 3.0
> 44 3.1
> 46 3.2
> 61 3.3
> 152 3.4
> 14 3.5
> 6 3.6
> 3 3.7
> 1 3.9
> 2 4.0
> 1 5.0
>
>
> So, what am I missing? I feel like a bozo, but I can't come up with a
> reason a full 5G volume would have an ESTCAP of 1.3G with no reclaimable
> space.
>
>
> - Allen S. Rout

-----------------------------------------
NOTICE: This communication is intended only for the person or
entity to whom it is addressed and may contain confidential,
proprietary, and/or privileged material. Unless you are the
intended addressee, any review, reliance, dissemination,
distribution, copying or use whatsoever of this communication is
strictly prohibited. If you received this in error, please reply
immediately and delete the material from all computers. Email sent
through the Internet is not secure. Do not use email to send us
confidential information such as credit card numbers, PIN numbers,
passwords, Social Security Numbers, Account numbers, or other
important and confidential information.